如何在python请求库中实现重试机制？
如何在python请求库中实现重试机制？
提问于 2014-04-24 11:19:06
我想在python请求库中添加一个重试机制，这样使用它的脚本就会重试非致命错误。
此时此刻，我确实认为有三种错误是可以恢复的：
- HTTP返回代码502,503,504
- 找不到主机(现在不太重要了)
- 请求超时
在第一阶段，我确实希望每分钟重试指定的5xx请求。
我希望能够透明地添加此功能，而不必手动实现从这些脚本或使用python请求的库中发出的每个HTTP调用的恢复。
回答 6
Stack Overflow用户
回答已采纳
发布于 2014-04-24 14:11:20
通过扩展requests.Session类，我能够获得所需的可靠性水平。
编辑的代码是：
from requests import Session
from requests.exceptions import ConnectionError
import logging
import time
class ResilientSession(Session):
"""
This class is supposed to retry requests that do return temporary errors.
At this moment it supports: 502, 503, 504
"""
def __recoverable(self, error, url, request, counter=1):
if hasattr(error,'status_code'):
if error.status_code in [502, 503, 504]:
error = "HTTP %s" % error.status_code
else:
return False
DELAY = 10 * counter
logging.warn("Got recoverable error [%s] from %s %s, retry #%s in %ss" % (error, request, url, counter, DELAY))
time.sleep(DELAY)
return True
def get(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).get(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'GET', counter):
continue
return r
def post(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).post(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'POST', counter):
continue
return r
def delete(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).delete(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'DELETE', counter):
continue
return r
def put(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).put(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'PUT', counter):
continue
return r
def head(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).head(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'HEAD', counter):
continue
return r
def patch(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).patch(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'PATCH', counter):
continue
return r
def options(self, url, **kwargs):
counter = 0
while True:
counter += 1
try:
r = super(ResilientSession, self).options(url, **kwargs)
except ConnectionError as e:
r = e.message
if self.__recoverable(r, url, 'OPTIONS', counter):
continue
return rStack Overflow用户
发布于 2016-02-25 19:19:26
这段代码片段将使来自同一会话的所有HTTP请求总共重试5次，在重试之间休眠，后退的次数增加了0、2、4s、8s、16s (第一次重试立即完成)。它将重新尝试基本的连接问题(包括DNS查找失败)，以及502、503和504的HTTP状态代码。
import logging
import requests
from requests.adapters import HTTPAdapter, Retry
logging.basicConfig(level=logging.DEBUG)
s = requests.Session()
retries = Retry(total=5, backoff_factor=1, status_forcelist=[ 502, 503, 504 ])
s.mount('http://', HTTPAdapter(max_retries=retries))
s.get("http://httpstat.us/503")详情请参见重试课。
Stack Overflow用户
发布于 2014-04-24 11:24:12
这是我用来重试用urllib2编写的请愿书的代码片段。也许你可以用它来达到你的目的：
retries = 1
success = False
while not success:
try:
response = urllib2.urlopen(request)
success = True
except Exception as e:
wait = retries * 30;
print 'Error! Waiting %s secs and re-trying...' % wait
sys.stdout.flush()
time.sleep(wait)
retries += 1等待时间递增，以避免从服务器上被禁止。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/23267409
复制相关文章
相似问题
腾讯云开发者
