python使用requests时报错requests.exceptions.SSLError: HTTPSConnectionPool

报错信息

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 640, in send
history = [resp for resp in gen] if allow_redirects else []
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 218, in resolve_redirects
**adapter_kwargs
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.baidu.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)'),))

过程

测试1

不指定headers时GET:

>>> import requests
>>> requests.get('http://www.baidu.com/')
<Response [200]>
>>> requests.get('http://www.baidu.com/')
<Response [200]>
>>> requests.get('http://www.baidu.com/')
<Response [200]>
>>> header = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1',}
>>> requests.get('http://www.baidu.com/', headers = header)
<Response [200]>

测试2

当指定headers的User-Agent为火狐浏览器时:

>>> header = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1',}
>>> requests.get('http://www.baidu.com/', headers = header)
<Response [200]>
>>> requests.get('http://www.baidu.com/', headers = header)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 640, in send
history = [resp for resp in gen] if allow_redirects else []
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 218, in resolve_redirects
**adapter_kwargs
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "D:\python\lib\site-packages\requests-2.18.3-py2.7.egg\requests\adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.baidu.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)'),))

分析

现象:第一次GET时正常,第二次GET时,会报错.

不同点:User-Agent不相同

分析:由于报错SSL证书验证失败,所以这次的访问应该是https协议.但是我们明明使用的是http,所以,猜测访问该网站后,被重定向到了https://www.baidu.com/

验证

首先,进行GET时,关闭证书验证.因为,如果不关闭,请求总是失败,不能获取到重定向的信息.

>>> response = requests.get('http://www.baidu.com/', headers = header, verify=False)
D:\python\lib\site-packages\urllib3\connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
>>> response.history
[<Response [302]>]
>>> response.url
u'https://www.baidu.com/'

当不指定User-Agent时

>>> response = requests.get('http://www.baidu.com/', verify=False)
>>> response.history
[]
>>> response.url
u'http://www.baidu.com/'

结论

当指定headers的User-Agent时,baidu的服务器会重定向到https的网址.因此报出SSL验证失败的错误.

解决方法

方法1:

在进行GET时,指定SSL证书.详情见附件

方法2:

关闭证书验证. 详情见附件

附件

[各浏览器的User-Agent] http://www.useragentstring.com/pages/useragentstring.php

[SSL 证书验证] http://docs.python-requests.org/zh_CN/latest/user/advanced.html#ssl

 转自:https://blog.csdn.net/win_turn/article/details/77142100

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏java相关

OmniGraffler软件和激活码

13930
来自专栏java达人

Ajax详解

image.png 一:什么是Ajax Ajax是(Asynchronous JavaScript And XML)是异步的JavaScript和xml。也...

22790
来自专栏比原链

剥开比原看代码10:比原是如何通过/create-key接口创建密钥的

Gitee地址:https://gitee.com/BytomBlockchain/bytom

13720
来自专栏极客慕白的成长之路

光棍节程序员闯关秀过关攻略

查看源代码,发现a标签的颜色和背景色一样,导致了我们看不到a过关地址,方法很简单,直接复制源代码中的key即可,或者ctrl+a然后单击进入下一关即可

15850
来自专栏Hadoop实操

如何为HttpFS服务配置SSL

前面Fayson也介绍过《如何在集群外节点跨网段向HDFS写数据》和《如何使用Java代码访问HDFS》。在非Kerberos的环境中,使用HttpFS是不需要...

47460
来自专栏熊二哥

快速入门系列--WebAPI--01基础

ASP.NET MVC和WebAPI已经是.NET Web部分的主流,刚开始时两个公用同一个管道,之后为了更加的轻量化(WebAPI是对WCF Restful的...

56470
来自专栏酷玩时刻

支付宝Wap支付你了解多少?

为了方便开发者生成一对RSA密钥支付宝提供一键生成工具,具体如何生成与配置密钥详见签名专区。

43620
来自专栏Kubernetes

Kubelet Deivce Manager源码分析

43250
来自专栏LanceToBigData

Hadoop(十)Hadoop IO之数据完整性

前言   上一篇我分享了Hadoop的压缩和编解码器,在我们开发的过程中其实是经常会用到的,所以一定要去掌握。这一篇给大家介绍的是Hadoop的数据完整性!  ...

24150
来自专栏编程坑太多

requests库使用方式

15740

扫码关注云+社区

领取腾讯云代金券