我想用bs4从亚马逊获得产品的标题
我想用bs4从亚马逊获得产品的标题
提问于 2020-08-22 09:43:14
我想通过BeuatifulSoup和请求获得这个亚马逊产品的标题。当我运行这句话时：
Traceback (most recent call last):
File "scraper.py", line 15, in <module>
title = soup.find('span', id='productTitle').get_text()
AttributeError: 'NoneType' object has no attribute 'get_text'请帮帮我
import bs4
import requests
from bs4 import BeautifulSoup
from urllib.request import urlopen
url = 'https://www.amazon.de/OnePlus-Smartphone-Almond-Display-Speicher/dp/B07RWL3K1Y/ref=sr_1_2? __mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=oneplus+7+pro&qid=1598088298&sr=8-2'
headers = {
"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36'
}
page = requests.get(url, headers = headers)
soup = BeautifulSoup(page.content, 'html.parser')
title = soup.find('span', id='productTitle').get_text()
print(title)回答 1
Stack Overflow用户
回答已采纳
发布于 2020-08-22 09:59:46
问题是使用'html.parser‘作为您的bs4解析器。改用lxml (它将更优雅地处理破碎的html )。这个错误试图告诉您，它从未找到<span id='productTitle'> --我们可以看到它在那里，所以这可能是一个与非标准HTML相关的解析失败。
import requests
from bs4 import BeautifulSoup
url = 'https://www.amazon.de/OnePlus-Smartphone-Almond-Display-Speicher/dp/B07RWL3K1Y/ref=sr_1_2? __mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=oneplus+7+pro&qid=1598088298&sr=8-2'
headers = {
"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36'
}
page = requests.get(url, headers = headers)
soup = BeautifulSoup(page.content, 'lxml')
title = soup.find('span', id='productTitle').get_text().strip()
print(title)输出：
OnePlus 7 Pro Smartphone Almond (16,9 cm) AMOLED Display 8 GB RAM + 256 GB Speicher, Triple Kamera (48 MP) Pop-up Kamera (16 MP) – Dual SIM Handy页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/63534767
复制相关文章
相似问题
腾讯云开发者
