以下代码未按预期工作:
import requests
from bs4 import BeautifulSoup
url = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/einfo.fcgi?retmode=xml&db=pmc'
response = requests.get(url)
soup = BeautifulSoup(response.text,'lxml')
links = soup.find_all('link')
links
结果产生了一堆`标签(只显示了一些):
<link/>
<name>pmc_sra</name>
<menu>SRA</menu>
<description>Links to SRA</description>
<dbto>sra</dbto>
<link/>
<name>pmc_structure</name>
<menu>Structure Links</menu>
<description>Published 3D structures</description>
<dbto>structure</dbto>
<link/>
打印response.text
显示(仅显示部分打印):
<Link>
<Name>pmc_sra</Name>
<Menu>SRA</Menu>
<Description>Links to SRA</Description>
<DbTo>sra</DbTo>
</Link>
<Link>
<Name>pmc_structure</Name>
<Menu>Structure Links</Menu>
<Description>Published 3D structures</Description>
<DbTo>structure</DbTo>
</Link>
<Link>
重要的是,每个Link
标签都包含其他标签,而漂亮汤建议链接标签是独立的。
如果我直接尝试lxml
,我会得到正确的链接标记:
from lxml import etree
#root = etree.fromstring(response.text)
root = etree.fromstring(response.text.encode('utf-8'),parser=etree.XMLParser(encoding='utf-8'))
for link in root.iter("Link"):
etree.dump(link)
产生:
<Link>
<Name>pmc_sra</Name>
<Menu>SRA</Menu>
<Description>Links to SRA</Description>
<DbTo>sra</DbTo>
</Link>
<Link>
<Name>pmc_structure</Name>
<Menu>Structure Links</Menu>
<Description>Published 3D structures</Description>
<DbTo>structure</DbTo>
</Link>
注意,我在使用更简单的etree.fromstring
调用时遇到了一个错误。也许BeautifulSoup的问题是编码问题?
我使用的是Python3.7.6中最新的BeautifulSoup (4.8.2)和LXML (4.5.0
发布于 2020-02-08 14:46:52
https://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-a-parser
我认为问题出在使用"lxml“而不是"lxml-xml”。如果我没记错,Beautiful Soup正试图将您的XML解析为HTML,并因此破坏数据。
https://stackoverflow.com/questions/60127851
复制相似问题