获取特定作者的所有href
获取特定作者的所有href
提问于 2020-03-15 02:29:50
您好，我正在尝试获取所有/pubmed/数字，这些数字将我链接到来自特定作者的文章摘要。问题是，当我尝试这样做时，我只能一次又一次地获得相同的数字，直到for循环结束。
我试图获取的href应该取自for line in lines循环的输出(具体的href在输出示例中)。这个循环似乎工作得很好，但是for abstract in abstracts循环只重复相同的href。
任何建议或想法我遗漏了什么或做错了什么。我没有太多使用bs4的经验，所以我可能没有很好地使用这个库。
#Obtain all the papers of a scientific author and write its abstract in a new file
from bs4 import BeautifulSoup
import re
import requests
url="https://www.ncbi.nlm.nih.gov/pubmed/?term=valvano"
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')
lines = soup.find_all("div",{"class": "rslt"})
authors= soup.find_all("p",{"class": "desc"})
scientist=[]
for author in authors:
#print('\n', author.text)
scientist.append(author.text)
s=[]
for i in scientist:
L=i.split(',')
s.append(L)
n=0
for line in lines:
if ' Valvano MA' in s[n] or 'Valvano MA' in s[n] :
print('\n',line.text)
#part of one output:
<a **href="/pubmed/32146294"** ...
found = soup.find("a",{"class": "status_icon nohighlight"})
web_abstract='https://www.ncbi.nlm.nih.gov{}'.format(found['href'])
response0 = requests.get(web_abstract)
sopa = BeautifulSoup(response0.content, 'lxml')
abstracts = sopa.find("div",{"class": "abstr"})
for abstract in abstracts:
#print (abstract.text)
print('https://www.ncbi.nlm.nih.gov{}'.format(found['href']))
#output:
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31919170
n=n+1
else:
n=n+1
#expected output:
https://www.ncbi.nlm.nih.gov/pubmed/32146294
https://www.ncbi.nlm.nih.gov/pubmed/32064693
https://www.ncbi.nlm.nih.gov/pubmed/31978399
https://www.ncbi.nlm.nih.gov/pubmed/31919170
https://www.ncbi.nlm.nih.gov/pubmed/31896348
https://www.ncbi.nlm.nih.gov/pubmed/31866961
https://www.ncbi.nlm.nih.gov/pubmed/31722994
https://www.ncbi.nlm.nih.gov/pubmed/31350337
https://www.ncbi.nlm.nih.gov/pubmed/31332863
https://www.ncbi.nlm.nih.gov/pubmed/31233657
https://www.ncbi.nlm.nih.gov/pubmed/31133642
https://www.ncbi.nlm.nih.gov/pubmed/30913267回答 2
Stack Overflow用户
回答已采纳
发布于 2020-03-15 04:34:52
它不需要那么复杂。只需使用：
ids = soup.select('dt + dd')
for i in ids:
pmid = i.text
print(f'https://www.ncbi.nlm.nih.gov/pubmed/{pmid}')Stack Overflow用户
发布于 2020-03-15 04:34:15
如果使用URL返回正确的结果，那么可以使用下面的正则表达式示例。
from bs4 import BeautifulSoup
import re
import requests
url = "https://www.ncbi.nlm.nih.gov/pubmed/?term=valvano+MA"
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')
for a in soup.select('div.rprt p a'):
if re.match('^/pubmed/[0-9]*$', a['href']) is not None:
print('https://www.ncbi.nlm.nih.gov{}'.format(a['href']))这将得到所有20个结果加上结果17的勘误值。
if re.match('^/pubmed/[0-9]*$', a['href']) is not None and a.get('ref') is not None:页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/60686029
复制相关文章
相似问题
腾讯云开发者
