现在尝试用BeautifulSoup和Python3从类中提取“href`”。
现在尝试用BeautifulSoup和Python3从类中提取“href`”。
提问于 2018-08-22 08:33:19
我好像不能让它起作用。我的脚本访问一个站点并将数据抓取到我的info变量中，但是当我试图从一个特定的类中取出href时，我得到了None，或者当我尝试各种不同的组合时，它就不起作用了。我哪里搞砸了？当我把它放到我的info变量中时，里面有一个class='business-name'和href。
import requests
from bs4 import BeautifulSoup
count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')
info = soup.findAll('div', {'class': 'info'})
for each_business in info:
# This is the spot that is broken. I can't make it work!
yp_bus_url = each_business.get('class_','business-name')['href']
print(yp_bus_url)回答 3
Stack Overflow用户
回答已采纳
发布于 2018-08-22 08:56:24
我认为这就是你所需要的：
for each_business in info:
yp_bus_url = each_business.find('a', {'class': 'business-name'}).get('href')
print(yp_bus_url)Stack Overflow用户
发布于 2018-08-22 09:04:42
您还可以执行以下操作：
import requests
from bs4 import BeautifulSoup
count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')使用此处的更改(请确保将列表分配给您想要的任何内容)：
#info = soup.findAll('div', {'class': 'info'})
info = soup.select("[class~=business-name]")
[i.get('href') for i in info]返回：
['/new-orleans-la/mip/upperline-restaurant-526381149?lid=1001797484770',
'/new-orleans-la/mip/brunos-tavern-451091659?lid=451091659',
'/new-orleans-la/mip/lafittes-blacksmith-shop-bar-19195002?lid=19195002',
'/new-orleans-la/mip/johnny-whites-pub-grill-5198728?lid=5198728',
'/new-orleans-la/mip/chart-room-6924442?lid=6924442',
'/new-orleans-la/mip/golden-lantern-8517918?lid=8517918',
'/new-orleans-la/mip/ryans-irish-pub-inc-851820?lid=851820',
'/new-orleans-la/mip/d-b-a-2084747?lid=2084747',
'/new-orleans-la/mip/parlays-13663513?lid=13663513',
'/new-orleans-la/mip/apple-barrel-18379645?lid=18379645',
'/new-orleans-la/mip/snake-jakes-xmas-club-lounge-4531421?lid=4531421',
'/new-orleans-la/mip/port-of-call-394043?lid=394043',
'/new-orleans-la/mip/coops-place-14511722?lid=14511722',
'/new-orleans-la/mip/twi-ro-pa-466224645?lid=466224645',
'/new-orleans-la/mip/krazy-korner-11594425?lid=11594425',
'/new-orleans-la/mip/bourbon-o-480103567?lid=480103567',
'/new-orleans-la/mip/hi-ho-lounge-458821090?lid=458821090',.....]Stack Overflow用户
发布于 2018-08-22 08:54:27
下面的代码应该可以为您工作：
import requests
from bs4 import BeautifulSoup
count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')
info = soup.findAll('div', {'class': 'info'})
for each_business in info:
# Your Fix here
for a in each_business.find_all('a', href=True):
print("Found the URL:", a['href'])页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/51958411
复制相关文章
相似问题
腾讯云开发者
