首页
学习
活动
专区
工具
TVP
发布
社区首页 >问答首页 >现在尝试用BeautifulSoup和Python3从类中提取“href`”。

现在尝试用BeautifulSoup和Python3从类中提取“href`”。
EN

Stack Overflow用户
提问于 2018-08-22 08:33:19
回答 3查看 3K关注 0票数 2

我好像不能让它起作用。我的脚本访问一个站点并将数据抓取到我的info变量中,但是当我试图从一个特定的类中取出href时,我得到了None,或者当我尝试各种不同的组合时,它就不起作用了。我哪里搞砸了?当我把它放到我的info变量中时,里面有一个class='business-name'href

代码语言:javascript
复制
import requests
from bs4 import BeautifulSoup

count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')
info = soup.findAll('div', {'class': 'info'})
for each_business in info:
    # This is the spot that is broken. I can't make it work! 
    yp_bus_url = each_business.get('class_','business-name')['href']
    print(yp_bus_url)
EN

回答 3

Stack Overflow用户

回答已采纳

发布于 2018-08-22 08:56:24

我认为这就是你所需要的:

代码语言:javascript
复制
for each_business in info:
    yp_bus_url = each_business.find('a', {'class': 'business-name'}).get('href')
    print(yp_bus_url)
票数 2
EN

Stack Overflow用户

发布于 2018-08-22 09:04:42

您还可以执行以下操作:

代码语言:javascript
复制
import requests
from bs4 import BeautifulSoup

count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')

使用此处的更改(请确保将列表分配给您想要的任何内容):

代码语言:javascript
复制
#info = soup.findAll('div', {'class': 'info'})
info = soup.select("[class~=business-name]")
[i.get('href') for i in info]

返回:

代码语言:javascript
复制
['/new-orleans-la/mip/upperline-restaurant-526381149?lid=1001797484770',
 '/new-orleans-la/mip/brunos-tavern-451091659?lid=451091659',
 '/new-orleans-la/mip/lafittes-blacksmith-shop-bar-19195002?lid=19195002',
 '/new-orleans-la/mip/johnny-whites-pub-grill-5198728?lid=5198728',
 '/new-orleans-la/mip/chart-room-6924442?lid=6924442',
 '/new-orleans-la/mip/golden-lantern-8517918?lid=8517918',
 '/new-orleans-la/mip/ryans-irish-pub-inc-851820?lid=851820',
 '/new-orleans-la/mip/d-b-a-2084747?lid=2084747',
 '/new-orleans-la/mip/parlays-13663513?lid=13663513',
 '/new-orleans-la/mip/apple-barrel-18379645?lid=18379645',
 '/new-orleans-la/mip/snake-jakes-xmas-club-lounge-4531421?lid=4531421',
 '/new-orleans-la/mip/port-of-call-394043?lid=394043',
 '/new-orleans-la/mip/coops-place-14511722?lid=14511722',
 '/new-orleans-la/mip/twi-ro-pa-466224645?lid=466224645',
 '/new-orleans-la/mip/krazy-korner-11594425?lid=11594425',
 '/new-orleans-la/mip/bourbon-o-480103567?lid=480103567',
 '/new-orleans-la/mip/hi-ho-lounge-458821090?lid=458821090',.....]
票数 3
EN

Stack Overflow用户

发布于 2018-08-22 08:54:27

下面的代码应该可以为您工作:

代码语言:javascript
复制
import requests
from bs4 import BeautifulSoup

count = 0
search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = {'search_terms': search_terms, 'geo_location_terms': location}
page = requests.get(url, params=q)
url_link = page.url
page_num = str(count)
searched_page = url_link + '&page=' + str(count)
page = requests.get(searched_page)
soup = BeautifulSoup(page.text, 'html.parser')
info = soup.findAll('div', {'class': 'info'})
for each_business in info:
   # Your Fix here
   for a in each_business.find_all('a', href=True):
      print("Found the URL:", a['href'])
票数 2
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/51958411

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档