BeautifulSoup - 获取div的属性我正在迭代

内容来源于 Stack Overflow,并遵循CC BY-SA 3.0许可协议进行翻译与使用

  • 回答 (2)
  • 关注 (0)
  • 查看 (92)

我正在使用BeautifulSoup来解析VC网站上的公司列表。我找到了正确的迭代元素,但我似乎无法自己获取这些元素的数据。

这是我要经历的示例HTML:

<div id="content" class="site-content">
    <main id="primary" class="content-area" role="main">
        <header class="page-header">
        <main id="portfolio-landing-company-list" class="page-content">
            <section id="portfolio__list--grid" class="portfolio__list--all">
            <div class="company company-stage--venturegrowth company-type--enterprise company--single-company">
                    <div class="company__thumbnail company__thumbnail-link">
                        <a href="http://www.domain1.com" title="Company1" target="_blank">
                    </div>      
            </div>
            <div class="company company-stage--seed company-type--bio company--single-company">
                    <div class="company__thumbnail company__thumbnail-link">
                        <a href="http://www.domain2.com" title="Company2" target="_blank">
                    </div>
            </div>

这就是我目前使用BeautifulSoup的方式,这部分工作得很好

portfolio = soup.find('div', attrs={'class': 'portfolio-tiles'})
for eachco in portfolio.find_all('article'):
  companyname = eachco.a['title']
  companyurl = eachco.a['href']

但我想要做的是从这里抓取类元素

<div class="company company-stage--venturegrowth company-type--enterprise company--single-company">
or
<div class="company company-stage--seed company-type--bio company--single-company">

(列表中的每个公司有多种变体)

我试过迭代:

portfolio = soup.find('div', attrs={'class': 'portfolio-tiles'})
for eachco in portfolio.find_all('article'):
  companyattributes = eachco.div['class']

但那会显示一排:

['company__thumbnail', 'company__thumbnail-link']

(又名,低于我正在寻找的水平)

如何迭代所有结果但是为每个结果获取类元素? 我觉得我错过了一些非常基本的东西,但是会很感激任何帮助,弄清楚那是什么东西!

提问于
用户回答回答于

这就是你要找的东西:

for i in range(len(soup)):
     print(soup.select('div[class*="stage"]')[i].attrs['class'])

输出

   ['company', 'company-stage--venturegrowth', 'company-type--enterprise', 'company--single-company']
   ['company', 'company-stage--seed', 'company-type--bio', 'company--single-company']y--single-company']
用户回答回答于

您可以使用re模块在类元素中查找特定文本。

from bs4 import BeautifulSoup
import re
html = """<html><div id="content" class="site-content">
    <main id="primary" class="content-area" role="main">
        <header class="page-header">
        <main id="portfolio-landing-company-list" class="page-content">
            <section id="portfolio__list--grid" class="portfolio__list--all">
            <div class="company company-stage--venturegrowth company-type--enterprise company--single-company">
                    <div class="company__thumbnail company__thumbnail-link">(
                        <a href="http://www.domain1.com" title="Company1" target="_blank">
                    </div>
            </div>
            <div class="company company-stage--venturegrowth company-type--enterprise company--single-company">
                    <div class="company__thumbnail company__thumbnail-link">
                        <a href="http://www.domain2.com" title="Company2" target="_blank">
                    </div>
            </div> </html>"""

soup = BeautifulSoup(html, 'html.parser')
divs = soup.find_all('div' ,class_=re.compile("stage"))
for div in divs:
    print(div['class'])

输出:

[u'company', u'company-stage--venturegrowth', u'company-type--enterprise', u'company--single-company']
[u'company', u'company-stage--venturegrowth', u'company-type--enterprise', u'company--single-company']

扫码关注云+社区

领取腾讯云代金券