首页
学习
活动
专区
圈层
工具
发布
首页
学习
活动
专区
圈层
工具
MCP广场
社区首页 >问答首页 >使用Selenium进行Web抓取,BeautifulSoup返回空列表

使用Selenium进行Web抓取,BeautifulSoup返回空列表
EN

Stack Overflow用户
提问于 2021-08-28 03:39:58
回答 1查看 82关注 0票数 1

我试图通过网站搜索,以确定某些项目的库存。我希望这个搜索网站,看看该项目是否“库存”,并通知用户发现的信息。我可以运行大部分代码,但这会不断地返回空列表。(我在BeautifulSoup4和Selenium中尝试了同样的结果。这两个版本都在下面).What,我错过了吗?任何帮助都将不胜感激!

代码语言:javascript
运行
复制
#selenium version
import os
import string
import subprocess
import traceback
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

def webscan():
    driver = webdriver.Chrome('/usr/lib/python3/dist-packages/selenium/webdriver/chrome/chromedriver')
    #driver.maximize_window()
    driver.get("https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11")

    #wait for the page to load fully

    time.sleep(5)

    status = driver.find_element_by_class('a-size-medium a-color-state')
    ps5_avail = status


    print(ps5_avail)
    if ps5_avail == "Currently unavailable":
        print("Sorry. Still not in stock...")
    elif ps5_avail == "Temporarily Out of Stock":
        print("Sorry. Still not in stock...")
    elif ps5_avail == "In stock soon.":
        print("sorr. Still not in stock...")
    else:
        print("Go NOW!!!!!")


    driver.close()


user_input = (input("Want a PS5?"))

if user_input == 'yes':
    webscan()
    print('scan is done')


####################################################################
#beautifulsoup version

import requests
from bs4 import BeautifulSoup

url = 'https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11'
res = requests.get(url)
html_page = res.content

soup = BeautifulSoup(html_page, 'html.parser')

text = soup.find_all(string='In Stock')

if text == True:
    print("Out of Stock")
else:
    print("Get it!")
EN

回答 1

Stack Overflow用户

回答已采纳

发布于 2021-08-28 03:58:52

您应该使用这个css选择器来代替:

代码:

代码语言:javascript
运行
复制
driver.get("https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11")

#wait for the page to load fully

time.sleep(5)

lngth = len(driver.find_elements(By.CSS_SELECTOR, "span.a-size-medium.a-color-state"))
print("value of lngth", lngth)

try:
    if len(driver.find_elements(By.CSS_SELECTOR, "span.a-size-medium.a-color-state")) > 0:
        print("Inside if")
        status = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div#availability span"))).text
        print(status)
    else:
        print("Looks like element was not found.")
except:
    print("Something went wrong")
    pass

导入:

代码语言:javascript
运行
复制
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/68961193

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档