使用Selenium进行Web抓取，BeautifulSoup返回空列表
使用Selenium进行Web抓取，BeautifulSoup返回空列表
提问于 2021-08-28 03:39:58
我试图通过网站搜索，以确定某些项目的库存。我希望这个搜索网站，看看该项目是否“库存”，并通知用户发现的信息。我可以运行大部分代码，但这会不断地返回空列表。(我在BeautifulSoup4和Selenium中尝试了同样的结果。这两个版本都在下面).What，我错过了吗？任何帮助都将不胜感激！
#selenium version
import os
import string
import subprocess
import traceback
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
def webscan():
driver = webdriver.Chrome('/usr/lib/python3/dist-packages/selenium/webdriver/chrome/chromedriver')
#driver.maximize_window()
driver.get("https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11")
#wait for the page to load fully
time.sleep(5)
status = driver.find_element_by_class('a-size-medium a-color-state')
ps5_avail = status
print(ps5_avail)
if ps5_avail == "Currently unavailable":
print("Sorry. Still not in stock...")
elif ps5_avail == "Temporarily Out of Stock":
print("Sorry. Still not in stock...")
elif ps5_avail == "In stock soon.":
print("sorr. Still not in stock...")
else:
print("Go NOW!!!!!")
driver.close()
user_input = (input("Want a PS5?"))
if user_input == 'yes':
webscan()
print('scan is done')
####################################################################
#beautifulsoup version
import requests
from bs4 import BeautifulSoup
url = 'https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11'
res = requests.get(url)
html_page = res.content
soup = BeautifulSoup(html_page, 'html.parser')
text = soup.find_all(string='In Stock')
if text == True:
print("Out of Stock")
else:
print("Get it!")回答 1
Stack Overflow用户
回答已采纳
发布于 2021-08-28 03:58:52
您应该使用这个css选择器来代替：
代码:
driver.get("https://www.amazon.com/GIGABYTE-GeForce-Windforce-Graphics-GV-N207SWF3OC-8GD/dp/B07WN6RVHH/ref=sr_1_11?crid=OOF3E7T9W54&dchild=1&keywords=graphics+card&qid=1630118746&sprefix=graphi%2Caps%2C190&sr=8-11")
#wait for the page to load fully
time.sleep(5)
lngth = len(driver.find_elements(By.CSS_SELECTOR, "span.a-size-medium.a-color-state"))
print("value of lngth", lngth)
try:
if len(driver.find_elements(By.CSS_SELECTOR, "span.a-size-medium.a-color-state")) > 0:
print("Inside if")
status = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div#availability span"))).text
print(status)
else:
print("Looks like element was not found.")
except:
print("Something went wrong")
pass导入:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/68961193
复制相关文章
相似问题
腾讯云开发者
