如何编辑代码以遍历页面？
如何编辑代码以遍历页面？
提问于 2019-09-25 20:55:27
如何修复下面的代码以遍历709页并返回我想要的结果？
import pandas as pd
from bs4 import BeautifulSoup,Tag
import requests
import re
res=requests.get("https://cupcakemaps.com/cupcakes/cupcakes-near-me/p:2")
soup = BeautifulSoup(res.text,'lxml')
listings=soup.findAll(class_='media')
data = []
for listing in listings:
listing_title=listing.find(True,{'title':True}).attrs['title']
listing_Description=listing.find('p',{'class':'summary-desc'})
if isinstance(listing_Description,Tag):
listing_Description = listing_Description.text.strip()
listing_address=listing.find('p',{'class':'contact-info'})
if isinstance(listing_address,Tag):
number_text = listing_address.text.strip()
listing_address = ''.join(filter(str.isdigit,number_text))
full_dict = {'Title': listing_title, 'Description': listing_Description, 'Address': listing_address}
data.append(full_dict)
df = pd.DataFrame(data)
df.to_csv("contact.csv")
print(df)我需要返回从page1：https://cupcakemaps.com/cupcakes/cupcakes-near-me/运行到: page709:https://cupcakemaps.com/cupcakes/cupcakes-near-me/p:709的webstie的所有709页的结果
回答 2
Stack Overflow用户
回答已采纳
发布于 2019-09-25 21:20:04
import pandas as pd
from bs4 import BeautifulSoup,Tag
import requests
import re
data = []
s = "https://cupcakemaps.com/cupcakes/cupcakes-near-me/p:"
for x in range(1,709):
res=requests.get(s+str(x))
soup = BeautifulSoup(res.text,'lxml')
listings=soup.findAll(class_='media')
for listing in listings:
listing_title=listing.find(True,{'title':True}).attrs['title']
listing_Description=listing.find('p',{'class':'summary-desc'})
if isinstance(listing_Description,Tag):
listing_Description = listing_Description.text.strip()
listing_address=listing.find('p',{'class':'contact-info'})
if isinstance(listing_address,Tag):
number_text = listing_address.text.strip()
listing_address = ''.join(filter(str.isdigit,number_text))
full_dict = {'Title': listing_title, 'Description': listing_Description, 'Address': listing_address}
data.append(full_dict)
df = pd.DataFrame(data)
df.to_csv("contact.csv")
print(df)未经测试，因为我在一个代理后面，但这应该工作，如果原始脚本工作…
Stack Overflow用户
发布于 2019-09-25 21:17:28
使用format()函数。
for n in range(1,709):
url="https://cupcakemaps.com/cupcakes/cupcakes-near-me/p:{}".format(n)
print(url)页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/58098943
复制相关文章
相似问题
腾讯云开发者
