首页
学习
活动
专区
工具
TVP
发布
社区首页 >问答首页 >Python程序随机挂起

Python程序随机挂起
EN

Stack Overflow用户
提问于 2018-09-20 00:15:10
回答 1查看 0关注 0票数 0

我正在使用BeautifulSoup在Python中进行一些webscraping,其中涉及访问大约500个类似的网页,以便将数据放入.txt文件中。

但是,我遇到了一些问题:

  • 我的程序的CPU使用率(通过任务管理器检查)随机下降到0%并保持这种状态。
  • 我的命令提示符本身没有响应,因为Ctrl + C不会终止程序。
  • 这发生在看似随机的时间,在第8个网页和第480个网页之间的任何地方。 def getAnime(): for index in range(2, 502): # gets anime statistics from HTML container = containers[index] ranking = container.td.text name = container.findAll('td', {'class', 't'}) link = 'https://www.animenewsnetwork.com' + name[0].a['href'] name = name[0].text statistics = container.findAll('td', {'class', 'r'}) rating = statistics[0].text numVotes = statistics[1].text # prints out anime stats to file currentAnime = Anime(name, ranking, rating, numVotes, link) animeFile.write('\nname: ' + name) animeFile.write('\nlink: ' + link) animeFile.write('\nranking: ' + ranking) animeFile.write('\nrating: ' + rating) animeFile.write('\nvotes: ' + numVotes) # Goes to the webpage for the current anime animeClient = uReq(link) animeHTML = animeClient.read() animeClient.close() pageSoup = soup(animeHTML, 'html.parser') # Genres of the current anime try: genreDiv = pageSoup.find(id='infotype-30') genres = genreDiv.findAll('span') genreList = [] for genre in genres: genreList.append(genre.a.text) currentAnime.genres = genreList except: currentAnime.genres = 'unknown' # Themes of the current anime try: themes = pageSoup.find(id='infotype-31').findAll('span') themeList = [] for theme in themes: themeList.append(theme.a.text) currentAnime.themes = themeList except: currentAnime.themes = 'unknown' # Premiere date of the current anime try: date = pageSoup.find(id='infotype-9').div.text currentAnime.premiereDate = date except: currentAnime.premiereDate = 'unknown' # Director of the current anime try: director = pageSoup.find('b', text='Director').parent.a.text currentAnime.director = director except: currentAnime.director = 'unknown' # Production Studio of the current anime try: productionStudio = pageSoup.find('b', text='Production').parent.a.text currentAnime.studio = productionStudio except: currentAnime.studio = 'unknown' # Prints the genres animeFile.write('\ngenres: ') for genre in currentAnime.genres: animeFile.write(genre + ', ') # Prints the themes animeFile.write('\nthemes: ') for theme in currentAnime.themes: animeFile.write(theme + ', ') # Prints the premiere date, director, and studio animeFile.write('\npremiere date: ' + currentAnime.premiereDate) animeFile.write('\ndirector: ' + currentAnime.director) animeFile.write('\nproduction studio: ' + currentAnime.studio) animeFile.write('\n')
EN

回答 1

Stack Overflow用户

发布于 2018-09-20 10:04:50

CTRL-C只是向Python发送键盘中断命令。这意味着在BeautifulSoup中发出HTML请求时,您可能只是提升了一个异常级别。CtrlBreak将完全停止该程序。

您的脚本可能会运行到无响应的网页中。您的CPU 将为 0%,因为它正在等待Web服务。我建议link在每次uReq调用之前打印代码中的值以跟踪发生这种情况的位置。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/-100006141

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档