首页
学习
活动
专区
工具
TVP
发布
社区首页 >问答首页 >在python中利用Reddit API爬取数据出现错误?

在python中利用Reddit API爬取数据出现错误?

提问于 2024-01-05 19:44:44
回答 1关注 0查看 82
代码语言:text
复制
import praw
import csv

client_id="03UcgbJEBrDsN-DBsuCU-w",		 # 您的客户 ID
client_secret="HDMkGK9ml2sf8NahSj74LRxTCPE_gA",	 # 您的客户秘密
user_agent="scraping"	 # 您的用户代理


# Reddit API 初始化
reddit = praw.Reddit(client_id=client_id, client_secret=client_secret, user_agent=user_agent)

# 关键词列表
keywords = ['hash']

#创建csv文件
with open('reddit_results.csv', 'w', encoding='utf-8', newline='') as csv_file:
    csv_writer = csv.writer(csv_file)
    csv_writer.writerow(['Author', 'Title', 'Content'])
# 爬取Reddit中的相关内容
for keyword in keywords:
    print(f"Results for keyword: {keyword}\n")
    # 使用搜索功能进行查询
    results = reddit.subreddit('all').search(keyword, sort='relevance', time_filter='all')

    # 输出相关内容的作者和标题
    for submission in results:
        author_id = submission.author.id if submission.author else 'N/A'
        title = submission.title
        content = submission.selftext if submission.is_self else 'Not a self-post'

        print(f"Author: {submission.author}")
        print(f"Title: {submission.title}")
        print(f"Content: {content}")
        print("\n---\n")
        # 写入CSV文件
        csv_writer.writerow([author_id, title, content])

当运行上述脚本时出现以下错误:

代码语言:text
复制
E:\Anaconda2\envs\my_env2\python.exe F:/GitHub/爬虫/venv/第三章:数据解析/reddit.py
Results for keyword: hash

Traceback (most recent call last):
  File "F:\GitHub\爬虫\venv\第三章:数据解析\reddit.py", line 26, in <module>
    for submission in results:
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\models\listing\generator.py", line 63, in __next__
    self._next_batch()
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\models\listing\generator.py", line 89, in _next_batch
    self._listing = self._reddit.get(self.url, params=self.params)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\util\deprecate_args.py", line 43, in wrapped
    return func(**dict(zip(_old_args, args)), **kwargs)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\reddit.py", line 712, in get
    return self._objectify_request(method="GET", params=params, path=path)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\reddit.py", line 517, in _objectify_request
    self.request(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\util\deprecate_args.py", line 43, in wrapped
    return func(**dict(zip(_old_args, args)), **kwargs)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\praw\reddit.py", line 941, in request
    return self._core.request(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 328, in request
    return self._request_with_retries(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 254, in _request_with_retries
    return self._do_retry(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 162, in _do_retry
    return self._request_with_retries(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 234, in _request_with_retries
    response, saved_exception = self._make_request(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 186, in _make_request
    response = self._rate_limiter.call(
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\rate_limit.py", line 46, in call
    kwargs["headers"] = set_header_callback()
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\sessions.py", line 282, in _set_header_callback
    self._authorizer.refresh()
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\auth.py", line 378, in refresh
    self._request_token(grant_type="client_credentials", **additional_kwargs)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\auth.py", line 155, in _request_token
    response = self._authenticator._post(url=url, **data)
  File "E:\Anaconda2\envs\my_env2\lib\site-packages\prawcore\auth.py", line 59, in _post
    raise ResponseException(response)
prawcore.exceptions.ResponseException: received 401 HTTP response

Process finished with exit code 1

回答

和开发者交流更多问题细节吧,去 写回答
相关文章

相似问题

相关问答用户
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档