使用S3、BOTO、Django时由对等方重置连接-用于静态文件的存储

内容来源于 Stack Overflow,并遵循CC BY-SA 3.0许可协议进行翻译与使用

  • 回答 (2)
  • 关注 (0)
  • 查看 (71)

我正在尝试切换到使用AmazonS 3来承载我们Django项目的静态文件。我正在使用Django,boto,Django-存储和Django-压缩器。当我在dev服务器上运行Collection静态时,我会得到错误。

socket.error: [Errno 104] Connection reset by peer 

我所有静态文件的大小都是74 MB,看起来不太大。以前有人见过这个吗,或者有任何调试提示?

这是完整的痕迹。

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 113, in collect
    handler(path, prefixed_path, storage)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 303, in copy_file
    self.storage.save(prefixed_path, source_file)
  File "/usr/local/lib/python2.7/dist-packages/django/core/files/storage.py", line 45, in save
    name = self._save(name, content)
  File "/usr/local/lib/python2.7/dist-packages/storages/backends/s3boto.py", line 392, in _save
    self._save_content(key, content, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/storages/backends/s3boto.py", line 403, in _save_content
    rewind=True, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1222, in set_contents_from_file
    chunked_transfer=chunked_transfer, size=size)
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 714, in send_file
    chunked_transfer=chunked_transfer, size=size)
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 890, in _send_file_internal
    query_args=query_args
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 547, in make_request
    retry_handler=retry_handler
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 966, in make_request
    retry_handler=retry_handler)
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 927, in _mexe
    raise e
socket.error: [Errno 104] Connection reset by peer

提问于
用户回答回答于

这是一个问题,当你第一次创建一个新的桶时,你必须等待几个小时或分钟才能开始上传。我不知道为什么S3表现得那样。要证明尝试创建一个新桶,请将Django存储指向它,当试图上传Django项目中的任何内容时,将看到它显示连接对等重置,但是等待几个小时或几分钟再试一次,它将工作。重复相同的步骤。

用户回答回答于

TL;DR

如果存储桶不在默认区域,则需要告诉boto要连接到哪个区域,例如,如果桶位于us-West-2中,则需要在setings.py中添加以下行:

 AWS_S3_HOST = 's3-us-west-2.amazonaws.com'

长篇解释:

这不是权限问题,您不应该将桶权限设置为“身份验证用户”。

如果在一个非默认区域创建桶(在我的例子中,我使用的是us-West-2),则会发生此问题。

如果不使用默认区域,并且不告诉boto桶驻留在哪个区域,boto将连接到默认区域,而S3将使用307重定向响应到桶所属的区域。

不幸的是,由于boto中的这个bug:

https://github.com/boto/boto/disks/2207

如果307回复在boto完成上传文件之前到达,boto将不会看到重定向,并将继续上传到默认区域。最后,S3关闭套接字,从而导致“由对等方重置连接”。

这是一种比赛条件,取决于被上传对象的大小和互联网连接的速度,这解释了为什么会随机发生。

OP在一段时间后停止查看错误有两个可能的原因:

- he later created a new bucket in the default region and the problem went away by itself. 
- he started uploading only small files, which are fast enough to be fully uploaded by the time S3 replies with 307

扫码关注云+社区