我有一个脚本,可以使用s3 AWS将目录中的文件上传到boto3。我已经使用它几个星期了,没有问题,今天它将随机上传大量文件(在5个文件总数中的1-3个之间),并抛出以下错误:
File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
我的代码如下所示,我添加了time.sleep(2)作为一种故障排除方法,以查看我是否提出请求过快,但它没有解决问题。
import boto3
from botocore.exceptions import NoCredentialsError
import os
import time
def upload_to_aws(local_file, bucket, s3_file):
access_key = 'ACCESS_KEY'
secret_key = 'SECRETS'
s3 = boto3.client('s3', aws_access_key_id=access_key,
aws_secret_access_key=secret_key)
try:
s3.upload_file(local_file, bucket, s3_file)
return True
except FileNotFoundError:
return False
except NoCredentialsError:
return False
def archive_to_s3(files):
for file in files:
upload_to_aws('daily_data/'+file, 'bucket', '2020/'+file)
os.remove('daily_data/'+file)
time.sleep(2)
其他资料:
Nameservers:
1.1.1.1
1.0.0.1
任何帮助都是非常感谢的。
更新:
如下所示,我对端点进行了挖掘,显示1.1.1.1作为服务器,并在我的终端上启用了调试日志记录。当我运行第一个测试来获取日志时,它运行得完美无缺,并按照预期上传了所有文件。
发布于 2020-05-06 18:22:55
这是我用来上传到我的AWS S3桶的东西,它工作得很完美:
import boto3
BUCKET = 'myBucket'
s3 = boto3.client("s3", aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'), aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY'))
bucket_resource = s3
bucket_resource.upload_file(Bucket = BUCKET, Filename=picture_fn, Key=picture_fn) # uploading
https://stackoverflow.com/questions/61642290
复制相似问题