google-cloud-storage - google cloud gsutil cp ResumableUploadAbortException: 408 Upload Client Broken Connection 错误
问题描述
我正在使用 gsutil cp 将一个大文件(大约 40g)从本地驱动程序移动到 gcs 存储桶。它以前运行良好,现在不知何故我不断收到“ResumableUploadAbortException: 408 Upload Client Broken Connection”。我试图做 gsutil cp 来移动一个小文件,它工作正常。
我在命令中放了一个-D:gsutil -D cp,这是我得到的跟踪:
DEBUG: Exception stack trace:iB] 80.0 MiB/s
Traceback (most recent call last):
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1592, in _PerformResumableUpload
additional_headers=addl_headers)
File "/home/test/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 1013, in StreamMedia
additional_headers=additional_headers, use_chunks=False)
File "/home/test/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 968, in __StreamMedia
raise exceptions.HttpError.FromResponse(response)
apitools.base.py.exceptions.HttpError: HttpError accessing <https://storage.googleapis.com/resumable/upload/storage/v1/b/charleszhan-dss-playground-bucket/o?alt=json&fields=etag%2Cgeneration%2Cmd5Hash%2Csize%2CcustomerEncryption%2Ccrc32c&uploadType=resumable&upload_id=AAANsUlrnIDRqoTNIeFM87J5TpvSOiKlxr-kfWMkYG0sO1qocki09C773h-z1V_JPAJ_jFGMyk-3fi7z35uf8-_ZsWBE8svx7w>: response: <{'x-guploader-uploadid': 'AAANsUlrnIDRqoTNIeFM87J5TpvSOiKlxr-kfWMkYG0sO1qocki09C773h-z1V_JPAJ_jFGMyk-3fi7z35uf8-_ZsWBE8svx7w', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 07 Aug 2020 15:28:34 GMT', 'vary': 'Origin, X-Origin', 'cache-control': 'no-cache, no-store, max-age=0, must-revalidate', 'expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'pragma': 'no-cache', 'content-length': '251', 'server': 'UploadServer', 'alt-svc': 'h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '408'}>, content <{
"error": {
"code": 408,
"message": "Upload Client Broken Connection",
"errors": [
{
"message": "Upload Client Broken Connection",
"domain": "global",
"reason": "uploadBrokenConnection"
}
]
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 1195, in RunCommand
seek_ahead_iterator=seek_ahead_iterator)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1515, in Apply
arg_checker, should_return_results, fail_on_error)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1586, in _SequentialApply
worker_thread.PerformTask(task, self)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2306, in PerformTask
results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 780, in _CopyFuncWrapper
preserve_posix=cls.preserve_posix_attrs)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 995, in CopyFunc
preserve_posix=preserve_posix)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3928, in PerformCopy
gzip_encoded=gzip_encoded)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2220, in _UploadFileToObject
parallel_composite_upload, logger)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2036, in _DelegateUploadFileToObject
elapsed_time, uploaded_object = upload_delegate()
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2209, in CallResumableUpload
gzip_encoded=gzip_encoded_file)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1837, in _UploadFileToObjectResumable
gzip_encoded=gzip_encoded)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 421, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1734, in UploadObjectResumable
gzip_encoded=gzip_encoded)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1517, in _UploadObject
additional_headers, progress_callback, gzip_encoded)
File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1655, in _PerformResumableUpload
raise resumable_ex
gslib.cloud_api.ResumableUploadAbortException: ResumableUploadAbortException: 408 Upload Client Broken Connection
解决方案
gsutil
每当您使用 cp 命令上传大于 8 MiB 的对象时,它都会自动执行可恢复上传。您不需要指定任何特殊的命令行选项来实现这一点。如果您的上传被中断,您可以通过运行与开始上传相同的 cp 命令重新开始上传。在上传成功完成之前,它在目标对象中不可见,并且不会替换上传要覆盖的任何现有对象。
推荐阅读
- database - 选择 Google Cloud 产品
- c++ - for循环中未处理的异常
- php - 输出特定行的 curl 输出
- angularjs - angularjs1.x youtube 视频仅使用 youtube iframe api 播放一次
- python - 在查找阶乘数时,无法理解在 python 中没有循环函数的函数是如何循环的
- xcode10 - preferredStatusBarStyle var 在 iOS12 中不起作用?
- javascript - Framework7如何路由为子页面
- apache-spark - Pyspark - 如何检查 RDD 操作中的变量
- types - 在 GraphQL 中重用输入类型作为片段
- php - 在我安装 Composer 后,Wamp 停止工作