python - 下载文件后如何避免权限被拒绝?
问题描述
我在多个线程中下载文件,每个线程都按流。
def download_file(self, dest:str, filename:str):
r = requests.get(self.url, stream=True, headers={
"Accept": "application/xml",
"Accept-Charset": "utf-8",
"Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4",
"Authorization": self.auth
})
with open(f"{dest}/{filename}", 'wb') as f:
total_length = r.headers.get("Content-length")
if not total_length:
f.write(r.content)
else:
for chunk in progress.bar(r.iter_content(chunk_size=1024), expected_size=(int(total_length)/1024) + 1):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
f.flush()
f.close()
return True
此后,我将这些下载的文件上传到 S3 存储桶。
def process_path_for_S3(local_file_path):
if not parsed_args.ignore_s3:
s3_repository = parsed_args.s3
logging.info(f"Upando o arquivo '{local_file_path['filename']}' para o S3...")
s3_key = f"{s3_repository.split('/')[1]}/{s3_repository.split('/')[2]}/{local_file_path['filename']}"
for i in range(0, parsed_args.tries):
try:
_s3.s3.meta.client.upload_file(Bucket=s3_repository.split('/')[0], Key=s3_key, Filename=local_file_path["local_path"])
except Exception as e:
logging.info(">>> ERRO <<<")
logging.info(e)
logging.info(">>> NÃO FOI POSSÍVEL REALIZAR O UPLOAD DO ARQUIVO PARA O S3 <<<")
logging.info("Tentando novamente...")
sleep(5)
continue
break
logging.info(f">>> ARQUIVO UPADO COM SUCESSO PARA O S3 <<<")
else:
logging.info(">>> IGNORANDO PROCESSO DE UPLOAD NO S3 <<<")
logging.info(f">>> PROCESSANDO UPLOAD PARA O S3 EM {parsed_args.s3_threads} THREADS <<<")
s3_multiprocessor._map(process_path_for_S3, local_file_paths) # Multithreading code.
问题是,当我尝试打开文件以将它们上传到 S3 时,出现权限错误。
[Errno 13] Permission denied: 'C:/Users/User/AppData/Roaming/...'
这不可能是由于多线程(一个线程保存和另一个试图同时打开),因为我已经尝试禁用多线程,并且我得到了同样的错误。我能想到的唯一原因是“按流下载”。我的同事建议我检查文件中的文件结尾,但是如果我无法打开它,我应该怎么做呢?
解决方案
推荐阅读
- javascript - 如何使用 document.write 在 div 标签中获取 javascript 对象的输出?
- sqlite - 如果 SQLite 中的模式不匹配,则替换表
- javascript - 如何为当前转换编写 if 表达式: translate() 值?
- python - Django:不能对模板表进行 GROUP BY
- html - 我想创建一个模糊按钮并在悬停时取消模糊它
- assembly - 如何在 Assembler 中不使用循环对 3 个变量进行排序?
- swift - 带有StackView的Storyboard Scrollview不滚动
- ios - 每个 iOS 应用程序在设备上都有自己的单一钥匙串吗?
- r - 查找与字符串中的模式匹配的嵌套子字符串
- php - 如何避免从 Github API 收到 403 错误?