时间戳

首页 > 解决方案 > django-storage with S3: TypeError: a bytes-like object is required, not 'str'

python - django-storage with S3: TypeError: a bytes-like object is required, not 'str'

问题描述

我有一个将 CSV 导出到 S3 的函数：

def export_csv(report_id):
    # MyModel has a JsonField called `json_data`
    items = MyModel.objects.all()

    data = [x.json_data for x in items]

    df = pd.DataFrame(data)

    file_buffer = io.StringIO()

    # QUOTE_ALL to avoid separators from throwing off columns
    df.to_csv(
        file_buffer,
        index=False,
        encoding='utf-8',
        quotechar='"',
        quoting=csv.QUOTE_ALL,
    )

    report.report_data_file.save(
        "data.csv", ContentFile(file_buffer.getvalue())
    )

我正在使用以下字段字段：

class PrivateMediaStorage(S3Boto3Storage):
    location = settings.AWS_PRIVATE_MEDIA_LOCATION
    default_acl = 'private'
    file_overwrite = False
    custom_domain = False

class Report(BaseModel):
    ...

    report_data_file = models.FileField(
        storage=PrivateMediaStorage(),
        upload_to=export_location,
        null=True,
        blank=True,
    )

这在我的本地环境中运行良好，其中PrivateMediaStorage定义为

class PrivateMediaStorage(FileSystemStorage):
    pass

在我使用django-storages的生产环境中，运行此函数时出现以下错误：

TypeError: a bytes-like object is required, not 'str'

这是完整的回溯：

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 650, in __protected_call__
    return self.run(*args, **kwargs)
  File "/code/apps/benchmark/tasks.py", line 45, in export_report_data_task
    report.export_json_data()
  File "/code/apps/benchmark/models.py", line 470, in export_json_data
    export_csv(self.id)
  File "/code/apps/benchmark/utils/export_report_data.py", line 46, in export_csv
    report.report_data_file.save(
  File "/usr/local/lib/python3.8/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/usr/local/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/usr/local/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 495, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/usr/local/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/usr/local/lib/python3.8/site-packages/boto3/s3/inject.py", line 619, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
  File "/usr/local/lib/python3.8/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/usr/local/lib/python3.8/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/s3transfer/upload.py", line 558, in _submit
    self._submit_multipart_request(
  File "/usr/local/lib/python3.8/site-packages/s3transfer/upload.py", line 622, in _submit_multipart_request
    for part_number, fileobj in part_iterator:
  File "/usr/local/lib/python3.8/site-packages/s3transfer/upload.py", line 270, in yield_upload_part_bodies
    fileobj, full_size = self._get_upload_part_fileobj_with_full_size(
  File "/usr/local/lib/python3.8/site-packages/s3transfer/upload.py", line 343, in _get_upload_part_fileobj_with_full_size
    return six.BytesIO(data), len(data)
TypeError: a bytes-like object is required, not 'str'

标签： pythondjangoamazon-s3boto3python-django-storages

解决方案

推荐阅读