amazon-web-services - Amazon SageMaker 从 S3 下载文件

Question

我将 midi 文件存储在 S3 存储桶中，并尝试将它们下载到 SageMake jupyter 笔记本中。我正在使用此代码

import os
import boto3  # Python library for Amazon API 
import botocore
from botocore.exceptions import ClientError
def download_from_s3(url):
    """ex: url = s3://sagemakerbucketname/data/validation.tfrecords"""
    url_parts = url.split("/")  # => ['s3:', '', 'sagemakerbucketname', 'data', ...
    bucket_name = url_parts[2]
    key = os.path.join(*url_parts[3:])
    filename = url_parts[-1]
    if not os.path.exists(filename):
        try:
            # Create an S3 client
            s3 = boto3.resource('s3')
            print('Downloading {} to {}'.format(url, filename))
            s3.Bucket(bucket_name).download_file(key, filename)
        except botocore.exceptions.ClientError as e:
            if e.response['Error']['Code'] == "404":
                print('The object {} does not exist in bucket {}'.format(
                    key, bucket_name))
            else:
                raise

但是我在调​​用 HeadObject 操作时出现错误（403）：禁止

以下是 S3 附加的权限：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

标签： amazon-web-servicesamazon-s3amazon-sagemaker

Answer 1

您使用的 S3 存储桶sagemakerbucketname应与 Sagemaker Notebook 实例位于同一区域。应授予与笔记本实例关联的 IAM 角色访问 S3 存储桶的权限。

在 sagemaker 笔记本中运行以下命令以获取 IAM 角色

role = get_execution_role()

验证用于启动笔记本的角色是否有权访问 S3 存储桶。这些是您应该拥有的权限

{ "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::sagemakerbucketname" ] }, { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::sagemakerbucketname/*" ] }