时间戳

首页 > 解决方案 > 使用 Cloudformation 启用 S3 清单

amazon-s3 - 使用 Cloudformation 启用 S3 清单

问题描述

如此图所示，我可以使用控制台启用库存。有没有办法用boto做同样的事情？

在此处输入图像描述

更新：

这是有效的完整脚本！

import boto3

s3_client = boto3.client(
    "s3",
    aws_access_key_id="XXX",
    aws_secret_access_key="XXX",
    region_name="us-east-1",
)

response = s3_client.put_bucket_inventory_configuration(
    Bucket="athenadata16",
    Id="myinventory",
    InventoryConfiguration={
        "Destination": {
            "S3BucketDestination": {
                "AccountId": "1234567890",
                "Bucket": "arn:aws:s3:::athenadata16",
                "Format": "ORC",
                "Prefix": "mypre",
            }
        },
        "IsEnabled": True,
        "Filter": {"Prefix": "myprefilter"},
        "Id": "myinventory",
        "IncludedObjectVersions": "Current",
        "OptionalFields": [
            "Size",
            "LastModifiedDate",
            "StorageClass",
            "ETag",
            "IsMultipartUploaded",
            "ReplicationStatus",
            "EncryptionStatus",
            "ObjectLockRetainUntilDate",
            "ObjectLockMode",
            "ObjectLockLegalHoldStatus",
            "IntelligentTieringAccessTier",
        ],
        "Schedule": {"Frequency": "Daily"},
    },
)

根据文档，可以使用 cloudformation 添加库存。

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-inventoryconfiguration.html

有人可以提供一个例子吗？

更新 2：

执行以下模板后，创建了一个名为“athenadata162a-bucketwithinventory-1snf1yx82si5c”的新存储桶。这是意料之中的。由于 BucketArn 设置，库存目标存储桶指向“athenadata162”。我需要将它指向当前的存储桶名称。可能吗？

Resources:
    BucketWithInventory: 
      Type: "AWS::S3::Bucket"
      Properties: 
        InventoryConfigurations: 
          - 
            Destination: 
              BucketAccountId: !Sub '${AWS::AccountId}'
              BucketArn: !Sub 'arn:aws:s3:::athenadata16'
              Format: CSV
              Prefix: mypre
            Enabled: true
            Id: myinventory
            IncludedObjectVersions: Current
            OptionalFields: 
              - Size
              - LastModifiedDate
              - StorageClass
              - ETag
              - IsMultipartUploaded
              - ReplicationStatus
              - ObjectLockRetainUntilDate
              - ObjectLockMode
              - ObjectLockLegalHoldStatus
              - IntelligentTieringAccessTier
            Prefix: myprefilter
            ScheduleFrequency: Daily

如果我把它改成这个，我会得到循环引用错误。

BucketArn: !Sub 'arn:aws:s3:::${BucketWithInventory}'

感谢 franklinsijo 的回答，这是创建带有库存的存储桶的完整代码，该存储桶将 csv 文件保存在同一个存储桶中。

Resources:
    BucketWithInventory: 
      Type: "AWS::S3::Bucket"
      Properties:
        BucketName: !Sub 'athenadata162-${AWS::AccountId}'
        InventoryConfigurations: 
          - 
            Destination: 
              BucketAccountId: !Sub '${AWS::AccountId}'
              BucketArn: !Sub 'arn:aws:s3:::athenadata162-${AWS::AccountId}'
              Format: CSV
              Prefix: mypre
            Enabled: true
            Id: myinventory
            IncludedObjectVersions: Current
            OptionalFields: 
              - Size
              - LastModifiedDate
              - StorageClass
              - ETag
              - IsMultipartUploaded
              - ReplicationStatus
              - ObjectLockRetainUntilDate
              - ObjectLockMode
              - ObjectLockLegalHoldStatus
              - IntelligentTieringAccessTier
            Prefix: myprefilter
            ScheduleFrequency: Daily

更新 4：

当我手动添加库存配置时，会自动添加以下存储桶策略。上面提到的 cloudformation 模板不包含此步骤，因此会出现“拒绝访问”错误。如何将其包含在该模板中？

{
    "Id": "S3-Console-Auto-Gen-Policy-1585038423058",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3PolicyStmt-DO-NOT-MODIFY-1585038422770",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::athenadata162-1234567890/*"
            ],
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": [
                        "arn:aws:s3:::athenadata162-1234567890"
                    ]
                },
                "StringEquals": {
                    "aws:SourceAccount": [
                        "1234567890"
                    ],
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}

更新 5

模板的最终版本将如下所示...

Resources:
    BucketWithInventory: 
      Type: "AWS::S3::Bucket"
      Properties:
        BucketName: !Sub '${AWS::StackName}-${AWS::AccountId}'
        InventoryConfigurations: 
          - 
            Destination: 
              BucketAccountId: !Sub '${AWS::AccountId}'
              BucketArn: !Sub 'arn:aws:s3:::${AWS::StackName}-${AWS::AccountId}'
              Format: CSV
              Prefix: mypre
            Enabled: true
            Id: myinventory
            IncludedObjectVersions: Current
            OptionalFields: 
              - Size
              - LastModifiedDate
              - StorageClass
              - ETag
              - IsMultipartUploaded
              - ReplicationStatus
              - ObjectLockRetainUntilDate
              - ObjectLockMode
              - ObjectLockLegalHoldStatus
              - IntelligentTieringAccessTier
            Prefix: myprefilter
            ScheduleFrequency: Daily

    BucketPolicyForInventoryBucket:
       Type: AWS::S3::BucketPolicy 
       Properties: 
         Bucket: !Ref BucketWithInventory
         PolicyDocument: 
            Statement:
            - 
                Effect: Allow
                Principal:
                    Service: s3.amazonaws.com
                Action:
                - s3:PutObject
                Resource:
                - !Join ["", ["arn:aws:s3:::", !Ref BucketWithInventory, "/*"]]
                Condition:
                    ArnLike:
                        aws:SourceArn:
                        - !Join ["", ["arn:aws:s3:::", !Ref BucketWithInventory, "/*"]]
                    StringEquals:
                        aws:SourceAccount:
                        - !Sub '${AWS::AccountId}'
                        s3:x-amz-acl: bucket-owner-full-control

标签： amazon-s3amazon-cloudformationboto3

解决方案

Boto3 参考列出了许多使用库存配置的方法，特别是：

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_bucket_inventory_configuration

如果您曾经问​​自己“boto3 可以这样做吗？”，请转到 boto3 文档，找到相应的服务参考 ( https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/index.html ) 和在该页面上快速按 Ctrl+F。很可能 boto3 确实可以做到这一点。

推荐阅读