amazon-web-services - 从 Kinesis 数据流到 Kinesis Firehose 到 S3 存储桶的 CloudFront 实时日志
问题描述
我能够通过控制台创建 CloudFront 实时日志,现在我想通过 Terraform 进行设置。
我目前有一个指向 S3 存储桶的 CloudFront 分配。
resource "aws_cloudfront_distribution" "www_distribution" {
default_cache_behavior {
realtime_log_config_arn = aws_cloudfront_realtime_log_config.analytics.arn
...
}
...
}
我创建了实时日志配置。
resource "aws_cloudfront_realtime_log_config" "analytics" {
name = "analytics"
sampling_rate = 100
fields = [
...
]
endpoint {
stream_type = "Kinesis"
kinesis_stream_config {
role_arn = aws_iam_role.analytics.arn
stream_arn = aws_kinesis_stream.analytics.arn
}
}
depends_on = [aws_iam_role_policy.analytics]
}
然后由 Kinesis 数据流管理
resource "aws_kinesis_stream" "analytics" {
name = "blog-cloudfront-analytics"
shard_count = 1
retention_period = 48
shard_level_metrics = [
"IncomingBytes",
"OutgoingBytes",
]
}
我希望它被 Kinesis Firehose 流消耗。
resource "aws_kinesis_firehose_delivery_stream" "extended_s3_stream" {
name = "example-cloudfront-analytics"
destination = "extended_s3"
kinesis_source_configuration {
kinesis_stream_arn = aws_kinesis_stream.analytics.arn
role_arn = aws_iam_role.kinesis_firehose.arn
}
extended_s3_configuration {
cloudwatch_logging_options {
log_group_name = "/aws/lambda/example_cloudfront_analytics"
log_stream_name = "example_stream"
enabled = true
}
role_arn = aws_iam_role.firehose_role.arn
bucket_arn = aws_s3_bucket.bucket.arn
}
}
resource "aws_s3_bucket" "bucket" {
bucket = "example-cloudfront-analytics"
acl = "private"
}
我已应用此配置,但 Kinesis 数据流控制台中的“监视器”选项卡显示没有任何内容发送到流。我该如何设置?
更新
以下是用于前面提到的不同服务的 IAM 角色。
这是用于Kinesis Firehose的那个
data "aws_iam_policy_document" "kinesis_firehose" {
statement {
effect="Allow"
actions = [
"kinesis:*",
"firehose:*"
]
resources = [
aws_kinesis_stream.analytics.arn,
aws_kinesis_firehose_delivery_stream.extended_s3_stream.arn
]
sid = "kinesisId"
}
}
resource "aws_iam_role" "kinesis_firehose" {
name = "cloudfront_kinesis_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
}
resource "aws_iam_role_policy" "kinesis_firehose_stream" {
policy = data.aws_iam_policy_document.kinesis_firehose.json
role = aws_iam_role.kinesis_firehose.id
}
这是实时日志中的一种用途。
resource "aws_iam_role" "analytics" {
name = "cloudfront-realtime-log"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "kinesis.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
}
resource "aws_iam_role_policy" "analytics" {
name = "cloudfront-realtime-log"
role = aws_iam_role.analytics.id
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kinesis:DescribeStreamSummary",
"kinesis:DescribeStream",
"kinesis:PutRecord",
"kinesis:PutRecords"
],
"Resource": "${aws_kinesis_stream.analytics.arn}"
}
]
}
EOF
}
这是用于S3存储桶的那个。
resource "aws_iam_role" "firehose_role" {
name = "firehose_cloudfront"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
data "aws_iam_policy_document" "kinesis_firehose_s3" {
statement {
effect="Allow"
actions = [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:PutObject",
]
resources = [
aws_s3_bucket.bucket.arn,
"${aws_s3_bucket.bucket.arn}/*",
]
sid = "kinesisId"
}
}
resource "aws_iam_role_policy" "kinesis_firehose_stream_s3" {
policy = data.aws_iam_policy_document.kinesis_firehose_s3.json
role = aws_iam_role.firehose_role.id
}
解决方案
您的aws_cloudfront_realtime_log_config.analytics
使用角色aws_iam_role.analytics.arn
。但是,它的原理是kinesis.amazonaws.com
。它应该是cloudfront.amazonaws.com
,如TF 文档中所示:
resource "aws_iam_role" "analytics" {
name = "cloudfront-realtime-log"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
}
可能还有其他尚未确定的问题。
推荐阅读
- java - 将对象从过滤器返回到资源
- javascript - 通过 Outlook 客户端使用 PHP 或 JavaScript 发送电子邮件
- docker - 在 Docker 中从源安装
- android - Android:基于 2 个微调器选择填充 ListView (JSON)
- c# - 在 Activity 的 OnCreate() 期间显示进度条?
- javascript - 时间序列图上的 Plotly.js 渐变颜色
- paypal - Paypal“创建支付资源”不起作用(401 Unauthorized)
- reactjs - 内在属性的默认值
- ios - 重新安装应用后未收到订阅收据
- ios - 如何在 WebKit 视图中获取单击元素的 ID?