mongodb - MongoDB Atlas 数据湖

问题描述

我正在尝试在 S3 存储桶中安装一组 CSV 或更确切地说是 TSV 文件。数据湖配置似乎没问题，但任何查询都以an internal error occurred.

> use s3-logs
switched to db s3-logs
> show collections
bucket1
> db.bucket1.find()
Error: error: { "ok" : 0, "errmsg" : "an internal error occurred", "code" : 1 }

这是我的配置：

> db.runCommand( { "storageGetConfig" : 1 } )
{
    "ok" : 1,
    "storage" : {
        "stores" : [
            {
                "s3" : {
                    "name" : "s3-logs",
                    "region" : "us-east-1",
                    "bucket" : "my-bucket",
                    "delimiter" : "/",
                    "prefix" : "/"
                }
            }
        ],
        "databases" : {
            "s3-logs" : {
                "bucket1" : [
                    {
                        "store" : "s3-logs",
                        "definition" : "/{filename string}"
                    }
                ]
            }
        }
    }
}

S3 存储桶已满文件（S3 访问日志）

$aws s3 ls my-bucket
2019-04-25 19:19:19        628 2019-04-25-23-19-18-F631D2AC624DBBCC
2019-04-25 19:20:36        608 2019-04-25-23-20-35-C398E9310C88B129
2019-04-25 19:21:01        628 2019-04-25-23-21-00-2787DD06D7E64751
2019-04-25 19:21:35        634 2019-04-25-23-21-33-227CA4A8439EC62E
2019-04-25 19:21:44       1215 2019-04-25-23-21-43-B1A51F63581E7447
2019-04-25 19:21:59        628 2019-04-25-23-21-58-7DA51B952DD9FAA1

文件示例 - 注意 - 没有标题。

53deb06d07d2d3404c3c9face2eae419ba989a5efe0a07bff7f148c6433488ab anton-iot-demo [25/Apr/2019:22:11:42 +0000] 24.246.45.35 arn:aws:iam::824967973088:user/antonum 148770891A83B6F4 REST.GET.ENCRYPTION - "GET /anton-iot-demo?encryption= HTTP/1.1" 404 ServerSideEncryptionConfigurationNotFoundError 357 - 3 - "-" "S3Console/0.4, aws-internal/3 aws-sdk-java/1.11.526 Linux/4.9.152-0.1.ac.221.79.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202 vendor/Oracle_Corporation" - 0EH+FcDKZvG3EJaLOg7D8CvgSncCp5DWiaZOg1tWR/sAtTCLrsmUnI+s8/FA2LOETrNZUNSiHhI= SigV4 ECDHE-RSA-AES128-SHA AuthHeader s3.amazonaws.com TLSv1.2
53deb06d07d2d3404c3c9face2eae419ba989a5efe0a07bff7f148c6433488ab anton-iot-demo [25/Apr/2019:22:11:42 +0000] 24.246.45.35 arn:aws:iam::824967973088:user/antonum 3970F31AEE6A9434 REST.GET.TAGGING - "GET /anton-iot-demo?tagging= HTTP/1.1" 404 NoSuchTagSet 294 - 82 - "-" "S3Console/0.4, aws-internal/3 aws-sdk-java/1.11.526 Linux/4.9.152-0.1.ac.221.79.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202 vendor/Oracle_Corporation" - 78c0hM+56hRGipoSUcBOeHHRZ9sfUmzrGtPOozqe+KkGkfFGqGyRstZQhI52os8XcR+5GPEUnJU= SigV4 ECDHE-RSA-AES128-SHA AuthHeader s3.amazonaws.com TLSv1.2

我在这里到底做错了什么？我希望能够指定文件标题和文档的详细信息，例如分隔符，但在文档中找不到任何内容。

标签： mongodbatlas-data-lake

mongodb - MongoDB Atlas 数据湖 - 挂载 CSV 文件

问题描述

解决方案

推荐阅读