首页 > 解决方案 > 使用 nifi 将 JSON 转换为 Avro

问题描述

我正在尝试读取 RabbitMQ 队列并将数据传输到配置单元。

我的流程是这样的:ConsumeAMQP -> ConvertJSONToAvro -> PutHiveStreaming。

我在 ConvertJSONTOAvro 过程中遇到错误。

JSON:

{
    "bn":"/27546/0","bt":48128.94568269015,"e":
    [
        {"n":"1000","sv":"8125333b8-5cae-4c8d-a5312-bbb215211dab"},
        {"n":"1001","v":57.520565032958984},
        {"n":"1002","v":22.45258230712891},
        {"n":"1003","v":1331.0},
        {"n":"1005","v":53.0},
        {"n":"1011","v":50.0},
        {"n":"5518","t":44119.703412761854},
        {"n":"1023","v":0.0},
        {"n":"1024","v":48128.94568269015},
        {"n":"1025","v":7.0}
    ]
}

记录架构:

{
    "type": "record",
    "namespace": "nifi",
    "fields": [{
            "name": "bn",
            "type": "string"
        },
        {
            "name": "bt",
            "type": "number"
        },
        {
            "name": "e",
            "type": "array",
            "items": {
                "type": "record",
                "fields": [{
                        "name": "n",
                        "type": "string"
                    },
                    {
                        "name": "sv",
                        "type": "string"
                    },
                    {
                        "name": "v",
                        "type": "number"
                    },
                    {
                        "name": "t",
                        "type": "number"
                    }
                ]
            }
        }
    ]
}

错误

-–Record schema validated against '{"type":"record"...

我不知道出了什么问题。

标签: jsonschema

解决方案


谢谢迈克,我添加了一个名字,但不起作用。

{
    "name": "iot",
    "type": "record",
    "namespace": "nifi",
    "fields": [{
            "name": "bn",
            "type": "string"
        },
        {
            "name": "bt",
            "type": "number"
        },
        {
            "name": "e",
            "type": "array",
            "items": {
                "type": "record",
                "fields": [{
                        "name": "n",
                        "type": "string"
                    },
                    {
                        "name": "sv",
                        "type": "string"
                    },
                    {
                        "name": "v",
                        "type": "number"
                    },
                    {
                        "name": "t",
                        "type": "number"
                    }
                ]
            }
        }
    ]
}

JsonToAvroError


推荐阅读