首页 > 解决方案 > 如何将值映射到 Java 中的嵌套 Avro Schema

问题描述

我正在尝试为 kafka 主题生成数据。作为要求的一部分,我从数据库中提取数据并将其映射到 Avro 模式。我知道如何解析数据,只要模式很简单,如下所示:

简单的 Avro 架构:

{
    "type":"record",
    "namespace":"demo",
    "name":"coreloader330",
    "doc":"sample avro schema file for core loader 330",
    "fields":[
        { "name" : "HEALTH_PLAN", "type" : ["null", "string"]}
    ]

}

通用构建器 java :

public GenericRecordBuilder getGenericBuilder() {

        GenericRecordBuilder builder = null;

        Schema.Parser parser = new Schema.Parser();
        try {
            Schema schema = parser.parse(new File("src/main/resources/coreloader.avsc"));
           builder = new GenericRecordBuilder(schema);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return builder;

将值映射到 Builder :

public void produceMessage(){

        //Create Producer
        KafkaProducer<String, GenericRecord> producer = new KafkaProducer<String, GenericRecord>(props);

        try{
            while(rs.next()){
               // System.out.println(rs.getString("HEALTH_PLAN"));
                builder.set("HEALTH_PLAN", rs.getString("HEALTH_PLAN")); ---> mapping values to avro schema field

                GenericData.Record  data = builder.build();
                ProducerRecord<String, GenericRecord> record = new ProducerRecord<String, GenericRecord>(topic, data);
                producer.send(record);
                producer.flush();
            }

但是,如果架构很复杂,我如何映射值,我如何才能到达映射的嵌套字段?

假设如果我有如下模式,我如何遍历并将值映射到 SourceSystem 字段?

"type": "record",
"name": "demo",
"namespace": "example",
"fields": [
    {
        "name": "data",
        "type": {
            "type": "record",
            "name": "Data",
            "fields": [
                {
                    "name": "SourceSystem",
                    "type": [
                        "null",
                        "string"
                    ],
                    "default": null
                }
            ]
        }
    }

下面代码中的映射应该是什么?

                builder.set(<?>, rs.getString("source system")); ---> mapping values to avro schema field

标签: javaavro

解决方案


推荐阅读