jdbc - Kafka JDBC 源连接器和 Oracle 数据库错误

Question

虽然我的 Kafka JDBC 连接器适用于一个简单的表，但对于大多数其他表，它失败并出现错误：

任务引发了一个未捕获且不可恢复的异常 (org.apache.kafka.connect.runtime.WorkerTask:179) org.apache.kafka.connect.errors.ConnectException：在 org.apache.kafka.connect.runtime 的错误处理程序中超出了容差。 errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178) at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104) at org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord( WorkerSourceTask.java:290) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:316) 在 org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:240) 在 org .apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177) 在 org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227) 在 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 在 java.util.concurrent.FutureTask.run(FutureTask.java:266) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor .java:1149) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 在 java.lang.Thread.run(Thread.java:748) 引起：java.lang.IllegalArgumentException：无效的小数位数：127（大于精度：64） 在 org.apache.avro.LogicalTypes$Decimal.validate(LogicalTypes.java:231) 在 org.apache.avro.LogicalType.addToSchema(LogicalType.java:68) 在 org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes .java:201) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:943) 在 io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1058) 在 io.confluent.connect。 avro.AvroData.fromConnectSchema(AvroData.java:899) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:731) 在 io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:725)在 io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:364) 在 io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:80) 在 org.apache.kafka.connect.storage.Converter .fromConnectData(Converter.java:62) 在 org.apache.kafka.connect。runtime.WorkerSourceTask.lambda$convertTransformedRecord$2(WorkerSourceTask.java:290) 在 org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128) 在 org.apache.kafka.connect.runtime.errors .RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162) ... 11 更多

我正在使用以下命令创建连接器：

curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{"name": "jdbc_source_oracle_03","config": {"connector.class": "io. confluent.connect.jdbc.JdbcSourceConnector","connection.url":"jdbc:oracle:thin:@//XOXO:1521/XOXO","connection.user":"XOXO","connection.password":"XOXO ","numeric.mapping":"best_fit","mode":"timestamp","poll.interval.ms":"1000","validate.non.null":"false","table.whitelist": "POLICY","timestamp.column.name":"CREATED_DATE","topic.prefix":"ora-","transforms":"addTopicSuffix,InsertTopic,InsertSourceDetails,copyFieldToKey,extractValuefromStruct","transforms.InsertTopic.type":"org.apache.kafka.connect.transforms.InsertField$Value","transforms.InsertTopic.topic.field":"messagetopic","transforms.InsertSourceDetails.type":" org.apache.kafka.connect.transforms.InsertField$Value","transforms.InsertSourceDetails.static.field":"messagesource","transforms.InsertSourceDetails.static.value":"来自 Oracle 的 JDBC Source Connector on asgard"," transforms.addTopicSuffix.type":"org.apache.kafka.connect.transforms.RegexRouter","transforms.addTopicSuffix.regex":"(.*)","transforms.addTopicSuffix.replacement":"$1-jdbc-02 ","transforms.copyFieldToKey.type":"org.apache.kafka.connect.transforms.ValueToKey","transforms.copyFieldToKey.fields":"ID","transforms.extractValuefromStruct.type":"org.apache.kafka.connect.transforms.ExtractField$Key","transforms.extractValuefromStruct.field":"ID"}}'

标签： jdbcapache-kafka

Answer 1

该问题与未声明精度和比例的数字列有关。Robin Moffatt 在这里很好地解释了： https ://rmoff.net/2018/05/21/kafka-connect-and-oracle-data-types