时间戳

首页 > 解决方案 > 初始化 TEST 连接器后雪花卡夫卡连接器失败

apache-kafka - 初始化 TEST 连接器后雪花卡夫卡连接器失败

问题描述

不知道出了什么问题。尝试设置 Snowflake kafka 连接，它似乎失败而没有抛出任何有用的日志

[2021-04-07 21:09:25,024] 信息创建 com.snowflake.kafka.connector.SnowflakeSinkConnector 类型的连接器 TEST_CONNECTOR (org.apache.kafka.connect.runtime.Worker:202) [2021-04-07 21: 09:25,028] INFO 实例化连接器 TEST_CONNECTOR，版本 1.5.0，类型为 com.snowflake.kafka.connector.SnowflakeSinkConnector (org.apache.kafka.connect.runtime.Worker:205) [2021-04-07 21:09: 25,029] INFO [SF_KAFKA_CONNECTOR] Snowflake Kafka 连接器版本：1.5.0 (com.snowflake.kafka.connector.Utils:99) [2021-04-07 21:09:25,092] WARN [SF_KAFKA_CONNECTOR] 连接器更新可用，请升级Snowflake Kafka 连接器 (1.5.0 -> 1.5.2) (com.snowflake.kafka.connector.Utils:136) [2021-04-07 21:09:25,092] INFO [SF_KAFKA_CONNECTOR] SnowflakeSinkConnector:start (com.snowflake. kafka.connector.SnowflakeSinkConnector:91) [2021-04-07 21:09:25,330] INFO [SF_KAFKA_CONNECTOR] 初始化雪花连接 (com.snowflake.kafka.connector.internal.SnowflakeConnectionServiceV1:38) [2021-04-07 21:09:25,336] INFO 完成创建连接器 TEST_CONNECTOR (org.apache.kafka.connect .runtime.Worker:224) [2021-04-07 21:09:25,337] INFO 跳过连接器 sflksink 的重新配置，因为它没有运行 (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:285) [2021- 04-07 21:09:25,338] 错误在连接器错误后停止 (org.apache.kafka.connect.cli.ConnectStandalone:113) java.lang.NullPointerException at org.apache.kafka.connect.cli.ConnectStandalone$1.onCompletion( ConnectStandalone.java:104) 在 org.apache.kafka.connect.cli.ConnectStandalone$1.onCompletion(ConnectStandalone.java:98) 在 org.apache.kafka.connect.util.ConvertingFutureCallback.onCompletion(ConvertingFutureCallback.java:44) 在 org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:107) 的 org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:185) [2021 -04-07 21:09:25,340] 信息卡夫卡连接停止 (org.apache.kafka.connect.runtime.Connect:65) [2021-04-07 21:09:25,341] 信息停止 REST 服务器 (org.apache. kafka.connect.runtime.rest.RestServer:211) [2021-04-07 21:09:25,345] 信息停止 http_8083@2cc0fa2a{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server. ServerConnector:306) [2021-04-07 21:09:25,354] 信息停止 oejsServletContextHandler@5c83ae01{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865) [2021-04-07 21 :09:25,360] INFO REST 服务器已停止 (org.apache.kafka.connect.runtime.rest.RestServer:222) [2021-04-07 21:09:25,360] INFO Herder 停止 (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:77) [2021-04-07 21:09:25,360] 信息停止连接器 TEST_CONNECTOR (org.apache.kafka.connect.runtime.Worker:305) [2021-04-07 21 :09:25,361] 信息 [SF_KAFKA_CONNECTOR] SnowflakeSinkConnector:stop (com.snowflake.kafka.connector.SnowflakeSinkConnector:141) [2021-04-07 21:09:25,362] 信息停止连接器 TEST_CONNECTOR (org.apache.kafka.connect. runtime.Worker:321) [2021-04-07 21:09:25,362] 信息工作者停止 (org.apache.kafka.connect.runtime.Worker:151) [2021-04-07 21:09:25,365] 信息停止FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:67) [2021-04-07 21:09:25,365] 信息工作者停止 (org.apache.kafka.connect.runtime.Worker:172) [2021-04 -07 21:09:25,369] INFO Herder 停止（org.apache.kafka.connect.runtime.standalone.StandaloneHerder:87）[2021-04-07 21:09:25，371]信息卡夫卡连接停止（org.apache.kafka.connect.runtime.Connect:70）

配置文件如下所示

name=sflksink
connector.class=com.snowflake.kafka.connector.SnowflakeSinkConnector
tasks.max=1
topics=snowflake-connect-test
buffer.count.records=10
buffer.flush.time=60
buffer.size.bytes=50
snowflake.url.name=url
snowflake.user.name=<user>
snowflake.database.name=<database>
snowflake.schema.name=<schema>
snowflake.private.key=<private_key>
snowflake.warehouse.name=MY_WAREHOUSE
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=com.snowflake.kafka.connector.records.SnowflakeJsonConverter

任何指示都会有所帮助

标签： apache-kafkasnowflake-cloud-data-platformapache-kafka-connect

解决方案

在日志消息中它说

Connector update is available, please upgrade Snowflake Kafka Connector (1.5.0 -> 1.5.2)

因此，我建议您将连接器 JAR 更新到最新版本，然后重试。

推荐阅读