apache-kafka - 代理地址更改后，Kafka Producer 无法重新连接和刷新元数据

Question

我使用 k8s 运行 Kafka，使用 k8s NodePort，它在 Kafka 客户端重建 pod 后不会更改以获取主题元数据。Pod Ip 和 Node Ip 可以在外部访问。

卡夫卡版本：2.11-0.11.0.2

测试脚本：

bin/kafka-producer-perf-test.sh \
   --print-metrics \
   --topic $TOPIC \
   --num-records 10000000 \
   --record-size 500 \
   --throughput 10 \
   --producer-props acks=1 metadata.max.age.ms=10000 \
   bootstrap.servers=$KAFKA_0 \
   buffer.memory=67108864 \
   batch.size=8196

输出：

52 records sent, 10.3 records/sec (0.00 MB/sec), 14.6 ms avg latency, 325.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 1.5 ms avg latency, 2.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 1.4 ms avg latency, 2.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 1.3 ms avg latency, 2.0 ms max latency.
51 records sent, 10.2 records/sec (0.00 MB/sec), 1.2 ms avg latency, 2.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 1.1 ms avg latency, 2.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 5.3 ms avg latency, 113.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 1.1 ms avg latency, 2.0 ms max latency.
50 records sent, 10.0 records/sec (0.00 MB/sec), 0.9 ms avg latency, 2.0 ms max latency.
[2019-12-05 10:24:41,072] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,128] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,175] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,225] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,403] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,410] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,778] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:41,807] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:42,567] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:42,680] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:43,417] WARN [Producer clientId=producer-1] Connection to node 14 (/10.197.129.81:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-12-05 10:24:43,709] WARN [Producer clientId=producer-1] Connection to node 12 (/10.197.128.77:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

26 records sent, 0.2 records/sec (0.00 MB/sec), 4616.4 ms avg latency, 120000.0 ms max latency.
org.apache.kafka.common.errors.TimeoutException: Expiring 15 record(s) for reconnect-test-2:120000 ms has passed since batch creation
org.apache.kafka.common.errors.TimeoutException: Expiring 15 record(s) for reconnect-test-2:120000 ms has passed since batch creation
org.apache.kafka.common.errors.TimeoutException: Expiring 15 record(s) for reconnect-test-2:120000 ms has passed since batch creation

问题：

After rebuilding the Kafka broker pod, pod IP changed, NodePort not change, the Kafka producer can not reconnect, and print the warning log forever. But there is no problem after restart the test script.

标签： apache-kafka