时间戳

首页 > 解决方案 > Kafka consumers all have assigned_partition set to 0 but at runtime are assigned differnt partitions

apache-kafka - Kafka consumers all have assigned_partition set to 0 but at runtime are assigned differnt partitions

问题描述

I am not sure if i am doing something wrong - but if someone can please help shed some light on this.

Scenario:

Before i fire up the producer - i wanted to see what partitions these consumers are assigned to and see :

Listening on Topic : TOPIC_2PART
Status :True
ClientId : consumer_3_1
***************
  CONFIG 
***************
bootstrap_servers -> $node1_port
client_id -> consumer_3_1
group_id -> CGRP_TOPIC_2PART
value_deserializer -> <function CreateConsumer.consume.<locals>.<lambda> at 0x0000020175217DC8>
partition_assignment_strategy -> (<class 'kafka.coordinator.assignors.range.RangePartitionAssignor'>, <class 'kafka.coordinator.assignors.roundrobin.RoundRobinPartitionAssignor'>)
metric_group_prefix -> consumer
***************
  METRICS 
***************         
    consumer-fetch-manager-metrics
    **assigned-partitions->0.0**

And for the second consumer :

Listening on Topic : TOPIC_2PART
Status :True
ClientId : consumer_3
***************
  CONFIG 
***************
bootstrap_servers -> $node1_port
client_id -> consumer_3
group_id -> CGRP_TOPIC_2PART
value_deserializer -> <function CreateConsumer.consume.<locals>.<lambda> at 0x00000118DC956DC8>
partition_assignment_strategy -> (<class 'kafka.coordinator.assignors.range.RangePartitionAssignor'>, <class 'kafka.coordinator.assignors.roundrobin.RoundRobinPartitionAssignor'>)
selector -> <class 'selectors.SelectSelector'>
***************
  METRICS 
***************
            kafka-metrics-count
            consumer-fetch-manager-metrics
            consumer-coordinator-metrics
    assigned-partitions->0.0

So above before either consumer starts consume assigned-partitions are 0

After producer starts sending messages to both partitions on the topic - i see one consumer having partition 0 and the other partition 1

ConsumerRecord(topic='TOPIC_2PART', **partition=0**, offset=14, timestamp=1616346804116, timestamp_type=0, key=None, value={xxxx}, headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=99, serialized_header_size=-1)

And the second consumer having

ConsumerRecord(topic='TOPIC_2PART', **partition=1**, offset=4, timestamp=1616346723367, timestamp_type=0, key=None, value={}, headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=83, serialized_header_size=-1)
ConsumerRecord(topic='TOPIC_2PART', **partition=1**, offset=5, timestamp=1616346743556, timestamp_type=0, key=None, value={}, headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=94, serialized_header_size=-1)

My question is why am i seeing the assigned-partition on both consumers set to 0 ...

Thanks

标签： apache-kafkakafka-consumer-apiapache-kafka-connectkafka-producer-api

解决方案

assigned-partitions指标是分配给每个消费者的分区数，而不是分区 ID 。有两个消费者和两个分区，一旦你订阅，这应该设置为 1。所以问题是 - 订阅何时发生？好像您粘贴的信息是在订阅之前提取的。

推荐阅读