apache-kafka - Kafka ordering with multiple producers on same topic and parititon

Question

Let's say I have two producers (ProducerA and ProducerB) writing to the same topic with a single partition. Each producer is writing it's own unique events serially. So if ProducerA fired 3 events and then ProducerB fired 3 events, my understanding is that Kafka cannot guarantee the order across the producer's events like this:

1. ProducerA_event_1
2. ProducerA_event_2
3. ProducerA_event_3
4. ProducerB_event_1
5. ProducerB_event_2
6. ProducerB_event_3

due to acking, retrying, etc.

However will individual producer's events still be in order? For example:

1. ProducerA_event_1
2. ProducerB_event_2
3. ProducerB_event_1
4. ProducerA_event_2
5. ProducerA_event_3
6. ProducerB_event_3

This is of course a simplified version of what I am doing, but I just want to guarantee that if I am reading from a topic for a specific producer's events, then those events will be in order even if other producer's events interleave them.

标签： apache-kafka

Answer 1

有一篇关于 medium的好文章指出，即使对于同一个生产者，Kafka 也不总是保证消息的顺序。这一切都取决于 Kafka 配置。特别是，max.in.flight.requests.per.connection必须设置为1。原因是如果有多个请求（例如 2 个）在运行，而第一个请求失败，则第二个请求会更早地附加到日志中，从而破坏排序。