首页 > 解决方案 > Kafka 删除日志段,即使它配置为无限保留

问题描述

Kafka (v1.0) 已从主题中删除了一个日志段,即使该主题配置为无限保留(retention.bytes = -1retention.ms = -1.

根据这个答案,这不应该发生!

以下是 Kafka 日志的摘录:

[2018-08-29 09:41:31,815] INFO Found deletable segments with base offsets [0] due to retention time 604800000ms breach (kafka.log.Log)
[2018-08-29 09:41:31,816] INFO Scheduling log segment 0 for log real-data-ingest-1 for deletion. (kafka.log.Log)
[2018-08-29 09:41:31,817] INFO Incrementing log start offset of partition real-data-ingest-1 to 79 in dir /opt/kafka/data/logs (kafka.log.Log)
[2018-08-29 09:41:31,819] INFO Cleared earliest 0 entries from epoch cache based on passed offset 79 leaving 1 in EpochFile for partition real-data-ingest-1 (kafka.server.epoch.LeaderEpochFileCache)
[2018-08-29 09:42:31,817] INFO Deleting segment 0 from log real-data-ingest-1. (kafka.log.Log)
[2018-08-29 09:42:31,817] INFO Deleting index /opt/kafka/data/logs/real-data-ingest-1/00000000000000000000.index.deleted (kafka.log.OffsetIndex)
[2018-08-29 09:42:31,817] INFO Deleting index /opt/kafka/data/logs/real-data-ingest-1/00000000000000000000.timeindex.deleted (kafka.log.TimeIndex)

和主题配置:

file.delete.delay.ms: "60000",
segment.ms: "604800000",
min.compaction.lag.ms: "0",
retention.bytes: "-1",
segment.index.bytes: "10485760",
cleanup.policy: "delete",
follower.replication.throttled.replicas: "",
message.timestamp.difference.max.ms: "9223372036854775807",
segment.jitter.ms: "0",
preallocate: "false",
segment.bytes: "1073741824",
message.timestamp.type: "CreateTime",
message.format.version: "1.0-IV0",
max.message.bytes: "1000012",
unclean.leader.election.enable: "false",
retention.ms: "-1",
flush.ms: "9223372036854775807",
delete.retention.ms: "86400000",
leader.replication.throttled.replicas: "",
min.insync.replicas: "2",
flush.messages: "9223372036854775807",
compression.type: "producer",
min.cleanable.dirty.ratio: "0.5",
index.interval.bytes: "4096"

有谁知道为什么会发生这种情况?

标签: apache-kafka

解决方案


推荐阅读