linux - 卡夫卡原木长得太高
问题描述
我可以看到kafka logs
正在迅速增长并淹没文件系统。
我如何更改 kafka 的设置以减少写入日志并经常轮换这些日志。
文件的位置是 -/opt/kafka/kafka_2.12-2.2.2/logs
以及它们的大小 -
5.9G server.log.2020-11-24-14
5.9G server.log.2020-11-24-15
5.9G server.log.2020-11-24-16
5.7G server.log.2020-11-24-17
来自上述文件的示例日志。
[2020-11-24 14:59:59,999] WARN Exception when following the leader (org.apache.zookeeper.server.quorum.Learner)
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at org.apache.zookeeper.common.AtomicFileOutputStream.write(AtomicFileOutputStream.java:74)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush(BufferedWriter.java:254)
at org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(QuorumPeer.java:1391)
at org.apache.zookeeper.server.quorum.QuorumPeer.setCurrentEpoch(QuorumPeer.java:1426)
at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:454)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:981)
[2020-11-24 14:59:59,999] INFO shutdown called (org.apache.zookeeper.server.quorum.Learner)
java.lang.Exception: shutdown Follower
at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:169)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:985)
[2020-11-24 14:59:59,999] INFO Shutting down (org.apache.zookeeper.server.quorum.FollowerZooKeeperServer)
[2020-11-24 14:59:59,999] INFO LOOKING (org.apache.zookeeper.server.quorum.QuorumPeer)
[2020-11-24 14:59:59,999] INFO New election. My id = 1, proposed zxid=0x1000001d2 (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2020-11-24 14:59:59,999] INFO Notification: 1 (message format version), 1 (n.leader), 0x1000001d2 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state) (org.apache.zookeeper.server.quorum.FastLeaderElection)
它还写入/opt/kafka/kafka_2.12-2.2.2/kafka.log
.
[2020-12-05 16:51:10,109] INFO [GroupMetadataManager brokerId=1] Finished loading offsets and group metadata from __consumer_offsets-30 in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-12-05 16:51:10,109] INFO [GroupMetadataManager brokerId=1] Finished loading offsets and group metadata from __consumer_offsets-36 in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-12-05 16:51:10,109] INFO [GroupMetadataManager brokerId=1] Finished loading offsets and group metadata from __consumer_offsets-42 in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-12-05 16:51:10,110] INFO [GroupMetadataManager brokerId=1] Finished loading offsets and group metadata from __consumer_offsets-48 in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-12-05 17:01:09,528] INFO [GroupMetadataManager brokerId=1] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2020-12-05 17:11:09,528] INFO [GroupMetadataManager brokerId=1] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
kafka 用于弹性堆栈。
以下是server.properties
文件中的条目。
# A comma seperated list of directories under which to store log files
log.dirs=/var/log/kafka
它的日志文件为
/var/log/kafka
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 heartbeat-1
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-12
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 auditbeat-0
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 apm-2
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-28
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 filebeat-2
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-38
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-44
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-6
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-16
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 metricbeat-0
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-22
drwxr-xr-x 2 kafka users 4.0K Dec 5 16:51 __consumer_offsets-32
-rw-r--r-- 1 kafka users 747 Dec 5 18:02 recovery-point-offset-checkpoint
-rw-r--r-- 1 kafka users 4 Dec 5 18:02 log-start-offset-checkpoint
-rw-r--r-- 1 kafka users 749 Dec 5 18:03 replication-offset-checkpoint
在路径中的文件中未启用任何DEBUG
级别日志。/opt/kafka/kafka_2.12-2.2.2/config
我如何确保它不会生成如此庞大的文件,/opt/kafka/kafka_2.12-2.2.2/logs
以及如何通过压缩定期旋转它们。
谢谢,
解决方案
log.dirs
是实际的代理存储,而不是进程日志,因此不应/var/log
与其他进程日志一起使用
一天差不多 6G 也不是不合理,但是可以修改 log4j.properties 文件,只保留滚动文件 appender 的 1 到 2 天左右
通常,与任何 Linux 管理任务一样,您将拥有单独的磁盘卷/var/log
、操作系统存储以及任何用于服务器数据的专用磁盘——比如挂载在/kafka
推荐阅读
- c++ - 用 std 实现替换 boost
- if-statement - 如何使用 if 语句在 FFMPEG 中添加颜色叠加
- python - 如何计算kmeans聚类中心之间的距离并在python中选择最小值?
- google-apps-script - 只在谷歌表格中调用自定义函数一次并将值保存到单元格
- javascript - nodejs mongodb chrome暴露密码潜在数据泄露修复
- excel - 如何全选或(Ctrl + A)动态表无关紧要
- angular - 如何将 ngModel 字符串转换为日期?
- windows - 错误:无法使用字符串条件执行 CURL 获取请求
- magento - 访问产品选项商店查看标题
- c - 这行代码“#define LIBINJECTION_SQLI_TOKEN_SIZE sizeof(((stoken_t*)(0))->val)”是做什么的?