flume - 如何解决 Flume Syslog 设置错误 - “事件大小大于指定的事件大小：2500。考虑增加最大事件大小”

问题描述

我正在尝试使用org.apache.flume.source.MultiportSyslogTCPSource. 设置和配置成功，但是当水槽从 syslog 生成事件时出现以下错误。

[INFO ] [2019-03-28 13:22:27.217] [[channel=file-channel] - CheckpointBackUpThread] [org.apache.flume.channel.file.EventQueueBackingStoreFile] - Checkpoint backup completed.

[WARN ] [2019-03-28 13:22:31.853] [NioProcessor-2] 

[org.apache.flume.source.MultiportSyslogTCPSource] - Event size larger than specified event size: 2500. Consider increasing the max event size.
    [INFO ] [2019-03-28 13:22:35.686] [SinkRunner-PollingRunner-DefaultSinkProcessor] [org.apache.flume.sink.LoggerSink] - Event: { headers:{flume.syslog.status=Invalid} body: 31 33 33 62 32 20 74 3C 31 33 34 3E 4D 61 72 20 133b2 t<134>Mar  }

这是我的配置：

#source,channel and sink
testagent.sources = testlog
testagent.channels = file-channel
testagent.sinks = logger-sink

#source
testagent.sources.testlog.type = org.apache.flume.source.MultiportSyslogTCPSource
testagent.sources.testlog.ports = 9002
testagent.sources.testlog.host = 127.0.0.1

#sink
testagent.sinks.logger-sink.type = logger

#channel
testagent.channels.file-channel.type = file
testagent.channels.file-channel.dataDirs = /test/data/01/
testagent.channels.file-channel.checkpointDir = /test/data/01/checkpoint
testagent.channels.file-channel.useDualCheckpoints = true
testagent.channels.file-channel.backupCheckpointDir = /test/data/01/checkpoint-backup
testagent.channels.file-channel.transactionCapacity = 10000
testagent.channels.file-channel.checkpointInterval = 20000
testagent.channels.file-channel.maxFileSize = 1072692224
testagent.channels.file-channel.minimumRequiredSpace = 524288000
testagent.channels.file-channel.capacity = 1000000
testagent.channels.file-channel.keep-alive = 3
testagent.channels.file-channel.checkpointOnClose = true
testagent.sinks.logger-sink.channel = file-channel
testagent.sources.testlog.channels  = file-channel

从我的测试应用程序生成的日志应该成功地显示到水槽代理日志中，并且应该从每个日志语句（行）中生成事件。

标签： flume

flume - 如何解决 Flume Syslog 设置错误 - “事件大小大于指定的事件大小：2500。考虑增加最大事件大小”

问题描述

解决方案

推荐阅读