elasticsearch - ELK - 从日志中获取日期时间

Question

我正在尝试使用 logstash 和 filebeats 将日志发送到 Kibana。我需要从日志文件中获取日志时间。但是 kibana 不会将数据和时间（在日志中）作为日期时间。取而代之的是字符串，也不能更改格式。我一直在尝试不同的时间格式，但没有运气。感谢您的帮助这里是配置。

日志存储过滤器

filter {
grok {
match => {
"message" => "(?application_[^/])[^ ] [%{TIMESTAMP_ISO8601:logTime}] %{LOGLEVEL:logLevel} %{GREEDYDATA:LogMessage}" }
}
}

样本日志

/yarn/container-logs/application_1621858977521_0151/container_1621858977521_0151_01_000004 [2021-06-28 02:38:10,542] INFO Started daemon with process name: 7796@slave1 (org.apache.spark.executor.CoarseGrainedExecutorBackend)
/yarn/container-logs/application_1621858977521_0151/container_1621858977521_0151_01_000004 [2021-06-28 02:38:10,547] INFO Registered signal handler for TERM (org.apache.spark.util.SignalUtils)
/yarn/container-logs/application_1621858977521_0151/container_1621858977521_0151_01_000004 [2021-06-28 02:38:10,548] INFO Registered signal handler for HUP (org.apache.spark.util.SignalUtils)

标签： elasticsearchlogstashkibanalogstash-grok

Answer 1

只需将logTime重命名为时间戳

filter {
grok {
match => {
"message" => "(?application_[^/])[^ ] [%{TIMESTAMP_ISO8601:timestamp}] %{LOGLEVEL:logLevel} %{GREEDYDATA:LogMessage}" }
}
}