时间戳

首页 > 解决方案 > 基于字符串过滤消息

logstash - 基于字符串过滤消息

问题描述

我在同一个日志文件中有以下日志
2019-11-23T14:38:43.495 backendorg [http-nio-8080-exec-45] INFO http-nio-8080-exec-45 SessionController http://localhost:8080/ABC/session/login abc.nayak@zinier.com backendorg

2019-11-23T14:38:44.235 backendorg [http-nio-8080-exec-45] INFO http-nio-8080-exec-45 SessionController userSession: backendorg 16CFAFCCFB14D9A3 16E978545E17BFEC 16E978545E1452FF

使用下面的过滤器根据字符串“ userSession ”解析上面的消息。

 input {
      file {
            tags => ["stacktrace"]
            type => "error_logs"
            path => ["/Users/znrind-a0053/Downloads/logs/zapp-audit.log"]
            start_position => "beginning"
            sincedb_path => "/tmp/sincedb_file"
            codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
            }
      }

}
filter {

      if "userSession" in [message]{
        grok {
        match => [ "message",
                 "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{URI:url}%{SPACE}(?<email>[\w.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:[.](?:[0-9A-Za-z][0-9A-Za-z‌​-]{0,62}))*)%{SPACE}%{USERNAME:orgnisation}"]
          }
      } else {

      grok {
      match => [ "message",
               "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVACLASS:javaClass} %{USERNAME:logmessage}:?%{SPACE}%{USERNAME:orgnisation}%{SPACE}%{USERNAME:loginUserId}%{SPACE}%{USERNAME:sessionId}%{SPACE}%{USERNAME:txnId}"]
        }
      }
}
output {
        elasticsearch {
            hosts => "localhost"
            index => "logs"
        }
        stdout{codec => json}
}

但收到 GROK 解析器错误。任何建议高度赞赏。

标签： logstashlogstash-grok

解决方案

试试这个filter：

filter {

      if "userSession" in [message]{
        grok {
        match => [ "message",
                 "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{USERNAME:logmessage}:?%{SPACE}%{USERNAME:orgnisation}%{SPACE}%{USERNAME:loginUserId}%{SPACE}%{USERNAME:sessionId}%{SPACE}%{USERNAME:txnId}"]
          }
      } else {

      grok {
      match => [ "message",
               "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{URI:url}%{SPACE}(?<email>[\w.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:[.](?:[0-9A-Za-z][0-9A-Za-z‌​-]{0,62}))*)%{SPACE}%{USERNAME:orgnisation}"]
        }
      }
}

推荐阅读