时间戳

首页 > 解决方案 > 我怎样才能在logstash中获得特定的登录？

logstash - 我怎样才能在logstash中获得特定的登录？

问题描述

我正在设置 ELK 堆栈，然后有 log4j 的日志。

但是，我必须获取特定的日志。

所以我猜想编辑logstash的输出或过滤器，然后我只想得到[board.boardList]行

我需要设置此代码

其实我一直在寻找很多的网站，但我找不到任何方法

    2019-09-16 15:54:53,360 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:53,393 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@772121a] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:53,394 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:53,394 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 
2019-09-16 15:54:53,483 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@437df7c8] will not be managed by Spring 
2019-09-16 15:54:53,483 DEBUG [board.boardTotal] ooo Using Connection [oracle.jdbc.driver.T4CConnection@437df7c8] 
2019-09-16 15:54:53,484 DEBUG [board.boardTotal] ==> Preparing: select count(*) from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') 
2019-09-16 15:54:53,484 DEBUG [board.boardTotal] ==> Parameters: (String), (String) 
2019-09-16 15:54:53,485 DEBUG [board.boardTotal] <== Total: 1 
2019-09-16 15:54:53,485 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@772121a] 
2019-09-16 15:54:53,485 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource 
2019-09-16 15:54:53,487 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:53,487 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@605e4482] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:53,487 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:53,487 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 2019-09-16 15:54:53,544 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@4311318b] will not be managed by Spring 
2019-09-16 15:54:53,544 DEBUG [board.boardList] ooo Using Connection [oracle.jdbc.driver.T4CConnection@4311318b] 
2019-09-16 15:54:53,544 DEBUG [board.boardList] ==> Preparing: select * from (select rownum rn, a.* from (select b., m.m_nick as m_nick, (select count() from bow_board_re br where b.brd_no=br.brd_no and re_del_yn='n') as replycount, length(brd_subject) as sublength from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') order by ref desc, re_step) a) where rn between ? and ? and BRD_DEL_YN = 'N' 
2019-09-16 15:54:53,544 DEBUG [board.boardList] ==> Parameters: (String), (String), 1(Integer), 10(Integer) 
2019-09-16 15:54:53,547 DEBUG [board.boardList] <== Total: 0 
2019-09-16 15:54:53,547 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@605e4482] 
2019-09-16 15:54:53,547 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource 
2019-09-16 15:54:54,706 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:54,706 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@5917189a] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:54,706 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:54,706 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 
2019-09-16 15:54:54,769 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@11267482] will not be managed by Spring 
2019-09-16 15:54:54,769 DEBUG [board.boardTotal] ooo Using Connection [oracle.jdbc.driver.T4CConnection@11267482] 
2019-09-16 15:54:54,769 DEBUG [board.boardTotal] ==> Preparing: select count(*) from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') 
2019-09-16 15:54:54,769 DEBUG [board.boardTotal] ==> Parameters: (String), (String) 
2019-09-16 15:54:54,771 DEBUG [board.boardTotal] <== Total: 1 
2019-09-16 15:54:54,771 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@5917189a] 
2019-09-16 15:54:54,772 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource 
2019-09-16 15:54:54,774 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:54,774 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@1a1a0788] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:54,775 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:54,775 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 
2019-09-16 15:54:54,844 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@44e4f245] will not be managed by Spring 
2019-09-16 15:54:54,845 DEBUG [board.boardList] ooo Using Connection [oracle.jdbc.driver.T4CConnection@44e4f245] 
2019-09-16 15:54:54,845 DEBUG [board.boardList] ==> Preparing: select * from (select rownum rn, a.* from (select b., m.m_nick as m_nick, (select count() from bow_board_re br where b.brd_no=br.brd_no and re_del_yn='n') as replycount, length(brd_subject) as sublength from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') order by ref desc, re_step) a) where rn between ? and ? and BRD_DEL_YN = 'N' 
2019-09-16 15:54:54,845 DEBUG [board.boardList] ==> Parameters: (String), (String), 1(Integer), 10(Integer) 2019-09-16 15:54:54,848 DEBUG [board.boardList] <== Total: 0 
2019-09-16 15:54:54,848 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@1a1a0788] 
2019-09-16 15:54:54,848 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource 
2019-09-16 15:54:55,843 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:55,843 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@29dd751] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:55,844 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:55,844 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 
2019-09-16 15:54:55,903 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@7319bd07] will not be managed by Spring 
2019-09-16 15:54:55,903 DEBUG [board.boardTotal] ooo Using Connection [oracle.jdbc.driver.T4CConnection@7319bd07] 
2019-09-16 15:54:55,903 DEBUG [board.boardTotal] ==> Preparing: select count(*) from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') 
2019-09-16 15:54:55,904 DEBUG [board.boardTotal] ==> Parameters: (String), (String) 
2019-09-16 15:54:55,905 DEBUG [board.boardTotal] <== Total: 1 2019-09-16 15:54:55,905 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@29dd751] 
2019-09-16 15:54:55,905 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource 
2019-09-16 15:54:55,907 DEBUG [org.mybatis.spring.SqlSessionUtils] Creating a new SqlSession 
2019-09-16 15:54:55,908 DEBUG [org.mybatis.spring.SqlSessionUtils] SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@66de0334] was not registered for synchronization because synchronization is not active 
2019-09-16 15:54:55,908 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Fetching JDBC Connection from DataSource 
2019-09-16 15:54:55,908 DEBUG [org.springframework.jdbc.datasource.SimpleDriverDataSource] Creating new JDBC Driver Connection to [jdbc:oracle:thin:@127.0.0.1:1521:local] 
2019-09-16 15:54:55,980 DEBUG [org.mybatis.spring.transaction.SpringManagedTransaction] JDBC Connection [oracle.jdbc.driver.T4CConnection@1faeed6d] will not be managed by Spring 
2019-09-16 15:54:55,981 DEBUG [board.boardList] ooo Using Connection [oracle.jdbc.driver.T4CConnection@1faeed6d] 
2019-09-16 15:54:55,981 DEBUG [board.boardList] ==> Preparing: select * from (select rownum rn, a.* from (select b., m.m_nick as m_nick, (select count() from bow_board_re br where b.brd_no=br.brd_no and re_del_yn='n') as replycount, length(brd_subject) as sublength from bow_board b, bow_member m where b.m_no=m.m_no and (brd_subject like '%'||?||'%' or brd_content like '%'||?||'%') order by ref desc, re_step) a) where rn between ? and ? and BRD_DEL_YN = 'N' 
2019-09-16 15:54:55,981 DEBUG [board.boardList] ==> Parameters: (String), (String), 1(Integer), 10(Integer) 
2019-09-16 15:54:55,984 DEBUG [board.boardList] <== Total: 0 
2019-09-16 15:54:55,984 DEBUG [org.mybatis.spring.SqlSessionUtils] Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@66de0334] 
2019-09-16 15:54:55,984 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource

这是我的logstash.conf

 # all input will come from filebeat, no local logs
    input {
      beats {
        port => 5044
      }
    }
    # filter { }
    output {
        elasticsearch {
            hosts => "xxx.xxx.xxx.xxx:9200"
            manage_template => false
            index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
            ssl => false
            ssl_certificate_verification => false
        }
        stdout {
        }
    }

标签： logstash

解决方案

您可以编写一个grok过滤器来过滤掉日志消息

请检查此示例 conf 文件

filter{
    grok{
        match => {"message" => "%{DATESTAMP:time} %{LOGLEVEL:level} \[board.boardTotal\] %{GREEDYDATA:data}"}
    }

}

output{
    if "_grokparsefailure" in [tags] {
        # do something
    } else {
        elasticsearch{
          hosts => ["localhost:9200"]
          index => "log-with-boardtotal"

       }
    }

    stdout { 
       codec => "rubydebug"
    }
}

在上面的 conf 文件中，grok 将与board.boardTotal. 没有的行在标签中board.boardTotal会有一个。_grokparsefailure

这是一个简单的 grok，您可以根据需要更改。我只解析到board.boardTotal

谢谢

推荐阅读