elasticsearch - jboss-eap 的 Fluentd 正则表达式

Question

我正在尝试从 Jboss-eap 服务器收集日志然后发送到 Elasticsearch，我在服务器上使用 td-agent，它在读取我的输入和正则表达式时总是说 unmatched_lines

我的输入配置

<source>
  @type tail
  read_from_head true
  tag file-jboss.log
  path C:\T24\JBOSS\jboss-eap-6.4\standalone\log\server.log
  #C:\T24\TAFJ\log\database.log
  pos_file c:\opt\td-agent\file-jboss.pos
  <parse>
    @type regexp
    expression /(?<time>\d{2}\:\d{2}\:\d{2}\,\d{3})\s+\s*(?<level>\w{1,6})\s+(?<service>\[[a-zA-Z.]*\])\s*(?<thread>\(.*\))\s+(?<message>[A-Za-z0-9]*\:.*)/
    
  </parse>
</source>

<filter file-jboss.log>
  @type record_transformer
  <record>
    host_param "#{Socket.gethostname}"
  </record>
</filter>

我使用了这个正则表达式，它并不匹配所有内容，但我需要的是匹配的

(?<time>\d{2}\:\d{2}\:\d{2}\,\d{3})\s+\s*(?<level>\w{1,6})\s+(?<service>\[[a-zA-Z.]*\])\s*(?<thread>\(.*\))\s+(?<message>[A-Za-z0-9]*\:.*)

匹配样本的日志

11:12:09,587 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014775:    New missing/unsatisfied dependencies:
      service jboss.naming.context.java.ConnectionFactory (missing) dependents: [service jboss.naming.context.java.comp.TAFJJEE_EAR.TAFJJEE_EJB.ARCMOBProcessingBean.env.jms.TopicConnectionFactory, service jboss.naming.context.java.comp.TAFJJEE_EAR.TAFJJEE_MDB.TAFJPhantomListenerMDB.env.jms.TAFJQueueConnectionFactory, service jboss.naming.context.java.comp.TAFJJEE_EAR.TAFJJEE_EJB.AMLProcessingBean.env.jms.TopicConnectionFactory, service jboss.naming.context.java.module.BrowserWeb.BrowserWeb.env.jms.jmsConnectionFactory, JBAS014799: ... and 6 more ] 
11:12:13,447 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-39) JBAS015974: Stopped subdeployment (runtime-name: TAFJJEE_EJB.jar) in 135ms
11:12:13,447 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-37) JBAS015974: Stopped subdeployment (runtime-name: TAFJJEE_MDB.jar) in 135ms

Fluentd 日志输出

2021-10-13 11:55:21 +0100 [info]: using configuration file: <ROOT>
  <source>
    @type tail
    read_from_head true
    tag "file-jboss.log"
    path "C:\\T24\\JBOSS\\jboss-eap-6.4\\standalone\\log\\server.log"
    pos_file "c:\\opt\\td-agent\\file-jboss.pos"
    <parse>
      @type "regexp"
      expression /(?<time>\d{2}\:\d{2}\:\d{2}\,\d{3})\s+\s*(?<level>\w{1,6})\s+(?<service>\[[a-zA-Z.]*\])\s*(?<thread>\(.*\))\s+(?<mes
sage>[A-Za-z0-9]*\:.*)/
      unmatched_lines
    </parse>
  </source>
  <filter file-jboss.log>
    @type record_transformer
    <record>
      host_param T24-MERCURY
    </record>
  </filter>
  <match file-jboss.log>
    @type file
    path "c:\\opt\\td-agent\\output\\tafj.log"
    <buffer time>
      path "c:\\opt\\td-agent\\output\\tafj.log"
    </buffer>
  </match>
</ROOT>
2021-10-13 11:55:21 +0100 [info]: starting fluentd-1.13.3 pid=3192 ruby="2.7.4"
2021-10-13 11:55:21 +0100 [info]: spawn command to main:  cmdline=["C:/opt/td-agent/bin/ruby.exe", "-Eascii-8bit:ascii-8bit", "C:/opt/
td-agent/bin/fluentd", "--under-supervisor"]
2021-10-13 11:55:24 +0100 [info]: adding filter pattern="file-jboss.log" type="record_transformer"
2021-10-13 11:55:24 +0100 [info]: adding match pattern="file-jboss.log" type="file"
2021-10-13 11:55:24 +0100 [info]: adding source type="tail"
2021-10-13 11:55:24 +0100 [info]: #0 starting fluentd worker pid=8120 ppid=3192 worker=0
2021-10-13 11:55:24 +0100 [info]: #0 following tail of C:\T24\JBOSS\jboss-eap-6.4\standalone\log\server.log
2021-10-13 11:55:24 +0100 [info]: #0 fluentd worker is now running worker=0

知道为什么 fluentd 不喜欢 regex 吗？

标签： elasticsearchjbosskibanafluentd