logging - 在 Fluentd 中将字符串参数解析为 json

Question

我正在使用 Fluentd 从文件中流式传输日志并将它们推送到 MongoDB。我做了一些如下配置，但无法将字符串提取为 json。

这是系统写入文件的示例日志

2021-10-11T12:29:08.0328851Z stdout F {"level":"info","msg":"[map[counter:148] Basic counter is counting]","time":"2021-10-11T12:29:08.032Z"}

这是流畅的配置

<source>
  @type tail
  path /var/log/containers/*.log
  pos_file fluentd-docker.pos
  read_from_head true
  tag kubernetes.*
  exclude_path ["/var/log/containers/fluent*"]
  <parse>
  @type multi_format
  <pattern>
    format json
    time_key time
    time_type string
    time_format "%Y-%m-%dT%H:%M:%S.%NZ"
    keep_time_key false
  </pattern>
  <pattern>
    format regexp
    expression /^(?<time>.+) (?<stream>stdout|stderr)( (?<logtag>.))? (?<log>.*)$/
    time_format '%Y-%m-%dT%H:%M:%S.%N%:z'
    keep_time_key false
  </pattern>
  </parse>
</source>
<match **>
  @type mongo  

  host "#{ENV['FLUENT_MONGO_HOST']}"
  port "#{ENV['FLUENT_MONGO_PORT']}"

  database admin
  collection logs

  # for capped collection  
  capped
  capped_size 1024m

  # authentication
  user "#{ENV['FLUENT_MONGO_USER']}"
  password "#{ENV['FLUENT_MONGO_PASSWORD']}"

  <inject>    
    # key name of timestamp    
    time_key time  
  </inject>  
  
  <buffer>    
    # flush    
    flush_interval 10s  
  </buffer>
</match>

这是输出

我想丢弃流、日志标签和日志字段。

这就是我要的：

{
   "level" : "info",
   "msg" : "Test message"
   "time" : 2021-10-11T12:46:15.376+00:00
}

标签： loggingfluentd