kubernetes - fluent bit kubernetes注解不起作用

Question

我将fluent bit部署到kubernetes。我部署了一个带有 annotation 的 pod fluentbit.io/parser: cri。ivyxjc但它仍然使用 .config中的解析器解析日志INPUT。

流利的位配置

apiVersion: v1
kind: ConfigMap
metadata:
    name: fluent-bit-config
    labels:
        k8s-app: fluent-bit
data:
    # Configuration files: server, input, filters and output
    # ======================================================
    fluent-bit.conf: |
        [SERVICE]
            Flush         1
            Log_Level     info
            Daemon        off
            Parsers_File  parsers.conf
            HTTP_Server   On
            HTTP_Listen   0.0.0.0
            HTTP_Port     2020

        @INCLUDE input-kubernetes.conf
        @INCLUDE filter-kubernetes.conf
        @INCLUDE output-elasticsearch.conf

    input-kubernetes.conf: |
        [INPUT]
            Name              tail
            Tag               kube.*
            Path              /var/log/containers/*.log
            Parser            ivyxjc
            DB                /var/log/flb_kube.db
            Mem_Buf_Limit     200MB
            Skip_Long_Lines   On
            Refresh_Interval  10

    filter-kubernetes.conf: |
        [FILTER]
            Name                kubernetes
            Match               kube.*
            Kube_URL            https://kubernetes.default.svc:443
            Kube_CA_File        /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
            Kube_Token_File     /var/run/secrets/kubernetes.io/serviceaccount/token
            Kube_Tag_Prefix     kube.var.log.containers.
            Merge_Log           On
            k8s-logging.parser  On
            K8S-Logging.Exclude On

    output-elasticsearch.conf: |
        [OUTPUT]
            Name            es
            Match           *
            Host            ${FLUENT_ELASTICSEARCH_HOST}
            Port            ${FLUENT_ELASTICSEARCH_PORT}
            HTTP_User       ${FLUENT_ELASTICSEARCH_USER}
            HTTP_Passwd     ${FLUENT_ELASTICSEARCH_PASSWD}
            Logstash_Format On
            Replace_Dots    On
            Retry_Limit     False
            tls             On
            tls.verify      Off

    parsers.conf: |
        [PARSER]
            # http://rubular.com/r/tjUt3Awgg4
            Name cri
            Format regex
            Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
            Time_Key    time
            Time_Format %Y-%m-%dT%H:%M:%S.%L%z

        [PARSER]
            Name ivyxjc
            Format regex
            Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag2>[^ ]*) (?<message2>.*)$
            Time_Key    time
            Time_Format %Y-%m-%dT%H:%M:%S.%L%z

吊舱说明：

...
Name:         logger-5c6658b5dd-66zkw
Namespace:    logger
Priority:     0
Start Time:   Fri, 15 Oct 2021 15:28:47 +0800
Labels:       app=logger
              pod-template-hash=5c6658b5dd
Annotations:  fluentbit.io/parser: cri
              fluentbit.io/parser_stderr: cri
              fluentbit.io/parser_stdout: cri
Status:       Running
...

标签： kubernetesfluent-bit

Answer 1

k8s-logging.parser

此选项告诉流利的位代理使用注释中的解析器，该注释将用于“log”关键字。INPUT 解析器将照常应用。然后“KUBERNETES”过滤器获取输入，然后由“fluentbit.io/parser: parser_name_here”指定的解析器将从“log”关键字中获取值。

参考文档。

不幸的是，这在某些情况下也不起作用，我还无法追查它，但如果我确实找到了一些东西，我会更新这个答案。