时间戳

首页 > 解决方案 > 如何使用fluentd解析kubernetes pod输出的多个日志

kubernetes - 如何使用fluentd解析kubernetes pod输出的多个日志

问题描述

我尝试使用 Fluentd 将 EFK 堆栈实现到我们当前的环境中。

我有一个配置是

    <source>
      ...
      path /var/log/containers/*.log
      ...
    </source>

它应该获取工作节点上所有 pod 的所有标准输出。但是当我 ssh 进入该节点并检查输出格式时，我发现多行的标准输出日志被分成不同的日志条目，例如：

{"log":"Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client\n","stream":"stderr","time":"2021-10-29T18:26:26.011079366Z"}
{"log":"    at ServerResponse.setHeader (_http_outgoing.js:530:11)\n","stream":"stderr","time":"2021-10-29T18:26:26.011130167Z"}
{"log":"    at sendEtagResponse (/app/node_modules/next/dist/next-server/server/send-payload.js:6:12)\n","stream":"stderr","time":"2021-10-29T18:26:26.011145267Z"}
{"log":"    at sendData (/app/node_modules/next/dist/next-server/server/api-utils.js:32:479)\n","stream":"stderr","time":"2021-10-29T18:26:26.011229869Z"}
{"log":"    at ServerResponse.apiRes.send (/app/node_modules/next/dist/next-server/server/api-utils.js:6:250)\n","stream":"stderr","time":"2021-10-29T18:26:26.011242369Z"}
{"log":"    at exports.modules.3626.__webpack_exports__.default (/app/.next/server/pages/api/users/[id]/organizations.js:350:34)\n","stream":"stderr","time":"2021-10-29T18:26:26.011252769Z"}
{"log":"    at runMicrotasks (\u003canonymous\u003e)\n","stream":"stderr","time":"2021-10-29T18:26:26.011264269Z"}
{"log":"    at processTicksAndRejections (internal/process/task_queues.js:97:5)\n","stream":"stderr","time":"2021-10-29T18:26:26.011275069Z"}
{"log":"    at async apiResolver (/app/node_modules/next/dist/next-server/server/api-utils.js:8:1)\n","stream":"stderr","time":"2021-10-29T18:26:26.011284869Z"}
{"log":"    at async Server.handleApiRequest (/app/node_modules/next/dist/next-server/server/next-server.js:66:462)\n","stream":"stderr","time":"2021-10-29T18:26:26.01129647Z"}
{"log":"    at async Object.fn (/app/node_modules/next/dist/next-server/server/next-server.js:58:580) {\n","stream":"stderr","time":"2021-10-29T18:26:26.01130717Z"}
{"log":"  code: 'ERR_HTTP_HEADERS_SENT'\n","stream":"stderr","time":"2021-10-29T18:26:26.01131707Z"}
{"log":"}\n","stream":"stderr","time":"2021-10-29T18:26:26.01132747Z"}

然后将所有这些行分成单独的日志片段并传输到 Elasticsearch，有没有一种方法可以将这些多行合并为一个片段？

感谢任何形式的帮助。

标签： kubernetesloggingelastic-stackfluentd

解决方案

您可以使用多行插件来实现这一点。

这提供了format_firstline 可以使用正则表达式的参数。

您没有分享太多的常规日志输出，所以这里有一个带有格式的时间戳的示例YYYY-MM-dd HH:mm:ss,zzz

firstline: /\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2},\d{3}/

您也可以尝试在行首进行匹配，例如^(Info|Error).

这样，fluentd 会将多行识别为一个日志条目。

查看文档以获取有关配置插件的更多信息。

推荐阅读