时间戳

首页 > 解决方案 > 从 Prometheus alertmanager 向 Google 聊天室发送通知

kubernetes - 从 Prometheus alertmanager 向 Google 聊天室发送通知

问题描述

我们最近开始在我们的组织中使用 Google Chat，因此我们需要将警报从 Prometheus-Alertmanager 发送到 Google 聊天室。下面是我用于 Google 聊天的 alertmanager 配置

receiver:
    - name: 'gchat-receiver'
      webhook_configs:
        - url: "https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
          send_resolved: false

当我们实施上述配置时，我们的警报管理器日志中出现以下错误：

level=error ts=2021-02-23T06:23:43.931Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.971Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.976Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=21 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.994Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"

对此的任何帮助或指导将不胜感激

标签： kubernetesprometheusprometheus-alertmanagergoogle-chat

解决方案

这是因为 Google Chat 期望将特定的 JSON 有效负载模式发布到该 webhook 地址，而默认的Alertmanager 有效负载与它不匹配，因此您会收到HTTP 400响应：错误请求。

您可能需要在两者之间放置一个自定义应用程序，以将 JSON 转换为 Google Chat 可接受的格式。Github 中已经有一些符合该一般标准的存储库。（例如calert）

推荐阅读