docker - 如何调试 Telegraf 为什么不将数据从 MQTT 发送到 influxdb
问题描述
我正在尝试按如下方式设置 TIG 堆栈:
传感器数据 -> MQTT (Eclipse-Mosquitto) -> Telegraf -> InfluxDB -> Grafana
我还没有设置传感器数据的来源,但在其当前状态下,一切编译和运行都没有明显错误,而且一切似乎都已连接,但发布到 MQTT 主题的数据并没有最终出现在我的 Influxdb 数据库中。
$ sudo docker-compose up
Creating network "tig-samson-feature-setup-docker-compose_default" with the default driver
Creating mosquitto ... done
Creating influxdb ... done
Creating grafana ... done
Creating telegraf ... done
Attaching to influxdb, mosquitto, grafana, telegraf
mosquitto | 1583407265: mosquitto version 1.6.9 starting
mosquitto | 1583407265: Config loaded from /mosquitto/config/mosquitto.conf.
mosquitto | 1583407265: Opening ipv4 listen socket on port 1883.
mosquitto | 1583407265: Opening ipv6 listen socket on port 1883.
mosquitto | 1583407266: New connection from 192.168.32.1 on port 1883.
mosquitto | 1583407266: New client connected from 192.168.32.1 as mosqsub|5779-jackbridge (p1, c1, k60).
grafana | t=2020-03-05T11:21:05+0000 lvl=info msg="Starting Grafana" logger=server version=6.6.2 commit=3fa63cfc34 branch=HEAD compiled=2020-02-20T12:03:49+0000
grafana | t=2020-03-05T11:21:05+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini
grafana | t=2020-03-05T11:21:05+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini
grafana | t=2020-03-05T11:21:05+0000 lvl=info msg="Config overridden from command line" logger=settings arg="default.paths.data=/var/lib/grafana"
...continues indefinitely.. can add more detail if it's helpfuL!
下面是我的一些代码和我尝试过的一些事情
在 ubuntu VM 上运行的文件架构(在 mac 上的 virtualbox 中运行)
docker-compose.yml
telegraf.conf
influxdb (folder)
-----data (folder)
-----------data (locked folder auto generated)
-----------meta (locked folder auto generated)
-----------wal (locked folder auto generated)
data
----mosquitto (empty folder)
码头工人撰写
# # docker-compose.yml
version: '3'
services:
influxdb:
container_name: influxdb
image: influxdb
ports:
- "8086:8086"
environment:
- INFLUXDB_DB=sensors
- INFLUXDB_ADMIN_USER=admin
- INFLUXDB_ADMIN_PASSWORD=telegraf
volumes:
- ./influxdb/data:/var/lib/influxdb
restart: always
telegraf:
container_name: telegraf
image: telegraf
restart: always
depends_on:
- mosquitto
- influxdb
volumes:
- ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
grafana:
container_name: grafana
image: grafana/grafana
links:
- influxdb
hostname: grafana
ports:
- "3000:3000"
mosquitto:
container_name: mosquitto
image: eclipse-mosquitto
ports:
- "1883:1883"
- "9001:9001"
restart: always
volumes:
- ./mosquitto.conf:/mqtt/config:ro
# - /data/mosquitto:mosquitto/data
(我用 docker-compose up 启动程序)
电报文件
[[outputs.file]]
# ## Files to write to, "stdout" is a specially handled file.
files = ["stdout"]
# # Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
servers = ["tcp://localhost:1883"]
topics = [
"sensors"
]
data_format = "influx"
# ## MQTT broker URLs to be used. The format should be scheme://host:port,
# ## schema can be tcp, ssl, or ws.
# servers = ["tcp://127.0.0.1:1883"]
[[outputs.influxdb]]
urls = ["http://influxdb:8086"]
database = "sensors"
skip_database_creation = true
username = "telegraf"
password = "telegraf"
我可以访问 telegraf docker 容器并 ping influxdb,所以它似乎已连接
root@fc1d9af8f0c6:/# curl -sl -I http://influxdb:8086/ping
HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 12b4923a-5ec7-11ea-8006-0242c0a81002
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.7.10
X-Request-Id: 12b4923a-5ec7-11ea-8006-0242c0a81002
Date: Thu, 05 Mar 2020 09:52:49 GMT
我可以在其他终端发布和订阅 MQTT:
$ mosquitto_pub -t 'sensors' -m 'strength, site=room1 value=HELLO tony'
这是在另一个订阅传感器的终端中收到的
$ mosquitto_sub -t 'sensors' -v # note this is executed before mosquitto_pub
sensors strength, site=room1 value=HELLO tony
当我运行 mosquitto_sub 时,运行 docker-compose 的终端中没有任何活动。
当我运行 mosquitto_pub 时,客户端被检测为正在连接,但发布事件似乎没有注册
mosquitto | 1583407850: New connection from 192.168.32.1 on port 1883.
mosquitto | 1583407850: New client connected from 192.168.32.1 as mosqpub|7039-jackbridge (p1, c1, k60).
mosquitto | 1583407850: Client mosqpub|7039-jackbridge disconnected.
Telegraf 似乎正在向 Influxdb 发送指标。在 Telegraf [[outputs.file]] 中,我正在发送您上面的 files =['stdout']。目前尚不清楚指标是否到达 influxdb,因为我无法找到它们。
终端输出示例
telegraf | disk,device=sda1,fstype=ext4,host=foobar,mode=rw,path=/etc/resolv.conf inodes_used=189723i,total=10499674112i,free=3154145280i,used=6791987200i,used_percent=68.2877210177679,inodes_total=655360i,inodes_free=465637i 1583406223000000000
telegraf | cpu,cpu=cpu-total,host=foobar usage_steal=0,usage_system=7.14285714285916,usage_softirq=0.8281573498966311,usage_nice=0,usage_iowait=0.10351966873709728,usage_irq=0,usage_guest=0,usage_guest_nice=0,usage_user=56.41821946171202,usage_idle=35.507246376802996 1583406223000000000
telegraf | cpu,cpu=cpu0,host=foobar usage_user=56.41821946171202,usage_irq=0,usage_steal=0,usage_system=7.14285714285916,usage_idle=35.507246376802996,usage_nice=0,usage_iowait=0.10351966873709728,usage_softirq=0.8281573498966311,usage_guest=0,usage_guest_nice=0 1583406223000000000
telegraf | processes,host=foobar sleeping=2i,paging=0i,idle=0i,dead=0i,total_threads=11i,blocked=0i,zombies=0i,stopped=0i,running=0i,total=2i,unknown=0i 1583406223000000000
telegraf | 2020-03-05T11:03:53Z D! [outputs.file] Wrote batch of 26 metrics in 1.676587ms
telegraf | 2020-03-05T11:03:53Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
telegraf | 2020-03-05T11:03:53Z D! [outputs.influxdb] Wrote batch of 26 metrics in 8.885827ms
telegraf | 2020-03-05T11:03:53Z D! [outputs.influxdb] Buffer fullness: 23 / 10000 metrics
在 Influxdb 中创建了我的传感器数据库
compose$ sudo docker exec -it influxdb influx
[sudo] password for jackbridger:
Connected to http://localhost:8086 version 1.7.10
InfluxDB shell version: 1.7.10
> show databases
name: databases
name
----
sensors
_internal
> use _internal
> show stats
name: runtime
Alloc Frees HeapAlloc HeapIdle HeapInUse HeapObjects HeapReleased HeapSys Lookups Mallocs NumGC NumGoroutine PauseTotalNs Sys TotalAlloc
----- ----- --------- -------- --------- ----------- ------------ ------- ------- ------- ----- ------------ ------------ --- ----------
16426120 26535115 16426120 44695552 21823488 225331 28131328 66519040 0 26760446 277 26 6132353 72483064 1337594448
name: queryExecutor
queriesActive queriesExecuted queriesFinished queryDurationNs recoveredPanics
------------- --------------- --------------- --------------- ---------------
1 3 2 3462237 0
name: database
tags: database=_internal
numMeasurements numSeries
--------------- ---------
12 467
...continues for a while.. can add more detail if it's helpfuL!
我尝试为 mosquitto 设置音量,但无法正常工作 - 我认为这可能是个问题
任何人都可以就我如何调试它以找出为什么 Telegraf 没有将添加到 MQTT 的消息发送到 Influxdb 中的传感器数据库提供任何指导吗?
谢谢!
解决方案
以下是朋友提供的非常有帮助的,它解决了这个问题
# # Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
servers = ["tcp://localhost:1883"]
topics = [
"sensors"
]
data_format = "influx"
# ## MQTT broker URLs to be used. The format should be scheme://host:port,
# ## schema can be tcp, ssl, or ws.
# servers = ["tcp://127.0.0.1:1883"]
localhost:1883 会将主机解析为电报容器。如果你想引用在同一个 docker compose 设置中运行的另一个容器,你可以使用服务名作为主机名,所以在这种情况下尝试 tcp://mosquitto:1883。在这里阅读:https ://docs.docker.com/compose/networking/
每个容器现在都可以查找主机名 web 或 db 并取回相应容器的 IP 地址。例如,web 的应用程序代码可以连接到 URL postgres://db:5432 并开始使用 Postgres 数据库。
推荐阅读
- python - Airflow 中的 BigQuery 运算符未将 sql 作为原始文件读取
- c# - 如何在 C# 中使用数组和 DTO 编写漂亮的代码块?
- android - Android AIDL - 为原生代码生成标头 + 实现 (android NDK)
- keras - Keras 中的多类分割
- rust - 解析 nom 中一定大小的一个数
- ruby-on-rails - Rails 在 iOS 上使用 Safari 和 Mobile Safari 设计 Omniauth 问题
- android - 我正在尝试使用编辑文本 id,但它显示错误,你能帮帮我吗
- java - 如何在 JFrames 中使用 2 类绘制通过 Netbeans 中的 Palette 生成代码?
- apache - 切换到 Cloudflare DNS 会触发此错误“Unsupported X-Forwarded-Proto: https, https for URL” with mod_pagespeed 1.11.33.5-0 @26313
- python - 如何获取祖先标签相对于已故孩子的 href 属性
使用 Selenium 和 Python 进行标记
我正在尝试获取一个
<a>
元素,以防其嵌套(内部)<h3>
标签包含特定文本。我该怎么做?假设结构如下:<a href="https://example.com"> <section class="section">