database - 集群中的TDengine服务已经启动，但是客户端连接不上

Question

在 2 个 vm 中具有 2 个 TDengine 实例的集群。集群在过去 3 个月内运行良好，甚至昨天还可以，但今天早上无法连接。

TDengine 版本：TDengine-server-2.2.0.0-Linux-x64.rpm

FQDN：192.168.44.127 hadoop1 192.168.44.130 hadoop2

登录节点1：

11/11 11:00:21.720098 00076468 SYN vgId:1, nodeId:3, 检查角色自对等点：离线 sstatus:init, self:unsynced sstatus:init 11/11 11:00:21.720117 00076468 SYN vgId:1, nodeId: 3、发送状态消息失败，重启 fd:-1 11/11 11:00:21.720121 00076468 SYN vgId:1, nodeId:3, 重启对等连接，最后 sstatus:init 11/11 11:00:21.720123 00076468 SYN vgId :1, nodeId:3, pfd:-1 sfd:-1 将关闭 11/11 11:00:21.720130 00076468 SYN vgId:1, nodeId:3, peer conn 重新启动并设置 sstatus:init 11/11 11: 00：21.720132 00076468 SYN vgId：1，nodeId：3，在 1000 毫秒内检查对等连接 11/11 11：00：21.720136 00076468 SYN vgId：1，nodeId：3，对等角色：离线更改为离线 11/11 11:00 ：21.720137 00076468 SYN vgId：1，选择master，replica：2 11/11 11：00：21.720139 00076468 SYN vgId：1，自onlineNum：1 replica：2 11/11 11:00:21以来没有master选举。720141 00076468 SYN vgId:1，未能选择主节点 11/11 11:00:21.780690 00076554 DND 用户：监视器，向 mnodes 发送身份验证消息 11/11 11:00:21.782719 00076481 RPC WARN DND-C 0x7f12a2许多重定向，退出 11/11 11:00:21.782774 00076554 DND 错误用户：监视器，从 mnodes 收到的身份验证消息，错误：无法建立连接 11/11 11:00:21.804686 00076480 RPC WARN DND-C 0x7f12a2ef8188（无），重定向太多，退出 11/11 11:00:21.804729 00076480 DND minfos 已更改，numOfEps:2 inUse:0 11/11 11:00:21.804735 00076480 DND minfo:0 hadoop1:6030 11/11 11:00:271.804740 00免打扰 minfo:1 hadoop2:6030 11/11 11:00:21.804747 00076480 收到免打扰错误状态 rsp，错误：无法建立连接 11/11 11:00:22.086289 00076554 免打扰用户：监视器，向 mnodes 发送身份验证消息 11/ 11 时 11 分 00 分 22 秒。088553 00076481 RPC WARN DND-C 0x7f12a2ef8478（nil），重定向太多，退出 11/11 11:00:22.088641 00076554 DND 错误用户：监视器，从 mnodes 收到的身份验证消息，错误：无法建立连接

登录节点2：

11/11 11:12:41.216128 00002867 DND minfo:0 hadoop1:6030 11/11 11:12:41.216133 00002867 DND minfo:1 hadoop2:6030 11/11 11:12:41.216139 00002867 收到 DND 错误，状态错误：无法建立连接 11/11 11:12:41.984212 00002859 来自 ip:192.168.44.127 的 SYN 对等 TCP 连接 11/11 11:12:41.984238 00002859 SYN ERROR vgId:1, vgId 找不到 11/11 11:12: 42.223138 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:42.223198 00002867 DND minfos 已更改，numOfEps:2 inUse:0 11/11 11:12:42.223206 0002.223206 00 hadoop1:6030 11/11 11:12:42.223214 00002867 DND minfo:1 hadoop2:6030 11/11 11:12:42.223223 00002867 DND ERROR 状态 rsp 收到，错误：无法建立连接 11/11 11:12:4002.990131 00来自 ip:192.168.44.127 11/11 11:12:42 的 SYN 对等 TCP 连接。990158 00002859 SYN ERROR vgId:1, vgId 找不到 11/11 11:12:43.227361 00002867 RPC WARN DND-C 0x7f88d5ef7300 (nil), 重定向太多，退出 11/11 11:12:43.227NDfofo 已更改 000028 分钟, numOfEps:2 inUse:1 11/11 11:12:43.227433 00002867 DND minfo:0 hadoop1:6030 11/11 11:12:43.227436 00002867 DND minfo:1 hadoop2:6030 11/11 11:12:43.227439 DND 错误086收到状态 rsp，错误：无法建立连接 11/11 11:12:43.994597 00002859 SYN peer TCP connection from ip:192.168.44.127 11/11 11:12:43.994623 00002859 SYN ERROR vgId:1, vgId 找不到11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:44.232905 00002867 DND minfos 已更改，numOfEps:2 inUse:0重定向过多，退出 11/11 11:12:43.227429 00002867 DND minfos 已更改，numOfEps:2 inUse:1 11/11 11:12:43.227433 00002867 DND minfo:0 hadoop1:6030 11/11 11:12:4208676 000 DND mininfo:1 hadoop2:6030 11/11 11:12:43.227439 00002867 DND ERROR status rsp is received, error:Unable to setup connection 11/11 11:12:43.994597 00002859 SYN peer TCP connection from ip:192.168.44.127 11/ 11 11:12:43.994623 00002859 SYN ERROR vgId:1，找不到 vgId 11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:45。 00002867 DND minfos 已更改，numOfEps:2 inUse:0重定向过多，退出 11/11 11:12:43.227429 00002867 DND minfos 已更改，numOfEps:2 inUse:1 11/11 11:12:43.227433 00002867 DND minfo:0 hadoop1:6030 11/11 11:12:4208676 000 DND mininfo:1 hadoop2:6030 11/11 11:12:43.227439 00002867 DND ERROR status rsp is received, error:Unable to setup connection 11/11 11:12:43.994597 00002859 SYN peer TCP connection from ip:192.168.44.127 11/ 11 11:12:43.994623 00002859 SYN ERROR vgId:1，找不到 vgId 11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:45。 00002867 DND minfos 已更改，numOfEps:2 inUse:0227439 00002867 收到 DND 错误状态 rsp，错误：无法建立连接 11/11 11:12:43.994597 00002859 SYN 对等 TCP 连接来自 ip:192.168.44.127 11/11 11:12:43.994623 0000:1859 SYN 错误 vgId找不到 11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:44.232905 00002867 DND minfos 已更改，numOfEps:2 inUse:0227439 00002867 收到 DND 错误状态 rsp，错误：无法建立连接 11/11 11:12:43.994597 00002859 SYN 对等 TCP 连接来自 ip:192.168.44.127 11/11 11:12:43.994623 0000:1859 SYN 错误 vgId找不到 11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil)，重定向太多，退出 11/11 11:12:44.232905 00002867 DND minfos 已更改，numOfEps:2 inUse:0

登录节点1：

11/11 11:00:21.720098 00076468 SYN vgId:1, nodeId:3, check roles since peer:offline sstatus:init, self:unsynced sstatus:init 
11/11 11:00:21.720117 00076468 SYN vgId:1, nodeId:3, failed to send status msg, restart fd:-1 
11/11 11:00:21.720121 00076468 SYN vgId:1, nodeId:3, restart peer connection, last sstatus:init 
11/11 11:00:21.720123 00076468 SYN vgId:1, nodeId:3, pfd:-1 sfd:-1 will be closed 
11/11 11:00:21.720130 00076468 SYN vgId:1, nodeId:3, peer conn is restart and set sstatus:init 
11/11 11:00:21.720132 00076468 SYN vgId:1, nodeId:3, check peer connection in 1000 ms 
11/11 11:00:21.720136 00076468 SYN vgId:1, nodeId:3, peer role:offline change to offline 
11/11 11:00:21.720137 00076468 SYN vgId:1, choose master, replica:2 
11/11 11:00:21.720139 00076468 SYN vgId:1, no master election since onlineNum:1 replica:2 
11/11 11:00:21.720141 00076468 SYN vgId:1, failed to choose master 
11/11 11:00:21.780690 00076554 DND user:monitor, send auth msg to mnodes 
11/11 11:00:21.782719 00076481 RPC WARN DND-C 0x7f12a2ef85f0 (nil), too many redirects, quit 
11/11 11:00:21.782774 00076554 DND ERROR user:monitor, auth msg received from mnodes, error:Unable to establish connection 
11/11 11:00:21.804686 00076480 RPC WARN DND-C 0x7f12a2ef8188 (nil), too many redirects, quit 
11/11 11:00:21.804729 00076480 DND minfos is changed, numOfEps:2 inUse:0 
11/11 11:00:21.804735 00076480 DND minfo:0 hadoop1:6030 
11/11 11:00:21.804740 00076480 DND minfo:1 hadoop2:6030 
11/11 11:00:21.804747 00076480 DND ERROR status rsp is received, error:Unable to establish connection 
11/11 11:00:22.086289 00076554 DND user:monitor, send auth msg to mnodes 
11/11 11:00:22.088553 00076481 RPC WARN DND-C 0x7f12a2ef8478 (nil), too many redirects, quit 
11/11 11:00:22.088641 00076554 DND ERROR user:monitor, auth msg received from mnodes, error:Unable to establish connection 

登录节点2：

11/11 11:12:41.216128 00002867 DND minfo:0 hadoop1:6030 
11/11 11:12:41.216133 00002867 DND minfo:1 hadoop2:6030 
11/11 11:12:41.216139 00002867 DND ERROR status rsp is received, error:Unable to establish connection 
11/11 11:12:41.984212 00002859 SYN peer TCP connection from ip:192.168.44.127 
11/11 11:12:41.984238 00002859 SYN ERROR vgId:1, vgId could not be found 
11/11 11:12:42.223138 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil), too many redirects, quit 
11/11 11:12:42.223198 00002867 DND minfos is changed, numOfEps:2 inUse:0 
11/11 11:12:42.223206 00002867 DND minfo:0 hadoop1:6030 
11/11 11:12:42.223214 00002867 DND minfo:1 hadoop2:6030 
11/11 11:12:42.223223 00002867 DND ERROR status rsp is received, error:Unable to establish connection 
11/11 11:12:42.990131 00002859 SYN peer TCP connection from ip:192.168.44.127 
11/11 11:12:42.990158 00002859 SYN ERROR vgId:1, vgId could not be found 
11/11 11:12:43.227361 00002867 RPC WARN DND-C 0x7f88d5ef7300 (nil), too many redirects, quit 
11/11 11:12:43.227429 00002867 DND minfos is changed, numOfEps:2 inUse:1 
11/11 11:12:43.227433 00002867 DND minfo:0 hadoop1:6030 
11/11 11:12:43.227436 00002867 DND minfo:1 hadoop2:6030 
11/11 11:12:43.227439 00002867 DND ERROR status rsp is received, error:Unable to establish connection 
11/11 11:12:43.994597 00002859 SYN peer TCP connection from ip:192.168.44.127 
11/11 11:12:43.994623 00002859 SYN ERROR vgId:1, vgId could not be found 
11/11 11:12:44.232866 00002867 RPC WARN DND-C 0x7f88d5ef91d8 (nil), too many redirects, quit 
11/11 11:12:44.232905 00002867 DND minfos is changed, numOfEps:2 inUse:0 

不知道发生了什么。

标签： databasetime-seriestd-engine