kubernetes - Kubernetes 新工作节点的问题
问题描述
我通过添加节点到我的集群kubeadm join
一切似乎都很好!
我在仪表板上看到了这个节点,而 heattz 似乎还可以。
但是,如果我开始部署某些东西,我会遇到类似 whis 的错误:
Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1229e227176dfefd55c0728b10045bff158d99430843aca44c8957c2850127f2" network for pod "mobsf-76dd9dc786-zjjlx": NetworkPlugin cni failed to set up pod "" network: Get https://[10.233.0.1]:443/api/v1/namespaces/default: dial tcp 10.233.0.1:443: i/o timeout
我使用印花布作为插件
操作系统:Centos 7 内核:5.3.6-1。
节点:
kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 143d v1.15.3
s001dev-ceph-node-01 Ready worker 40d v1.15.3
s001dev-ceph-node-02 Ready worker 40d v1.15.3
s001dev-ceph-node-03 Ready worker 40d v1.15.3
worker1 Ready worker 137d v1.15.3
worker2 Ready worker 137d v1.15.3
worker3 Ready worker 137d v1.15.3
印花布:
``
kubectl get all --all-namespaces | grep -i calico
kube-system pod/calico-kube-controllers-6d57b44787-xlj89 1/1 Running 35 143d
kube-system pod/calico-node-flhx4 1/1 Running 5 40d
kube-system pod/calico-node-hhgzk 1/1 Running 30 143d
kube-system pod/calico-node-qp2dr 1/1 Running 6 40d
kube-system pod/calico-node-t698j 1/1 Running 183 40d
kube-system pod/calico-node-tk4mp 1/1 Running 323 137d
kube-system pod/calico-node-vg5qb 1/1 Running 7 40d
kube-system pod/calico-node-w7zvs 1/1 Running 325 137d
kube-system daemonset.apps/calico-node 7 7 7 7 7 <none> 143d
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 143d
kube-system replicaset.apps/calico-kube-controllers-6d57b44787 1 1 1 143d
来自第一个 pod pod/calico-node-flhx4 的 Calico 日志:
2020-02-03 06:55:38.094 [INFO][90] xdp_state.go 559: Finished XDP resync. family=4 resyncDuration=18.636208ms
2020-02-03 06:55:38.094 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=18.958571
2020-02-03 06:55:41.046 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:55:41.046 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:55:41.046 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:55:41.050 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.51651ms
2020-02-03 06:55:41.050 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.949963
2020-02-03 06:55:51.369 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:55:51.369 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:55:51.370 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:55:51.373 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.50255ms
2020-02-03 06:55:51.373 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=4.002705
2020-02-03 06:56:00.293 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:00.293 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="nat"
2020-02-03 06:56:00.295 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="nat"
2020-02-03 06:56:00.305 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=11.403867
2020-02-03 06:56:01.433 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:01.433 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:01.433 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:01.437 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.505643ms
2020-02-03 06:56:01.437 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.87718
2020-02-03 06:56:10.788 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:10.788 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="raw"
2020-02-03 06:56:10.790 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="raw"
2020-02-03 06:56:10.793 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=5.292443
2020-02-03 06:56:12.341 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:12.342 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:12.342 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:12.344 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=1.878346ms
2020-02-03 06:56:12.344 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=2.1785340000000004
2020-02-03 06:56:21.277 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:21.277 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="filter"
2020-02-03 06:56:21.281 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="filter"
2020-02-03 06:56:21.286 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=8.782817
2020-02-03 06:56:22.914 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:22.914 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:22.914 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:22.917 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=2.961011ms
2020-02-03 06:56:22.917 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.528314
从节点worker1和worker2我可以ping 10.233.0.1(API),但不能在新节点worker3:
ping 10.233.0.1
PING 10.233.0.1 (10.233.0.1) 56(84) bytes of data.
From 10.254.0.4 icmp_seq=1 Time to live exceeded
From 10.254.0.4 icmp_seq=2 Time to live exceeded
我还注意到 promlem 节点上缺少印花布网络接口
解决方案
我解决了这个问题!问题是问题节点上的 nginx-proxy pod 处于待处理状态
推荐阅读
- sql - 基于事务的 SQL 分析
- excel - 从 SpinButton 获取值以确定循环数
- java - 在 Eclipse 中运行 Selenium 时在哪里可以找到 System.out.println 输出?
- php - 即使值存在,也试图获取非对象的属性“月”
- dictionary - 如何将多个重复节点连接成一个节点 - BizTalk
- javascript - 从 ajax 请求将嵌套数据加载到制表器表
- oracle - 当我启动 toad 应用程序时出现错误
- c# - 如何在方法参数中引用变量?
- javascript - 机器人不响应前缀
- reactjs - 使用锚按钮(不删除锚)滚动 fullpage.js 的“添加冷却时间”的干净方法?