kubernetes - 为什么 Kube-proxy 不工作？没有 DNS 解析
问题描述
我刚刚用 kubeadm 和 kubernetes 1.21 建立了一个新的集群。所有 pod 都已准备好标记。但我无法访问其中任何一个。深入研究问题后，似乎无法进行 DNS 解析。似乎 kube-proxy 不起作用。
这是 kube-proxy pod 的日志
I0712 05:50:46.511967 1 node.go:172] Successfully retrieved node IP: x.x.x.x
I0712 05:50:46.512039 1 server_others.go:140] Detected node IP x.x.x.x
W0712 05:50:46.512077 1 server_others.go:598] Unknown proxy mode "", assuming iptables proxy
I0712 05:50:46.545626 1 server_others.go:206] kube-proxy running in dual-stack mode, IPv4-primary
I0712 05:50:46.545672 1 server_others.go:212] Using iptables Proxier.
I0712 05:50:46.545692 1 server_others.go:219] creating dualStackProxier for iptables.
W0712 05:50:46.545715 1 server_others.go:512] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6
I0712 05:50:46.546089 1 server.go:643] Version: v1.21.2
I0712 05:50:46.549861 1 conntrack.go:52] Setting nf_conntrack_max to 196608
I0712 05:50:46.550300 1 config.go:224] Starting endpoint slice config controller
I0712 05:50:46.550338 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0712 05:50:46.550332 1 config.go:315] Starting service config controller
I0712 05:50:46.550354 1 shared_informer.go:240] Waiting for caches to sync for service config
W0712 05:50:46.553020 1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W0712 05:50:46.555115 1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
I0712 05:50:46.650614 1 shared_informer.go:247] Caches are synced for service config
I0712 05:50:46.650634 1 shared_informer.go:247] Caches are synced for endpoint slice config
W0712 05:57:14.556916 1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W0712 06:06:34.558550 1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
这是我正在运行的豆荚：
kube-system pod/coredns-558bd4d5db-qpf5m 1/1 Running 1 8h
kube-system pod/coredns-558bd4d5db-r5jwz 1/1 Running 0 8h
kube-system pod/etcd-master2 1/1 Running 3 20h
kube-system pod/kube-apiserver-master2 1/1 Running 3 20h
kube-system pod/kube-controller-manager-master2 1/1 Running 3 8h
kube-system pod/kube-flannel-ds-b7xrm 1/1 Running 0 8h
kube-system pod/kube-flannel-ds-hcn7f 1/1 Running 0 8h
kube-system pod/kube-flannel-ds-rx8j6 1/1 Running 1 8h
kube-system pod/kube-flannel-ds-wc2jc 1/1 Running 0 8h
kube-system pod/kube-proxy-48wmr 1/1 Running 0 25m
kube-system pod/kube-proxy-4gw8t 1/1 Running 0 25m
kube-system pod/kube-proxy-h9djp 1/1 Running 0 25m
kube-system pod/kube-proxy-r4k9t 1/1 Running 0 24m
kube-system pod/kube-scheduler-master2 1/1 Running 3 20h
命令kubectl run -it --rm --restart=Never busybox --image=gcr.io/google-containers/busybox nslookup kubernetes.default给我：
Address 1: x.x.x.x
nslookup: can't resolve 'kubernetes.default'
pod "busybox" deleted
pod default/busybox terminated (Error)
我的 iptables 规则：
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-NODEPORTS all -- anywhere anywhere /* kubernetes health check service ports */
KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL all -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- anywhere anywhere /* kubernetes forwarding rules */
KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- 10.244.0.0/16 anywhere
ACCEPT all -- anywhere 10.244.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
KUBE-FIREWALL all -- anywhere anywhere
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain KUBE-EXTERNAL-SERVICES (2 references)
target prot opt source destination
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- anywhere anywhere /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP all -- !127.0.0.0/8 127.0.0.0/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
Chain KUBE-FORWARD (1 references)
target prot opt source destination
DROP all -- anywhere anywhere ctstate INVALID
ACCEPT all -- anywhere anywhere /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-NODEPORTS (1 references)
target prot opt source destination
Chain KUBE-PROXY-CANARY (0 references)
target prot opt source destination
Chain KUBE-SERVICES (2 references)
target prot opt source destination
任何想法？
[编辑]
#kubectl edit cm -n kube-system kubelet-config-1.21
apiVersion: v1
data:
kubelet: |
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 0s
cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
#kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 22h
解决方案
Kube-proxy 它的网络服务。负责 DNS 解析的 DNS 提供者。如我所见，您已经安装了 coredns。检查您的 kubelet 配置。它应该指向正确的服务，并且该服务应该可以在您的 pod 中访问。另外请检查您的firewalld或iptables服务是否在所有节点上都被禁用。
像这样：
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
enabled: true
x509:
clientCAFile: "/var/lib/kubernetes/ca.pem"
authorization:
mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
- "10.33.0.10"
kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.33.0.10 <none> 53/UDP,53/TCP,9153/TCP 35h
接着：
kubectl exec -ti net-diag-86589fd8f5-r28qq -- nslookup kubernetes.default
Server: 10.33.0.10
Address: 10.33.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.33.0.1
UPD。
我刚刚注意到您将 Docker 作为容器运行时，将 flannel 作为网络提供者。根据我的理解，问题可能是 Docker 弄乱了您的 iptables 规则，尝试将所有 docker 规则设置为 prmissive 并查看它是否会起作用。
我不是 iptables 配置方面的大专家，但这样的事情可能会有所帮助：
https://unrouted.io/2017/08/15/docker-firewall/
此外，如果您使用法兰绒，请确保您使用的是正确的iface选项。如果您正在运行非云安装，这可能很关键。
https://github.com/flannel-io/flannel/blob/master/Documentation/configuration.md#key-command-line-options
推荐阅读
- graphql - 在graphql中，过滤同一属性的多个值并分页
- javascript - 使用中间件快速访问获取请求
- vba - Excel - VBA：Worksheet.Range.Delete 是否有列数限制？
- streamsets - Streamsets SQL Server 更改跟踪来源的奇怪行为
- php - PHP上传脚本在一段时间后停止/不返回
- java - Eclipselink IDE 支持
- c++ - 从 Eigen::Tensor 创建 tensorflow::Tensor
- highcharts - Highcharts，无内轴极坐标
- javascript - 在js中的函数内使用三元运算符
- agda - 为什么 Agda 给出错误“预期：ℕ，实际：ℕ”