kubernetes - Metallb LoadBalancer 卡在待处理状态
问题描述
我正在尝试设置一个裸机 Kubernetes 服务器，包括 metallb LoadBalancer。因此，我按照 kind 项目网站上发布的说明进行操作https://kind.sigs.k8s.io/docs/user/loadbalancer/。为了测试我的安装，我尝试部署默认的nginxWeb 服务器并通过浏览器访问它。不幸的是，创建的 LoadBalancer 类型的服务卡在EXTERNAL IP <Pending>. 我会附上尽可能多的信息。我希望有人可以帮助我解决这个问题。如果您需要更多信息，请告诉我。
这就是我所做的：
- 根据安装 metallb LoadBalancer
https://kind.sigs.k8s.io/docs/user/loadbalancer/ - 通过创建 nginx 部署
kubectl create deployment nginx --image nginx - 创建了相应的服务
kubectl expose deployment/nginx --type="LoadBalancer" --port 8080
输出
版本和节点
$ kubectl version --short
Client Version: v1.20.2
Server Version: v1.20.2
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gemini Ready control-plane,master 3d v1.20.2 192.168.2.203 <none> Ubuntu 20.10 5.8.0-1013-raspi docker://19.3.13
phoenix Ready <none> 2d23h v1.20.2 192.168.2.129 <none> Ubuntu 20.10 5.8.0-1013-raspi docker://19.3.13
taurus Ready <none> 2d23h v1.20.2 192.168.2.201 <none> Ubuntu 20.10 5.8.0-1013-raspi docker://19.3.13
virgo Ready <none> 2d23h v1.20.2 192.168.2.202 <none> Ubuntu 20.10 5.8.0-1013-raspi docker://19.3.13
所有命名空间
$ kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/nginx-6799fc88d8-62cjd 1/1 Running 0 59m
kube-system pod/calico-kube-controllers-86bddfcff-ccrhg 1/1 Running 5 3d
kube-system pod/calico-node-djt5s 0/1 CrashLoopBackOff 1007 3d
kube-system pod/calico-node-jddnl 1/1 Running 3 3d
kube-system pod/calico-node-nxwlw 1/1 Running 5 3d
kube-system pod/calico-node-zrxzl 1/1 Running 3 3d
kube-system pod/coredns-74ff55c5b-kb2nm 1/1 Running 5 3d1h
kube-system pod/coredns-74ff55c5b-wsgs5 1/1 Running 5 3d1h
kube-system pod/etcd-gemini 1/1 Running 6 3d1h
kube-system pod/kube-apiserver-gemini 1/1 Running 6 3d1h
kube-system pod/kube-controller-manager-gemini 1/1 Running 6 3d1h
kube-system pod/kube-proxy-7fcjz 1/1 Running 6 3d1h
kube-system pod/kube-proxy-84rr7 1/1 Running 3 3d
kube-system pod/kube-proxy-lc88w 1/1 Running 3 3d
kube-system pod/kube-proxy-v4qd9 1/1 Running 3 3d
kube-system pod/kube-scheduler-gemini 1/1 Running 6 3d1h
kubernetes-dashboard pod/dashboard-metrics-scraper-79c5968bdc-mlb4s 1/1 Running 5 3d
kubernetes-dashboard pod/kubernetes-dashboard-7448ffc97b-nq5c9 1/1 Running 5 3d
metallb-system pod/controller-5c797bc4f8-2zx7l 1/1 Running 0 101m
metallb-system pod/speaker-5kxc9 1/1 Running 0 101m
metallb-system pod/speaker-b6lbh 1/1 Running 0 101m
metallb-system pod/speaker-c8sr7 1/1 Running 0 101m
metallb-system pod/speaker-fwbgt 1/1 Running 1 101m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h
default service/nginx LoadBalancer 10.110.25.14 <pending> 8080:30444/TCP 24m
kube-system service/calico-etcd ClusterIP 10.96.232.136 <none> 6666/TCP 3d
kube-system service/calico-typha ClusterIP 10.109.108.233 <none> 5473/TCP 3d
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 3d1h
kubernetes-dashboard service/dashboard-metrics-scraper ClusterIP 10.110.70.52 <none> 8000/TCP 3d
kubernetes-dashboard service/kubernetes-dashboard NodePort 10.106.194.127 <none> 443:31741/TCP 3d
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/calico-node 4 4 3 4 3 kubernetes.io/os=linux 3d
kube-system daemonset.apps/kube-proxy 4 4 4 4 4 kubernetes.io/os=linux 3d1h
metallb-system daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 101m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default deployment.apps/nginx 1/1 1 1 59m
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 3d
kube-system deployment.apps/coredns 2/2 2 2 3d1h
kubernetes-dashboard deployment.apps/dashboard-metrics-scraper 1/1 1 1 3d
kubernetes-dashboard deployment.apps/kubernetes-dashboard 1/1 1 1 3d
metallb-system deployment.apps/controller 1/1 1 1 101m
NAMESPACE NAME DESIRED CURRENT READY AGE
default replicaset.apps/nginx-6799fc88d8 1 1 1 59m
kube-system replicaset.apps/calico-kube-controllers-56b44cd6d5 0 0 0 3d
kube-system replicaset.apps/calico-kube-controllers-86bddfcff 1 1 1 3d
kube-system replicaset.apps/coredns-74ff55c5b 2 2 2 3d1h
kubernetes-dashboard replicaset.apps/dashboard-metrics-scraper-79c5968bdc 1 1 1 3d
kubernetes-dashboard replicaset.apps/kubernetes-dashboard-7448ffc97b 1 1 1 3d
metallb-system replicaset.apps/controller-5c797bc4f8 1 1 1 101m
命名空间默认值：
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-6799fc88d8-62cjd 1/1 Running 0 29m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h
service/nginx LoadBalancer 10.110.25.14 <pending> 8080:30444/TCP 29m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 29m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-6799fc88d8 1 1 1 29m
命名空间 metallb 系统：
$ kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-5c797bc4f8-2zx7l 1/1 Running 0 73m
pod/speaker-5kxc9 1/1 Running 0 73m
pod/speaker-b6lbh 1/1 Running 0 73m
pod/speaker-c8sr7 1/1 Running 0 73m
pod/speaker-fwbgt 1/1 Running 1 73m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 73m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 73m
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-5c797bc4f8 1 1 1 73m
配置图
$ kubectl describe configmap config -n metallb-system
Name: config
Namespace: metallb-system
Labels: <none>
Annotations: <none>
Data
====
config:
----
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.2.210-192.168.2.250
Events: <none>
Kube 代理
$ kubectl describe configmap -n kube-system kube-proxy
Name: kube-proxy
Namespace: kube-system
Labels: app=kube-proxy
Annotations: kubeadm.kubernetes.io/component-config.hash: sha256:038b6c93b0ae1da89eb9c0c589c8e3439e9c91b849975435c29f905041b9b9fc
Data
====
config.conf:
----
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
bindAddressHardFail: false
clientConnection:
acceptContentTypes: ""
burst: 0
contentType: ""
kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
qps: 0
clusterCIDR: 192.168.0.0/16
configSyncPeriod: 0s
conntrack:
maxPerCore: null
min: null
tcpCloseWaitTimeout: null
tcpEstablishedTimeout: null
detectLocalMode: ""
enableProfiling: false
healthzBindAddress: ""
hostnameOverride: ""
iptables:
masqueradeAll: false
masqueradeBit: null
minSyncPeriod: 0s
syncPeriod: 0s
ipvs:
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
strictARP: true
syncPeriod: 0s
tcpFinTimeout: 0s
tcpTimeout: 0s
udpTimeout: 0s
kind: KubeProxyConfiguration
metricsBindAddress: ""
mode: ""
nodePortAddresses: null
oomScoreAdj: null
portRange: ""
showHiddenMetricsForVersion: ""
udpIdleTimeout: 0s
winkernel:
enableDSR: false
networkName: ""
sourceVip: ""
kubeconfig.conf:
----
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
server: https://192.168.2.203:6443
name: default
contexts:
- context:
cluster: default
namespace: default
user: default
name: default
current-context: default
users:
- name: default
user:
tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
Events: <none>
Metallb 控制器和其中一个扬声器
$ kubectl describe pods -n metallb-system
Name: controller-5c797bc4f8-2zx7l
Namespace: metallb-system
Priority: 0
Node: virgo/192.168.2.202
Start Time: Tue, 09 Feb 2021 14:10:16 +0000
Labels: app=metallb
component=controller
pod-template-hash=5c797bc4f8
Annotations: cni.projectcalico.org/podIP: 192.168.9.246/32
cni.projectcalico.org/podIPs: 192.168.9.246/32
prometheus.io/port: 7472
prometheus.io/scrape: true
Status: Running
IP: 192.168.9.246
IPs:
IP: 192.168.9.246
Controlled By: ReplicaSet/controller-5c797bc4f8
Containers:
controller:
Container ID: docker://b10ee06c88facfc97fcd246bcd214c746deea60073dc8928f478c89ca3a4feb9
Image: quay.io/metallb/controller:main
Image ID: docker-pullable://quay.io/metallb/controller@sha256:1e1a1bd75cebcecfe84db28c9a8605caeac3e30e57905185822c79fc00e5dcaf
Port: 7472/TCP
Host Port: 0/TCP
Args:
--port=7472
--config=config
State: Running
Started: Tue, 09 Feb 2021 14:10:43 +0000
Ready: True
Restart Count: 0
Limits:
cpu: 100m
memory: 100Mi
Requests:
cpu: 100m
memory: 100Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from controller-token-ghmtk (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
controller-token-ghmtk:
Type: Secret (a volume populated by a Secret)
SecretName: controller-token-ghmtk
Optional: false
QoS Class: Guaranteed
Node-Selectors: kubernetes.io/os=linux
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 89m default-scheduler Successfully assigned metallb-system/controller-5c797bc4f8-2zx7l to virgo
Normal Pulling 89m kubelet Pulling image "quay.io/metallb/controller:main"
Normal Pulled 89m kubelet Successfully pulled image "quay.io/metallb/controller:main" in 23.412886883s
Normal Created 89m kubelet Created container controller
Normal Started 89m kubelet Started container controller
Name: speaker-5kxc9
Namespace: metallb-system
Priority: 0
Node: taurus/192.168.2.201
Start Time: Tue, 09 Feb 2021 14:10:16 +0000
Labels: app=metallb
component=speaker
controller-revision-hash=7548d88c64
pod-template-generation=1
Annotations: prometheus.io/port: 7472
prometheus.io/scrape: true
Status: Running
IP: 192.168.2.201
IPs:
IP: 192.168.2.201
Controlled By: DaemonSet/speaker
Containers:
speaker:
Container ID: docker://e5ff811c47191f67c7089ce2464abff4fb8e4d3084f8053eb591c9d21e9a2276
Image: quay.io/metallb/speaker:main
Image ID: docker-pullable://quay.io/metallb/speaker@sha256:90a8cbdd543870e7db841a49743f3d8cd72b7074b5299086e09c5c1d92331eb4
Port: 7472/TCP
Host Port: 7472/TCP
Args:
--port=7472
--config=config
State: Running
Started: Tue, 09 Feb 2021 14:10:33 +0000
Ready: True
Restart Count: 0
Limits:
cpu: 100m
memory: 100Mi
Requests:
cpu: 100m
memory: 100Mi
Environment:
METALLB_NODE_NAME: (v1:spec.nodeName)
METALLB_HOST: (v1:status.hostIP)
METALLB_ML_BIND_ADDR: (v1:status.podIP)
METALLB_ML_LABELS: app=metallb,component=speaker
METALLB_ML_NAMESPACE: metallb-system (v1:metadata.namespace)
METALLB_ML_SECRET_KEY: <set to the key 'secretkey' in secret 'memberlist'> Optional: false
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from speaker-token-xfz9p (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
speaker-token-xfz9p:
Type: Secret (a volume populated by a Secret)
SecretName: speaker-token-xfz9p
Optional: false
QoS Class: Guaranteed
Node-Selectors: kubernetes.io/os=linux
Tolerations: node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 89m default-scheduler Successfully assigned metallb-system/speaker-5kxc9 to taurus
Normal Pulling 89m kubelet Pulling image "quay.io/metallb/speaker:main"
Normal Pulled 89m kubelet Successfully pulled image "quay.io/metallb/speaker:main" in 14.647284366s
Normal Created 89m kubelet Created container speaker
Normal Started 89m kubelet Started container speaker
Warning DNSConfigForming 4m14s (x71 over 89m) kubelet Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 8.8.8.8 1.1.1.1 192.168.2.1
解决方案
关于 MetalLB 控制器的问题。我也面临这个问题，因为我再次重新安装 MetalLB。参考：https ://github.com/metallb/metallb/issues/673
消除
kubectl delete -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
kubectl delete -f metallb-configMap.yaml
安装
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
kubectl apply -f metallb-configMap.yaml
不要担心分配了 IP 的现有服务。在您删除 MetalLB 控制器时，该服务可以提供服务并且仍然可以正常工作。
推荐阅读
- perl - 如何在 Perl 中将变量数据从子/子文件发送到顶级/父文件
- sql - 为什么联合所有查询都在重复数据？
- node.js - 如何修复 vuetify npm 问题
- laravel - 使用 Laravel 8 和 Pusher 在帐户验证事件上出现 array_merge 错误
- sdk - PyCharm：未解决的参考 'com'
- pine-script - Pine 脚本，如果问题
- html - 正则表达式：如何从 html 中获取正则表达式
- lisp - 如何获取 McCLIM 的光标位置
- netty - Netty 4.1 - 检测拒绝连接导致完全连接
- tomcat - 如何覆盖 Http11Nio2Protocol 类以加密 server.xml 中的密钥库密码