kubernetes - 禁用 Internet 时,kube-service-catalog pod 处于“CrashLoopBackOff”状态
问题描述
我正在尝试使用 ansible 在没有互联网的情况下安装 openshift origin 的开源版本,即 OKD v3.11。在完整的安装过程中,我的 Internet 在环境中被禁用。安装成功后,我观察到kube-service-catalog命名空间中的两个 pod,即apiserver和controller-manager没有运行。在调查了剧本后,我发现剧本会生成 API Server 密钥。
API 服务器密钥的生成是否需要有效的 Internet 连接?apiserver 和 controller-manager pod 是否有任何 Internet 依赖项才能处于运行状态?
我试过:- 启用互联网并重新部署 kube-service-catalog 命名空间的 pod。它们处于运行状态,没有像预期的那样重新启动。
预期行为:- kube-service-catalog 命名空间中的两个 pod 应该是稳定的,并且处于Running状态,并且禁用了 internet。
实际行为:- kube-service-catalog 命名空间中的两个 pod 处于CrashLoopBackOff状态。
版本:- OKD- 3.11,ansible- 2.9
apiserver pod 的日志:-
I0512 04:53:30.258151 1 feature_gate.go:194] feature gates: map[OriginatingIdentity:true NamespacedServiceBroker:true]
I0512 04:53:30.258177 1 hyperkube.go:192] Service Catalog version v3.11.0-0.1.35+8d4f895-2;Upstream:v0.1.35 (built 2019-01-08T23:12:26Z)
W0512 04:53:31.020172 1 util.go:112] OpenAPI spec will not be served
I0512 04:53:31.021577 1 util.go:182] Admission control plugin names: [NamespaceLifecycle MutatingAdmissionWebhook ValidatingAdmissionWebhook ServicePlanChangeValidator BrokerAuthSarCheck DefaultServicePlan ServiceBindingsLifecycle]
I0512 04:53:31.021949 1 plugins.go:158] Loaded 6 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,MutatingAdmissionWebhook,ServicePlanChangeValidator,BrokerAuthSarCheck,DefaultServicePlan,ServiceBindingsLifecycle.
I0512 04:53:31.021971 1 plugins.go:161] Loaded 1 validating admission controller(s) successfully in the following order: ValidatingAdmissionWebhook.
I0512 04:53:31.023932 1 storage_factory.go:285] storing {servicecatalog.k8s.io clusterservicebrokers} in servicecatalog.k8s.io/v1beta1, reading as servicecatalog.k8s.io/__internal from storagebackend.Config{Type:"", Prefix:"/registry", ServerList:[]string{"https://cic-90-master.novalocal:2379"}, KeyFile:"/etc/origin/master/master.etcd-client.key", CertFile:"/etc/origin/master/master.etcd-client.crt", CAFile:"/etc/origin/master/master.etcd-ca.crt", Quorum:true, Paging:true, DeserializationCacheSize:0, Codec:runtime.Codec(nil), Transformer:value.Transformer(nil), CompactionInterval:300000000000, CountMetricPollPeriod:60000000000}
I0512 04:53:31.023978 1 storage_factory.go:285] storing {servicecatalog.k8s.io clusterserviceclasses} in servicecatalog.k8s.io/v1beta1, reading as servicecatalog.k8s.io/__internal from storagebackend.Config{Type:"", Prefix:"/registry", ServerList:[]string{"https://cic-90-master.novalocal:2379"}, KeyFile:"/etc/origin/master/master.etcd-client.key", CertFile:"/etc/origin/master/master.etcd-client.crt", CAFile:"/etc/origin/master/master.etcd-ca.crt", Quorum:true, Paging:true, DeserializationCacheSize:0, Codec:runtime.Codec(nil), Transformer:value.Transformer(nil), CompactionInterval:300000000000, CountMetricPollPeriod:60000000000}
I0512 04:53:31.023998 1 storage_factory.go:285] storing {servicecatalog.k8s.io clusterserviceplans} in servicecatalog.k8s.io/v1beta1, reading as servicecatalog.k8s.io/__internal from storagebackend.Config{Type:"", Prefix:"/registry", ServerList:[]string{"https://cic-90-master.novalocal:2379"}, KeyFile:"/etc/origin/master/master.etcd-client.key", CertFile:"/etc/origin/master/master.etcd-client.crt", CAFile:"/etc/origin/master/master.etcd-ca.crt", Quorum:true, Paging:true, DeserializationCacheSize:0, Codec:runtime.Codec(nil), Transformer:value.Transformer(nil), CompactionInterval:300000000000, CountMetricPollPeriod:60000000000}
I0512 04:53:31.024031 1 storage_factory.go:285] storing {servicecatalog.k8s.io serviceinstances} in servicecatalog.k8s.io/v1beta1, reading as servicecatalog.k8s.io/__internal from storagebackend.Config{Type:"", Prefix:"/registry", ServerList:[]string{"https://cic-90-master.novalocal:2379"}, KeyFile:"/etc/origin/master/master.etcd-client.key", CertFile:"/etc/origin/master/master.etcd-client.crt", CAFile:"/etc/origin/master/master.etcd-ca.crt", Quorum:true, Paging:true, DeserializationCacheSize:0, Codec:runtime.Codec(nil), Transformer:value.Transformer(nil), CompactionInterval:300000000000, CountMetricPollPeriod:60000000000}
I0512 04:53:31.024055 1 storage_factory.go:285] storing {servicecatalog.k8s.io servicebindings} in servicecatalog.k8s.io/v1beta1, reading as servicecatalog.k8s.io/__internal from storagebackend.Config{Type:"", Prefix:"/registry", ServerList:[]string{"https://cic-90-master.novalocal:2379"}, KeyFile:"/etc/origin/master/master.etcd-client.key", CertFile:"/etc/origin/master/master.etcd-client.crt", CAFile:"/etc/origin/master/master.etcd-ca.crt", Quorum:true, Paging:true, DeserializationCacheSize:0, Codec:runtime.Codec(nil), Transformer:value.Transformer(nil), CompactionInterval:300000000000, CountMetricPollPeriod:60000000000}
F0512 04:53:51.025999 1 storage_decorator.go:57] Unable to create storage backend: config (&{ /registry [https://cic-90-master.novalocal:2379] /etc/origin/master/master.etcd-client.key /etc/origin/master/master.etcd-client.crt /etc/origin/master/master.etcd-ca.crt true true 0 {0xc420345080 0xc420345100} <nil> 5m0s 1m0s}), err (context deadline exceeded)
控制器管理器 pod 的日志:-
I0512 05:05:01.273888 1 feature_gate.go:194] feature gates: map[OriginatingIdentity:true]
I0512 05:05:01.274109 1 feature_gate.go:194] feature gates: map[OriginatingIdentity:true AsyncBindingOperations:true]
I0512 05:05:01.274128 1 feature_gate.go:194] feature gates: map[NamespacedServiceBroker:true OriginatingIdentity:true AsyncBindingOperations:true]
I0512 05:05:01.274155 1 hyperkube.go:192] Service Catalog version v3.11.0-0.1.35+8d4f895-2;Upstream:v0.1.35 (built 2019-01-08T23:12:26Z)
I0512 05:05:01.276689 1 leaderelection.go:185] attempting to acquire leader lease kube-service-catalog/service-catalog-controller-manager...
I0512 05:05:01.303464 1 leaderelection.go:194] successfully acquired lease kube-service-catalog/service-catalog-controller-manager
I0512 05:05:01.303609 1 event.go:221] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"kube-service-catalog", Name:"service-catalog-controller-manager", UID:"724069a9-9362-11ea-b5c1-fa163e86d97a", APIVersion:"v1", ResourceVersion:"126373", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' controller-manager-jvx4f-external-service-catalog-controller became leader
F0512 05:05:01.332950 1 controller_manager.go:237] error running controllers: failed to get api versions from server: failed to get supported resources from server: unable to retrieve the complete list of server APIs: servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
kubectl get 事件的输出:-
LAST SEEN FIRST SEEN COUNT NAME KIND SUBOBJECT TYPE REASON SOURCE MESSAGE
2h 2h 1 service-catalog-controller-manager.160e29595b5f2ac8 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 1h 1 service-catalog-controller-manager.160e29a1c8d44d5f ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 1h 1 service-catalog-controller-manager.160e29e88bcdabf4 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 1h 1 service-catalog-controller-manager.160e2a2ea2d553cf ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 1h 1 service-catalog-controller-manager.160e2abce844b1a6 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 1h 1 service-catalog-controller-manager.160e2bd884a3fd98 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
1h 17h 183 apiserver-28mjt.160df6e8ab679328 Pod spec.containers{apiserver} Normal Pulled kubelet, cic-90-master.novalocal Container image "docker.io/openshift/origin-service-catalog:v3.11.0" already present on machine
1h 1h 1 service-catalog-controller-manager.160e2c1f807c24b0 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
59m 59m 1 service-catalog-controller-manager.160e2cac5f27eb61 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
48m 48m 1 service-catalog-controller-manager.160e2d3d315161ed ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
43m 43m 1 service-catalog-controller-manager.160e2d84348e29c6 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
38m 38m 1 service-catalog-controller-manager.160e2dcbb5d88e66 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
33m 33m 1 service-catalog-controller-manager.160e2e13307a6011 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
23m 23m 1 service-catalog-controller-manager.160e2ea16c9db85d ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
8m 8m 1 service-catalog-controller-manager.160e2f75c0f6468a ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
4m 17h 4491 apiserver-28mjt.160df6f2fa5c8d45 Pod spec.containers{apiserver} Warning BackOff kubelet, cic-90-master.novalocal Back-off restarting failed container
2m 2m 1 service-catalog-controller-manager.160e2fbf5d9a2418 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-jvx4f-external-service-catalog-controller became leader
2m 20h 5739 controller-manager-jvx4f.160dec6599cd8b00 Pod spec.containers{controller-manager} Warning BackOff kubelet, cic-90-master.novalocal Back-off restarting failed container
解决方案
推荐阅读
- java - 这种设计模式是否有一个名称,其中具体类实现了一个特定接口,该接口实现了 CRUD 操作的基本接口?
- rest - Spring Security,基于 JWT (REST) 的身份验证流程
- layout - Graphviz 和 dot 文件——水平和垂直节点对齐;中间节点
- reactjs - 带有路由器的功能复合组件的类型是什么?
- python-3.x - 无法使用 condas 安装 Keras - 版本错误
- r - 在 Databricks 中安装 RDCOMClient
- python - 设置默认位置以使用 python 保存文件
- java - 如何使用 Criteria Builder 过滤参数列表?
- ios - 本机模块不能为空 - RCTSystemSetting
- reactjs - 移除元素后事件处理卡住