时间戳

首页 > 解决方案 > Pull images from local (insecure) registry on kind cluster

kubernetes - Pull images from local (insecure) registry on kind cluster

问题描述

I have been trying to use some custom, local built image on my kind cluster by following the instructions on https://kind.sigs.k8s.io/docs/user/local-registry - i.e. applying the following containerdConfigPatches to my cluster.cfg:

containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.83.82:5000"]
    endpoint = ["http://192.168.83.82:5000"]

192.168.83.82:5000 is the IP of the VM where the local (insecure) registry is running alongside the kind cluster and its exposed port.

Once the cluster is created, I can verify the settings where applied to all nodes:

docker exec kind-worker3 cat /etc/containerd/config.toml
# [...]
# Relevant sections:
# [...]
  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.83.82:5000"]
          endpoint = ["http://192.168.83.82:5000"]

However, pods keep failing with to ErrImagePull and these Event log:

Events:
  Type     Reason     Age                From                             Message
  ----     ------     ----               ----                             -------
  Normal   Scheduled  32s                default-scheduler                Successfully assigned default/test-6bc95ff8c5-g6g86 to kind-worker3
  Normal   Pulled     31s                kubelet, kind-worker3  Container image "docker.elastic.co/beats/filebeat-oss:6.4.2" already present on machine
  Normal   Created    31s                kubelet, kind-worker3  Created container test-log-agent
  Normal   Started    31s                kubelet, kind-worker3  Started container test-log-agent
  Normal   Pulling    16s (x2 over 31s)  kubelet, kind-worker3  Pulling image "192.168.83.82:5000/test/image:2.2.1"
  Warning  Failed     16s (x2 over 31s)  kubelet, kind-worker3  Failed to pull image "192.168.83.82:5000/test/image:2.2.1": rpc error: code = Unknown desc = failed to resolve image "192.168.83.82:5000/test/image:2.2.1": no available registry endpoint: failed to do request: Head https://192.168.83.82:5000/v2/test/image/manifests/2.2.1: http: server gave HTTP response to HTTPS client
  Warning  Failed     16s (x2 over 31s)  kubelet, kind-worker3  Error: ErrImagePull
  Normal   BackOff    3s (x3 over 30s)   kubelet, kind-worker3  Back-off pulling image "192.168.83.82:5000/test/image:2.2.1"
  Warning  Failed     3s (x3 over 30s)   kubelet, kind-worker3  Error: ImagePullBackOff

TL;DR: "http: server gave HTTP response to HTTPS client" - which I thought would be solved with the ConfigPatch above (As it happens when you tweak docker's daemon.json).

Also, as an alternative, tried loading the image from the host to the cluster nodes:

kind load docker-image 192.168.83.82:5000/test/image:2.2.1 --name="kind-cluster"

And verified the image was loaded to all nodes by listing them:

sysadmin@ubuntu:~/kind$ docker exec kind-worker3 crictl images
IMAGE                                TAG                 IMAGE ID            SIZE
192.168.83.82:5000/test/image        2.2.1               ba1601dfa9c48       822MB
docker.io/kindest/kindnetd           0.5.0               ef97cccdfdb50       83.6MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180f       40.5MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4ff       258MB
k8s.gcr.io/kube-apiserver            v1.15.3             be321f2ded3f3       249MB
k8s.gcr.io/kube-controller-manager   v1.15.3             ac7d3fe5b34b7       200MB
k8s.gcr.io/kube-proxy                v1.15.3             d428039608992       97.3MB
k8s.gcr.io/kube-scheduler            v1.15.3             a44f53b10fee0       96.5MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca19       746kB

Hoping that the image now would be read from the image cache. However, the result was exactly the same.

Any clue on how to approach this one? Is there something I might have overlooked?

标签： kubernetescontainerdkind

解决方案

正如马特上面提到的，尝试在你的 config.toml 文件中使用 insecure_skip_verify 标志：

  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.83.82:5000"]
          endpoint = ["http://192.168.83.82:5000"]
      [plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.83.82:5000".tls]
          insecure_skip_verify = true

推荐阅读