nginx - 通过 Ingress 访问 K8s 服务
问题描述
我们已经配置了 MetalLB,因为我们的 K8s 集群托管在裸机基础设施上。它似乎运行良好,所有 pod 都启动并运行。
[~]# kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-b78574c59-47qfv 1/1 Running 0 24h
pod/speaker-4q2vm 1/1 Running 0 24h
pod/speaker-m8kwk 1/1 Running 0 24h
pod/speaker-t4rvs 1/1 Running 0 24h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 3 3 3 3 3 kubernetes.io/os=linux 24h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 24h
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-b78574c59 1 1 1 24h
我们已经通过 helm 从 https://github.com/kubernetes/ingress-nginx/releases/tag/helm-chart-3.29.0配置了入口控制器,并分别在文件中更新hostNetwork
, ingressClass
, kind
to true
, 。helm 安装似乎在所有运行的 daemonset pod 以及为创建的入口控制器服务提供的 LB ip 的情况下运行良好。ingress-nginx
DaemonSet
values.yaml
[~]# kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/devingress-ingress-nginx-controller-c2x42 1/1 Running 0 18h
pod/devingress-ingress-nginx-controller-wtmgw 1/1 Running 0 18h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/devingress-ingress-nginx-controller LoadBalancer x.x.x.x 1.2.3.40 80:32386/TCP,443:30020/TCP 18h
service/devingress-ingress-nginx-controller-admission ClusterIP x.x.x.x <none> 443/TCP 18h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/devingress-ingress-nginx-controller 2 2 2 2 2 kubernetes.io/os=linux 18h
现在我们已经部署了两个 Pod,分别nginx
是LoadBalancer
服务类型和nginx-deploy-main
服务ClusterIP
类型。
[~]# kubectl get all -n default
NAME READY STATUS RESTARTS AGE
pod/nginx-854cf6b4d7-lv5ss 1/1 Running 0 18h
pod/nginx-deploy-main-6b5457fbb5-7tg9z 1/1 Running 0 18h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx LoadBalancer x.x.x.x 1.2.3.41 8080:31101/TCP 18h
service/nginx-deploy-main ClusterIP x.x.x.x <none> 80/TCP 18h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 18h
deployment.apps/nginx-deploy-main 1/1 1 1 18h
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-854cf6b4d7 1 1 1 18h
replicaset.apps/nginx-deploy-main-6b5457fbb5 1 1 1 18h
下面是要访问的入口资源设置nginx-deploy-main
。
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: nginx
spec:
ingressClassName: nginx
rules:
- host: nginx-main.int.org.com
http:
paths:
- path: /
backend:
serviceName: nginx-deploy-main
servicePort: 80
并且入口资源似乎被正确创建并指向nginx-deploy-main
服务。
[~]# kubectl get ing -n default
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-resource nginx nginx-main.int.org.com 80 19h
[~]# kubectl describe ing/ingress-resource -n default
Name: ingress-resource
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
nginx-main.int.org.com
/ nginx-deploy-main:80 (x.x.x.x:80)
Annotations: kubernetes.io/ingress.class: nginx
Events: <none>
在 K8s 集群之外,我们将 nginx 设置为具有域 int.org.com 解析的反向代理。
下面是 nginx 配置,它应该可以帮助我点击 urlhttp://nginx-main.int.org.com
并获得响应,但返回的响应是 404。
upstream nginx-main.int.org.com {
server 1.2.3.40:80; ## Ingress Controller Service IP
}
server {
listen 80;
server_name nginx-main.int.org.com;
location / {
proxy_pass http://nginx-main.int.org.com;
}
}
现在,当我尝试使用具有以下配置的服务 IP 访问 nginx pod(不是 nginx-main)LoadBalancer
时,它能够提供响应并且工作正常
upstream nginx.int.org.com {
server 1.2.3.41:8080;
}
server {
listen 80;
server_name nginx.int.org.com;
location / {
proxy_pass http://nginx.int.org.com;
}
}
关于入口控制器或资源,我在这里遗漏了什么吗?端口转发工作正常,并且能够使用它访问服务。
这确实是一个障碍,任何帮助或文档参考都会非常有用。
解决方案
我们尝试了另一个入口控制器,即https://github.com/nginxinc/kubernetes-ingress并且能够使其工作。
以下是完成的步骤。
[~] git clone https://github.com/nginxinc/kubernetes-ingress/
[~] cd kubernetes-ingress/deployments
[~] git checkout v1.11.1
[~] kubectl apply -f common/ns-and-sa.yaml
[~] kubectl apply -f rbac/rbac.yaml
[~] kubectl apply -f common/default-server-secret.yaml
[~] kubectl apply -f common/nginx-config.yaml
[~] kubectl apply -f common/ingress-class.yaml
--enable-custom-resources=false
由于控制器日志中的以下问题,创建了带有额外环境参数的守护程序集 pod,即在 yaml 中添加
参考:Kubernetes 集群工作但从 NGINX 控制器收到此错误
[~] kubectl apply -f daemon-set/nginx-ingress.yaml
[~] kubectl get all -n nginx-ingress -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginx-ingress-gd8gw 1/1 Running 0 3h55m x.x.x.x worker1 <none> <none>
pod/nginx-ingress-kr9lx 1/1 Running 0 3h55m x.x.x.x worker2 <none> <none>
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
daemonset.apps/nginx-ingress 2 2 2 2 2 <none> 5h14m nginx-ingress nginx/nginx-ingress:1.11.1 app=nginx-ingress
在端口 80 上访问相应的工作节点,404 响应意味着它工作正常。
使用 github 链接https://github.com/vipin-k/Ingress-Controller-v1.9.0/blob/main/hotel.yml部署了一个示例应用程序,并将 Ingress 对象中的主机条目更新为hotel.int.org.com
[~] kubectl create -f hotel.yaml
[~] kubectl get all -n hotel -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/hotel-65d644c8f7-bj597 1/1 Running 0 3h51m x.x.x.x worker1 <none> <none>
pod/hotel-65d644c8f7-csvgp 1/1 Running 0 3h51m x.x.x.x worker2 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/hotel-svc ClusterIP x.x.x.x <none> 80/TCP 3h51m app=hotel
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/hotel 2/2 2 2 3h51m hotel nginxdemos/hello:plain-text app=hotel
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/hotel-65d644c8f7 2 2 2 3h51m hotel nginxdemos/hello:plain-text app=hotel,pod-template-hash=65d644c8f7
[~] kubectl get ing -n hotel
NAME CLASS HOSTS ADDRESS PORTS AGE
hotel-ingress nginx hotel.int.org.com 80 3h52m
[~] kubectl describe ing hotel-ingress -n hotel
Name: hotel-ingress
Namespace: hotel
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
hotel.int.org.com
/ hotel-svc:80 (x.x.x.x:80,x.x.x.x:80)
Annotations: Events: <none>
更新了启用域解析的外部 nginx 配置。
upstream hotel.int.org.com {
server 1.2.3.41:80; #worker1
server 1.2.3.42:80; #worker2
}
server {
listen 80;
server_name hotel.int.org.com;
location / {
proxy_pass http://hotel.int.org.com;
}
}
重新启动 nginx 并验证是否能够通过浏览器访问来自各个正在运行的酒店命名空间守护程序 pod 的服务响应。
[~]# curl hotel.int.org.com
Server address: x.x.x.x:80
Server name: hotel-65d644c8f7-bj597
Date: 28/Apr/2021:05:47:15 +0000
URI: /
Request ID: 28d5cfab4ea28beea49879422b7e8f4c
[~]# curl hotel.int.org.com
Server address: x.x.x.x:80
Server name: hotel-65d644c8f7-csvgp
Date: 28/Apr/2021:05:52:06 +0000
URI: /
Request ID: 4135cacf83f8bf41c9677104500e610b
也与 MetalLB 一起探索,一旦成功,将发布解决方案
推荐阅读
- python - 在调用带有 tqdm 进度条的函数之前的打印语句会导致在 tddm 之后打印打印语句
- ejabberd - 如何在 ejabberd 服务器上上传文件?
- python - 给定一个字符串列表,找到最频繁的字符串对,第二频繁的对,......,然后是最频繁的字符串三元组,等等
- php - IIS 上的 PHP 问题 - 在 PHP Excel 中写入大量数据时返回错误 500
- python - Pycharm路径引用
- python-3.x - 来自 R&S rto 示波器的流式数据 - UnicodeDecodeError python3.6
- elasticsearch - 在嵌套的热门点击聚合中包含父 _source 字段
- react-admin - React Admin - 过滤器组件中的按钮显示两次?
- bash - bash脚本aws cloudfront创建无效路径不正确
- angular - ng-katex 没有正确安装