kubernetes - Nginx-Ingress Helm 部署 --tcp-services-configmap 未找到参数
问题描述
我正在尝试通过入口进行 TCP/UDP 端口转发。
遵循文档:https ://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
它说设置:--tcp-services-configmap
但没有告诉你在哪里设置它。我假设它是命令行参数。然后我搜索了 nginx-ingress 的命令行参数列表
https://kubernetes.github.io/ingress-nginx/user-guide/cli-arguments/
在这里您可以清楚地看到它的控制器参数:
--tcp-services-configmap 包含要公开的 TCP 服务定义的 ConfigMap 的名称。映射中的键表示要使用的外部端口。该值是对“命名空间/名称:端口”形式的服务的引用,其中“端口”可以是端口号或名称。TCP 端口 80 和 443 由控制器保留用于服务 HTTP 流量。
第一个问题:如何动态添加到 nginx-ingress helm 图表的容器参数中,我在任何地方都没有看到记录?
第二个问题:使用当前版本的 nginx-ingress 设置它的正确方法是什么,因为设置命令行参数会导致容器启动失败,因为二进制文件没有该参数选项。
在默认的舵图 values.yaml 中,有一些选项可以为 tcp-services 的 configmap 设置命名空间,但鉴于文档说我必须将其设置为参数,但该参数启动失败我不确定你是如何实际设置这个。
https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/values.yaml
我手动编辑了部署并在容器 args 上设置了标志:
- args:
- -nginx-plus=false
- -nginx-reload-timeout=60000
- -enable-app-protect=false
- -nginx-configmaps=$(POD_NAMESPACE)/emoney-nginx-controller-nginx-ingress
- -default-server-tls-secret=$(POD_NAMESPACE)/emoney-nginx-controller-nginx-ingress-default-server-tls
- -ingress-class=emoney-ingress
- -health-status=false
- -health-status-uri=/nginx-health
- -tcp-services-configmap=emoney-node/tcp-services-configmap
- -nginx-debug=false
- -v=1
- -nginx-status=true
- -nginx-status-port=8080
- -nginx-status-allow-cidrs=127.0.0.1
- -report-ingress-status
- -external-service=emoney-nginx-controller-nginx-ingress
- -enable-leader-election=true
- -leader-election-lock-name=emoney-nginx-controller-nginx-ingress-leader-election
- -enable-prometheus-metrics=true
- -prometheus-metrics-listen-port=9113
- -prometheus-tls-secret=
- -enable-custom-resources=true
- -enable-tls-passthrough=false
- -enable-snippets=false
- -enable-preview-policies=false
- -ready-status=true
- -ready-status-port=8081
- -enable-latency-metrics=false
env:
当我像文档所说的那样设置它时,pod 可能无法启动,因为它错误地指出该参数不是二进制文件的选项。
kubectl logs emoney-nginx-controller-nginx-ingress-5769565cc7-vmgrf -n emoney-node
flag provided but not defined: -tcp-services-configmap
Usage of /nginx-ingress:
-alsologtostderr
log to standard error as well as files
-default-server-tls-secret string
A Secret with a TLS certificate and key for TLS termination of the default server. Format: <namespace>/<name>.
If not set, than the certificate and key in the file "/etc/nginx/secrets/default" are used.
If "/etc/nginx/secrets/default" doesn't exist, the Ingress Controller will configure NGINX to reject TLS connections to the default server.
If a secret is set, but the Ingress controller is not able to fetch it from Kubernetes API or it is not set and the Ingress Controller
fails to read the file "/etc/nginx/secrets/default", the Ingress controller will fail to start.
-enable-app-protect
Enable support for NGINX App Protect. Requires -nginx-plus.
-enable-custom-resources
Enable custom resources (default true)
-enable-internal-routes
Enable support for internal routes with NGINX Service Mesh. Requires -spire-agent-address and -nginx-plus. Is for use with NGINX Service Mesh only.
-enable-latency-metrics
Enable collection of latency metrics for upstreams. Requires -enable-prometheus-metrics
-enable-leader-election
Enable Leader election to avoid multiple replicas of the controller reporting the status of Ingress, VirtualServer and VirtualServerRoute resources -- only one replica will report status (default true). See -report-ingress-status flag. (default true)
-enable-preview-policies
Enable preview policies
-enable-prometheus-metrics
Enable exposing NGINX or NGINX Plus metrics in the Prometheus format
-enable-snippets
Enable custom NGINX configuration snippets in Ingress, VirtualServer, VirtualServerRoute and TransportServer resources.
-enable-tls-passthrough
Enable TLS Passthrough on port 443. Requires -enable-custom-resources
-external-service string
Specifies the name of the service with the type LoadBalancer through which the Ingress controller pods are exposed externally.
The external address of the service is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires -report-ingress-status.
-global-configuration string
The namespace/name of the GlobalConfiguration resource for global configuration of the Ingress Controller. Requires -enable-custom-resources. Format: <namespace>/<name>
-health-status
Add a location based on the value of health-status-uri to the default server. The location responds with the 200 status code for any request.
Useful for external health-checking of the Ingress controller
-health-status-uri string
Sets the URI of health status location in the default server. Requires -health-status (default "/nginx-health")
-ingress-class string
A class of the Ingress controller.
An IngressClass resource with the name equal to the class must be deployed. Otherwise, the Ingress Controller will fail to start.
The Ingress controller only processes resources that belong to its class - i.e. have the "ingressClassName" field resource equal to the class.
The Ingress Controller processes all the VirtualServer/VirtualServerRoute/TransportServer resources that do not have the "ingressClassName" field for all versions of kubernetes. (default "nginx")
-ingress-template-path string
Path to the ingress NGINX configuration template for an ingress resource.
(default for NGINX "nginx.ingress.tmpl"; default for NGINX Plus "nginx-plus.ingress.tmpl")
-ingresslink string
Specifies the name of the IngressLink resource, which exposes the Ingress Controller pods via a BIG-IP system.
The IP of the BIG-IP system is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires -report-ingress-status.
-leader-election-lock-name string
Specifies the name of the ConfigMap, within the same namespace as the controller, used as the lock for leader election. Requires -enable-leader-election. (default "nginx-ingress-leader-election")
-log_backtrace_at value
when logging hits line file:N, emit a stack trace
-log_dir string
If non-empty, write log files in this directory
-logtostderr
log to standard error instead of files
-main-template-path string
Path to the main NGINX configuration template. (default for NGINX "nginx.tmpl"; default for NGINX Plus "nginx-plus.tmpl")
-nginx-configmaps string
A ConfigMap resource for customizing NGINX configuration. If a ConfigMap is set,
but the Ingress controller is not able to fetch it from Kubernetes API, the Ingress controller will fail to start.
Format: <namespace>/<name>
-nginx-debug
Enable debugging for NGINX. Uses the nginx-debug binary. Requires 'error-log-level: debug' in the ConfigMap.
-nginx-plus
Enable support for NGINX Plus
-nginx-reload-timeout int
The timeout in milliseconds which the Ingress Controller will wait for a successful NGINX reload after a change or at the initial start. (default 60000) (default 60000)
-nginx-status
Enable the NGINX stub_status, or the NGINX Plus API. (default true)
-nginx-status-allow-cidrs string
Add IPv4 IP/CIDR blocks to the allow list for NGINX stub_status or the NGINX Plus API. Separate multiple IP/CIDR by commas. (default "127.0.0.1")
-nginx-status-port int
Set the port where the NGINX stub_status or the NGINX Plus API is exposed. [1024 - 65535] (default 8080)
-prometheus-metrics-listen-port int
Set the port where the Prometheus metrics are exposed. [1024 - 65535] (default 9113)
-prometheus-tls-secret string
A Secret with a TLS certificate and key for TLS termination of the prometheus endpoint.
-proxy string
Use a proxy server to connect to Kubernetes API started by "kubectl proxy" command. For testing purposes only.
The Ingress controller does not start NGINX and does not write any generated NGINX configuration files to disk
-ready-status
Enables the readiness endpoint '/nginx-ready'. The endpoint returns a success code when NGINX has loaded all the config after the startup (default true)
-ready-status-port int
Set the port where the readiness endpoint is exposed. [1024 - 65535] (default 8081)
-report-ingress-status
Updates the address field in the status of Ingress resources. Requires the -external-service or -ingresslink flag, or the 'external-status-address' key in the ConfigMap.
-spire-agent-address string
Specifies the address of the running Spire agent. Requires -nginx-plus and is for use with NGINX Service Mesh only. If the flag is set,
but the Ingress Controller is not able to connect with the Spire Agent, the Ingress Controller will fail to start.
-stderrthreshold value
logs at or above this threshold go to stderr
-transportserver-template-path string
Path to the TransportServer NGINX configuration template for a TransportServer resource.
(default for NGINX "nginx.transportserver.tmpl"; default for NGINX Plus "nginx-plus.transportserver.tmpl")
-v value
log level for V logs
-version
Print the version, git-commit hash and build date and exit
-virtualserver-template-path string
Path to the VirtualServer NGINX configuration template for a VirtualServer resource.
(default for NGINX "nginx.virtualserver.tmpl"; default for NGINX Plus "nginx-plus.virtualserver.tmpl")
-vmodule value
comma-separated list of pattern=N settings for file-filtered logging
-watch-namespace string
Namespace to watch for Ingress resources. By default the Ingress controller watches all namespaces
-wildcard-tls-secret string
A Secret with a TLS certificate and key for TLS termination of every Ingress host for which TLS termination is enabled but the Secret is not specified.
Format: <namespace>/<name>. If the argument is not set, for such Ingress hosts NGINX will break any attempt to establish a TLS connection.
If the argument is set, but the Ingress controller is not able to fetch the Secret from Kubernetes API, the Ingress controller will fail to start.
配置图
apiVersion: v1
data:
"1317": emoney-node/emoney-api:1317
"9090": emoney-node/emoney-grpc:9090
"26656": emoney-node/emoney:26656
"26657": emoney-node/emoney-rpc:26657
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: emoney
meta.helm.sh/release-namespace: emoney-node
creationTimestamp: "2021-11-01T18:06:49Z"
labels:
app.kubernetes.io/managed-by: Helm
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:1317: {}
f:9090: {}
f:26656: {}
f:26657: {}
f:metadata:
f:annotations:
.: {}
f:meta.helm.sh/release-name: {}
f:meta.helm.sh/release-namespace: {}
f:labels:
.: {}
f:app.kubernetes.io/managed-by: {}
manager: helm
operation: Update
time: "2021-11-01T18:06:49Z"
name: tcp-services-configmap
namespace: emoney-node
resourceVersion: "2056146"
selfLink: /api/v1/namespaces/emoney-node/configmaps/tcp-services-configmap
uid: 188f5dc8-02f9-4ee5-a5e3-819d00ff8b67
Name: emoney
Namespace: emoney-node
Labels: app.kubernetes.io/instance=emoney
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ibcnode
app.kubernetes.io/version=1.16.0
helm.sh/chart=ibcnode-0.1.0
Annotations: meta.helm.sh/release-name: emoney
meta.helm.sh/release-namespace: emoney-node
Selector: app.kubernetes.io/instance=emoney,app.kubernetes.io/name=ibcnode
Type: ClusterIP
IP: 172.20.30.240
Port: p2p 26656/TCP
TargetPort: 26656/TCP
Endpoints: 10.0.36.192:26656
Session Affinity: None
Events: <none>
Name: emoney-api
Namespace: emoney-node
Labels: app.kubernetes.io/instance=emoney
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ibcnode
app.kubernetes.io/version=1.16.0
helm.sh/chart=ibcnode-0.1.0
Annotations: meta.helm.sh/release-name: emoney
meta.helm.sh/release-namespace: emoney-node
Selector: app.kubernetes.io/instance=emoney,app.kubernetes.io/name=ibcnode
Type: ClusterIP
IP: 172.20.166.97
Port: api 1317/TCP
TargetPort: 1317/TCP
Endpoints: 10.0.36.192:1317
Session Affinity: None
Events: <none>
Name: emoney-grpc
Namespace: emoney-node
Labels: app.kubernetes.io/instance=emoney
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ibcnode
app.kubernetes.io/version=1.16.0
helm.sh/chart=ibcnode-0.1.0
Annotations: meta.helm.sh/release-name: emoney
meta.helm.sh/release-namespace: emoney-node
Selector: app.kubernetes.io/instance=emoney,app.kubernetes.io/name=ibcnode
Type: ClusterIP
IP: 172.20.136.177
Port: grpc 9090/TCP
TargetPort: 9090/TCP
Endpoints: 10.0.36.192:9090
Session Affinity: None
Events: <none>
Name: emoney-nginx-controller-nginx-ingress
Namespace: emoney-node
Labels: app.kubernetes.io/instance=emoney-nginx-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=emoney-nginx-controller-nginx-ingress
helm.sh/chart=nginx-ingress-0.11.3
Annotations: meta.helm.sh/release-name: emoney-nginx-controller
meta.helm.sh/release-namespace: emoney-node
Selector: app=emoney-nginx-controller-nginx-ingress
Type: LoadBalancer
IP: 172.20.16.202
LoadBalancer Ingress: lb removed
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 32250/TCP
Endpoints: 10.0.43.32:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 32375/TCP
Endpoints: 10.0.43.32:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 30904
Events: <none>
Name: emoney-rpc
Namespace: emoney-node
Labels: app.kubernetes.io/instance=emoney
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ibcnode
app.kubernetes.io/version=1.16.0
helm.sh/chart=ibcnode-0.1.0
Annotations: meta.helm.sh/release-name: emoney
meta.helm.sh/release-namespace: emoney-node
Selector: app.kubernetes.io/instance=emoney,app.kubernetes.io/name=ibcnode
Type: ClusterIP
IP: 172.20.42.163
Port: rpc 26657/TCP
TargetPort: 26657/TCP
Endpoints: 10.0.36.192:26657
Session Affinity: None
Events: <none>
helm repo add nginx-stable https://helm.nginx.com/stable --kubeconfig=./kubeconfig || echo "helm repo already added"
helm repo update --kubeconfig=./kubeconfig || echo "helm repo already updated"
helm upgrade ${app_name}-nginx-controller -n ${app_namespace} nginx-stable/nginx-ingress \
--install \
--kubeconfig=./kubeconfig \
--create-namespace \
--set controller.service.type=LoadBalancer \
--set controller.tcp.configMapNamespace=${app_namespace} \
--set controller.ingressClass="${app_name}-ingress"
kubectl rollout status -w deployment/${app_name} --kubeconfig=./kubeconfig -n ${app_namespace}
#- --tcp-services-configmap=emoney-node/tcp-services-configmap
解决方案
您可以说 helm 图表有偏见,因为它没有公开将这些 args 设置为图表值的选项。当根据值需要时,它将根据条件逻辑自行设置它们。
当我检查 repo 中的 nginx 模板时,我看到额外的参数是从params 帮助文件中的模板传递的。这些似乎是动态生成的。IE
{{- if .Values.tcp }}
- --tcp-services-configmap={{ default "$(POD_NAMESPACE)" .Values.controller.tcp.configMapNamespace }}/{{ include "ingress-nginx.fullname" . }}-tcp
{{- end }}
因此,似乎只有在 tcp 值不为空时才会使用此标志。在相同的条件下,它将创建 configmap。
此外, tcp 值允许您设置一个 key configMapNamespace
。因此,如果您仅设置此键,则该标志将作为参数助手使用。现在您只需要在提供的命名空间中创建您的 configmap 并让它与 name 匹配{{ include "ingress-nginx.fullname" . }}-tcp
。
因此,您可以在命名空间中创建配置映射default
并为其命名ingress-nginx-tcp
或类似名称,具体取决于您设置发布名称的方式。
kubectl create configmap ingress-nginx-tcp --from-literal 1883=mqtt/emqx:1883 -n default
helm install --set controller.tcp.configMapNamespace=default ingress-nginx ingress-nginx/ingress-nginx
我认为唯一的问题是你不能在 中创建它.Release.Namespace
,因为当 tcp 不为空时,它会尝试自行在那里创建一个 configmap,这会导致冲突。至少我是这样解释图表存储库中的模板的。
我个人已经通过我传递给 helm 的值文件配置了 TCP -f
。
helm install -f values.yaml ingress-nginx ingress-nginx/ingress-nginx
# configure the tcp configmap
tcp:
1883: mqtt/emqx:1883
8883: mqtt/emqx:8883
# enable the service and expose the tcp ports.
# be careful as this will pontentially make them
# availble on the public web
controller:
service:
enabled: true
ports:
http: 80
https: 443
mqtt: 1883
mqttssl: 8883
targetPorts:
http: http
https: https
mqtt: mqtt
mqttssl: mqttssl
推荐阅读
- r - 如何在 R 中过滤大型 geojson 文件
- google-cloud-firestore - Firebase Cloud Function 和多个请求
- reactjs - React 挂钩中的多个提供程序和路由器
- python - 为什么我们在 python 类中使用 super()?
- azure - 自定义策略中 Azure AD B2C 登录期间的休息 API 调用
- apache-kafka - 使用 kafka 配置模式注册表时出现侦听器错误
- php - Laravel Voyager 自定义 browse.blade.php
- php - PHPExcel没有用公式写最后一个单元格
- jsf - 如何在单个页面中包含的两个数据表中使用相同的 ajax 事件?
- wordpress - Facebook Pixel 无法在 iOS 移动设备上运行(Wordpress/Woocommerce)