时间戳

首页 > 解决方案 > TCP入口点的Nginx-Ingress不起作用

logstash - TCP入口点的Nginx-Ingress不起作用

问题描述

我使用 Nginx 作为 Kubernetes Ingress 控制器。在遵循这个简单的例子之后，我能够设置这个例子

现在我正在尝试使用以下配置为 logstash 设置 TCP 入口点

日志存储：

apiVersion: v1
kind: Secret
metadata:
  name: logstash-secret
  namespace: kube-logging
type: Opaque
data:
  tls.crt: "<base64 encoded>" #For logstash.test.domain.com
  tls.key: "<base64 encoded>" #For logstash.test.domain.com

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: logstash-config
  namespace: kube-logging
  labels:
    app: logstash
data:
  syslog.conf: |-
    input {
      tcp {
          port => 5050
          type => syslog
      }
    }

    filter {
        grok {
          match => {"message" => "%{SYSLOGLINE}"}
        }
    }

    output {
      elasticsearch {
        hosts => ["http://elasticsearch:9200"] #elasticsearch running in same namespace (kube-logging)
        index => "syslog-%{+YYYY.MM.dd}"
      }
      stdout { codec => rubydebug }
    }

---

kind: Deployment
apiVersion: apps/v1
metadata:
  name: logstash
  namespace: kube-logging
  labels:
    app: logstash
spec:
  replicas: 1
  selector:
    matchLabels:
      app: logstash
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: logstash
    spec:
      #serviceAccountName: logstash
      containers:
      - name: logstash
        image: docker.elastic.co/logstash/logstash:7.2.1
        imagePullPolicy: Always
        env:
        - name: ELASTICSEARCH_HOST
          value: elasticsearch
        - name: ELASTICSEARCH_PORT
          value: "9200"
        - name: ELASTICSEARCH_USERNAME
          value: elastic
        - name: ELASTICSEARCH_PASSWORD
          value: changeme
        - name: ELASTIC_CLOUD_ID
          value:
        - name: ELASTIC_CLOUD_AUTH
          value:
        ports:
        - name: logstash
          containerPort: 5050
          protocol: TCP
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: config
          mountPath: /usr/share/logstash/pipeline/syslog.conf
          readOnly: true
          subPath: syslog.conf
      volumes:
      - name: config
        configMap:
          defaultMode: 0600
          name: logstash-config

---

kind: Service
apiVersion: v1
metadata:
  name: logstash
  namespace: kube-logging
  labels:
    app: logstash
spec:
  ports:
    - name: tcp-port
      protocol: TCP
      port: 5050
      targetPort: 5050
  selector:
    app: logstash

Nginx 入口：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress
  namespace: kube-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:1.5.7
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: tcp5050
          containerPort: 5050
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
          - -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics
         #- -enable-custom-resources

负载均衡器：

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-external
  namespace: kube-ingress
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
  type: LoadBalancer
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
  - name: https
    protocol: TCP
    port: 443
    targetPort: 443
  - name: tcp5050
    protocol: TCP
    port: 5050
    targetPort: 5050
  selector:
    app: nginx-ingress

入口：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: logstash-ingress
  namespace: kube-logging
spec:
  tls:
  - hosts:
    - logstash.test.domain.com
    secretName: logstash-secret #This has self-signed cert for logstash.test.domain.com
  rules:
  - host: logstash.test.domain.com
    http:
      paths:
      - path: /
        backend:
          serviceName: logstash
          servicePort: 5050

使用此配置，它显示以下内容，

NAME               HOSTS                         ADDRESS   PORTS     AGE
logstash-ingress   logstash.test.domain.com                 80, 443   79m

为什么这里没有列出 5050 端口？

只想logstash通过公共端点公开服务。当我 openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050在集群中使用时，我得到

$ openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050
CONNECTED(00000005)

但是从集群外部openssl s_client -connect logstash.test.domain.com:5050我得到

$ openssl s_client -connect logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

和

$ openssl s_client -cert logstash_test_domain_com.crt -key logstash_test_domain_com.key -servername logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

我需要做什么才能让这个工作？

标签： logstashkubernetes-ingressnginx-ingress

解决方案

看起来你有点困惑。因此，让我们从订购您的服务和入口开始。

首先，Kubernetes 中有 3 种类型的服务。ClusterIP这允许您在 k8s 内部公开您的部署。这与通过每个节点的外部 IP和一个范围为 ~30K-32K的PORTNodeport相同，ClusterIP但也会公开您的部署。最后还有一项服务，它与云提供商 LoadBalancer 分配的特定外部 IP 地址相同，但也将您的应用程序公开。LoadBalancerClusterIP

您创建的NodePort服务将使 logstash 可以通过 30K 到 32K 范围内的随机端口中的每个节点外部 IP 访问；找到正在运行的端口kubectl get services | grep nginx-ingress并检查最后一列。要获取节点的外部 IP 地址，请运行kubectl get node -o wide. 您创建的LoadBalancer服务将使 logstash 可以通过端口 5050 中的外部 IP 地址访问。要找出 IP，请运行kubectl get services | grep nginx-ingress-external。最后，您还创建了一个入口资源来访问 logstash。为此，您已经定义了一个主机，它可以在端口 443 中访问给定的 TLS，并且入站流量将被重定向到ClusterIP端口 5050 中的 logstash 服务类型。所以您有 3 种方式可以访问 logstash。LoadBalancer鉴于它是一个特定的端口， 我会去。

推荐阅读