ssl - 在入口控制器为超集启用 SSL 终止

问题描述

我们已将超集部署到 EKS 集群。我们希望 TLS 在 Ingress 控制器处终止。这就是我设置整个设置的方式。 GSLB Vip → Nginx ingress controller → http → ingress → service → pod

当我们在浏览器上输入 URL/login/ 时，它会login在我们提供凭据后再次被重定向到。如果我在新标签上打开/superset/welcome它就可以了。我们假设问题是因为 Superset 最初从 https 重定向到 http，并且无法访问登录，但我可能是错的

当我们卷曲 WIP 时，这是响应

curl -ivk https://superset-aws-ssl.g.xx.com/login
*   Trying 10.20.xx.yy...
* TCP_NODELAY set
* Connected to superset-aws-ssl.g.xx.com (10.20.xx.66) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / xxx-xxx-xxx-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=superset-aws-ssl.g.xx.com; OU=xx; O=YY Inc.; ST=CC; C=US
*  start date: Oct 13 20:01:12 2021 GMT
*  expire date: Nov 12 20:01:11 2023 GMT
*  issuer: CN=xx CA 1; OU=Certification Authority; O=xx; C=US
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: superset-aws-ssl.g.xx.com
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 308 PERMANENT REDIRECT
HTTP/1.1 308 PERMANENT REDIRECT
< Server: nginx/1.17.8
Server: nginx/1.17.8
< Date: Fri, 15 Oct 2021 06:47:44 GMT
Date: Fri, 15 Oct 2021 06:47:44 GMT
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Content-Length: 331
Content-Length: 331
< Connection: keep-alive
Connection: keep-alive
< Location: http://superset-aws-ssl.g.xx.com/login/
Location: http://superset-aws-ssl.g.xx.com/login/
< 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
* Connection #0 to host superset-aws-ssl.g.xx.com left intact
<p>You should be redirected automatically to target URL: <a href="http://superset-aws-ssl.g.xx.com/login/">http://superset-aws-ssl.g.xx.com/login/</a>.  If not click the link.* Closing connection 0

在上面的 curl 中，我们看到该位置被重定向到 HTTP。加上login重定向到login

这是入口控制器 yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "X-Forwarded-Proto: $scheme";
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:vvvv:certificate/xx-x-x-x-xxx
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  finalizers:
  - service.kubernetes.io/load-balancer-cleanup
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  name: ingress-nginx
  namespace: ingress-nginx
  resourceVersion: "6835154"
  selfLink: /api/v1/namespaces/ingress-nginx/services/ingress-nginx
  uid: xx-xx-xx-xx-xxxxx
spec:
  clusterIP: xx.xxx.xxx.xxx
  externalTrafficPolicy: Local
  healthCheckNodePort: 30599
  ports:
  - name: http
    nodePort: 31840
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 30330
    port: 443
    protocol: TCP
    targetPort: http
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  sessionAffinity: None
  type: LoadBalancer

这是超集服务顶部的入口：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "X-Forwarded-Proto: https";
  name: superset-ingress
  namespace: superset-ssl-v2
spec:
  rules:
  - host: https://superset-aws-ssl.g.xx.com
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          serviceName: superset
          servicePort: 8080

当我们移植服务时，它工作得很好。

这是 values.yaml

bootstrapScript: |
  #!/bin/bash
  if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi

security_manager: |
 ""

extraConfigs: 
 datasources-init.yaml: |
     databases:
     - database_name: {{ .Values.datasources.druid_db_name }}
       sqlalchemy_uri: {{ .Values.datasources.druid_uri }}
  
datasources:
  druid_uri: ""
  druid_db_name: ""  

configMountPath: "/app/pythonpath"

extraConfigMountPath: "/app/configs"

image:
  repository: docker.xx.com/superset
  tag: latest
  pullPolicy: Always

imagePullSecrets: []


service:
 type: LoadBalancer
 port: 8080

ingress:
  enabled: false
  annotations: 
    kubernetes.io/ingress.class: "plb.v1"
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - superset
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: 
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  limits:
    cpu: 2000m
    memory: 512Mi
  requests:
    cpu: 2000m
    memory: 512Mi
##
## Superset node configuration
supersetNode:
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; /usr/bin/docker-entrypoint.sh"
  connections: 
    db_host: ""
    db_port: ""
    db_user: ""
    db_pass: ""
    db_name: ""
  forceReload: false # If true, forces deployment to reload on each upgrade
  initContainers:
    - name: wait-for-postgres
      image: docker.xx.com/busybox:latest
      imagePullPolicy: IfNotPresent
      envFrom:
        - secretRef:
            name: '{{ tpl .Values.envFromSecret . }}'
      
      command: [ "/bin/sh", "-c", "until nc -zv $DB_HOST $DB_PORT -w1; do echo $DB_HOST; echo $DB_PORT; echo $DB_PASS; echo 'waiting for db'; sleep 1; done" ]
   
  ## Annotations to be added to supersetNode deployment
  deploymentAnnotations: {}

  ## Annotations to be added to supersetNode pods
  podAnnotations: {}

##
## Superset beat configuration (to trigger scheduled jobs like reports)
supersetCeleryBeat:
  # This is only required if you intend to use alerts and reports
  enabled: false
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; celery beat --app=superset.tasks.celery_app:app --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule"
  forceReload: false # If true, forces deployment to reload on each upgrade
  initContainers:
    - name: wait-for-postgres
      image: docker.xx.com/busybox:latest
      imagePullPolicy: IfNotPresent
      envFrom:
        - secretRef:
            name: '{{ tpl .Values.envFromSecret . }}'
      command: [ "/bin/sh", "-c", "until nc -zv $DB_HOST $DB_PORT -w1; do echo 'waiting for db'; sleep 1; done" ]
    
  ## Annotations to be added to supersetCeleryBeat deployment
  deploymentAnnotations: {}

  ## Annotations to be added to supersetCeleryBeat pods
  podAnnotations: {}

configOverrides:  
  superset_config.py: |
    CACHE_CONFIG = {
      'CACHE_TYPE': 'simple',
      'CACHE_DEFAULT_TIMEOUT': 300,
      'CACHE_KEY_PREFIX': 'superset_simple'
    }
    DATA_CACHE_CONFIG = {
      'CACHE_TYPE': 'simple',
      "CACHE_NO_NULL_WARNING": True,
      'CACHE_DEFAULT_TIMEOUT': 86400,
      'CACHE_THRESHOLD': 86400,
      'CACHE_KEY_PREFIX': 'superset_simple'
    }

    if "SUPERSET_HOME" in os.environ:
      DATA_DIR = os.environ["SUPERSET_HOME"]
    else:
      DATA_DIR = os.path.join(os.path.expanduser("~"), ".superset")
    LOG_FORMAT = "%(asctime)s:%(levelname)s:%(pathname)s:%(lineno)d:%(funcName)s:%(name)s:%(message)s"
    LOG_LEVEL = "DEBUG"

    # ---------------------------------------------------
    # Enable Time Rotate Log Handler
    # ---------------------------------------------------
    # LOG_LEVEL = DEBUG, INFO, WARNING, ERROR, CRITICAL

    ENABLE_TIME_ROTATE = True
    TIME_ROTATE_LOG_LEVEL = "DEBUG"
    FILENAME = os.path.join(DATA_DIR, "superset.log")
    ROLLOVER = "midnight"
    INTERVAL = 1
    BACKUP_COUNT = 30
    ENABLE_PROXY_FIX = True
    
    FEATURE_FLAGS = {
      "ROW_LEVEL_SECURITY": True,
      "DASHBOARD_RBAC": True,
      "DASHBOARD_CACHE": True,
      "CLIENT_CACHE": True,
      "ENABLE_PROXY_FIX": True
    }
    
#    from security_manager import MySecurityManager
#    CUSTOM_SECURITY_MANAGER = MySecurityManager

  enableProxyFix: "ENABLE_PROXY_FIX = True"
  enable_oauth: |
    ENABLE_PROXY_FIX = True
    
## Init job configuration
init:
  # Configure resources
  # Warning: fab commant consumes a lot of ram and can
  # cause the process to be killed due to OOM if it exceeds limit
  resources: 
    # limits:
    #   cpu:
    #   memory:
    # requests:
    #   cpu:
    #   memory:
    
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; . {{ .Values.configMountPath }}/superset_init.sh"
  enabled: true
  loadExamples: false
  adminUser:
    username: admin
    firstname: admin
    lastname: admin
    email: admin@superset.com
    password: admin
  initContainers:
    - name: wait-for-postgres
      image: docker.xx.com/busybox:latest
      imagePullPolicy: IfNotPresent
      envFrom:
        - secretRef:
            name: '{{ tpl .Values.envFromSecret . }}'

  initscript: |-
    #!/bin/sh
    echo "Upgrading DB schema..."
    superset db upgrade
    echo "Initializing roles..."
    superset init
    echo "Creating admin user..."
    superset fab create-admin \
                        --username {{ .Values.init.adminUser.username }} \
                        --firstname {{ .Values.init.adminUser.firstname }} \
                        --lastname {{ .Values.init.adminUser.lastname }} \
                        --email {{ .Values.init.adminUser.email }} \
                        --password {{ .Values.init.adminUser.password }} \
                        || true
    {{ if .Values.init.loadExamples }}
    echo "Loading examples..."
    superset load_examples
    {{- end }}
    if [ -f "{{ .Values.extraConfigMountPath }}/datasources-init.yaml" ]; then
      echo "Importing database connections.... "
      superset import_datasources -p {{ .Values.extraConfigMountPath }}/datasources-init.yaml
    fi
redis:
  enabled: false
postgresql:
  enabled: false

  cluster:
    enabled: false

nodeSelector: {}

tolerations: []

affinity: {}

livenessProbe:
  httpGet:
    path: /health
    port: http
  initialDelaySeconds: 80
  timeoutSeconds: 5
  periodSeconds: 10
  failureThreshold: 2
readinessProbe:
  httpGet:
    path: /health
    port: http
  initialDelaySeconds: 60
  timeoutSeconds: 5
  periodSeconds: 10
  failureThreshold: 2

标签： sslkubernetes-ingressnginx-ingressapache-superset

ssl - 在入口控制器为超集启用 SSL 终止

问题描述

解决方案

推荐阅读