时间戳

首页 > 解决方案 > 尝试连接到 grpc 服务，在隔离的 Docker 群上运行，通过 Traefik 暴露

docker - 尝试连接到 grpc 服务，在隔离的 Docker 群上运行，通过 Traefik 暴露

问题描述

我一直在尝试将 Traefik 配置为我的开发 docker swarm 的反向代理，它在自己的 VPN 上运行，与外界隔离。swarm 由 3 个管理节点和 3 个工作节点组成，它们之间没有任何 TLS 通信。我已经设法正确设置了一些前端（带有主机规则）和指向在 swarm 中运行的服务的后端。下面是我的 docker-compose 文件，我用它来为 swarm 上的 traefik 部署 docker 堆栈。

version: '3.6'

networks:
  traefik:
    driver: overlay

configs:
  traefik:
    file: ./config.dev.toml

services:
  traefik:
    image: traefik:1.7
    command:
      - --docker
      - --docker.swarmMode
      - --docker.domain=traefik.dev.pap
      - --docker.exposedByDefault=false
      - --entryPoints=Name:http Address::80 Redirect.EntryPoint:https
      - --entryPoints=Name:https Address::443 TLS:/certs/certfile.cert,/certs/keyfile.key
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    configs:
      - source: traefik
        target: /traefik.toml
    networks:
      - traefik
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
      - target: 8081
        published: 8081
        mode: host
      - target: 8082
        published: 8082
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: on-failure

traefik 的 toml 文件

defaultEntryPoints = ["http", "https"]

logLevel = "WARN"

insecureSkipVerify = true

[entryPoints]
  [entryPoints.http]
    address = ":80"
    [entryPoints.http.auth.basic]
    users = ["traefik:$2y$05$ZF08gDSv0f88vEUVzmwuxu.9seYIAZVzTMXnrETBDhQT30dxu4vOu"]
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
    address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      certFile = """-----BEGIN CERTIFICATE-----
                    MIICsjCCAZoCCQDP9WReX5fgQjANBgkqhkiG9w0BAQsFADAbMRkwFwYDVQQDDBAq
                    dHJhZWZpay5kZXYucGFwMB4XDTE5MDUyNDIwMTYzMloXDTIwMDUyMzIwMTYzMlow
                    GzEZMBcGA1UEAwwQKnRyYWVmaWsuZGV2LnBhcDCCASIwDQYJKoZIhvcNAQEBBQAD
                    ggEPADCCAQoCggEBAJ86nJMpyENKQrsDlfrMI8TwdAuHXXLpglMSPVmmTzm0Y7F7
                    CVRAwl8NJdbBJnBmK3+vaG1P3WfXfu/pBfn8GDrvlJd+1XqKdExMYMCDy2Ay1ghk
                    1mOcRkJ2XS2h7FjvGjAmE7/icYZprRziek/4hLCtpTjPwPIx/5tN/ZR+lpMXe+dG
                    24CqvcTftZCS+RZSMuACG0tKlmhOx/xjoUOQMiEA8ysmQE+RrEs7ZSvu1SLYMILa
                    jGrHmH/52kIia+zOJqi8k4pVTkRFB1CNeSURgFOQf1zWzSI49GTh0UGsmlgeMoOX
                    FgrEjsbmMKxCz8ItPiTMjJ1zLd5YyBt+ATeDl0kCAwEAATANBgkqhkiG9w0BAQsF
                    AAOCAQEAYXrBbyJ75TMInYmPjkPD/Vdh3dI62qwHrKJsXN519FC8gbYSJFSdr5Pd
                    y/stM2Xn8KGgnkLSuo62MzGD15X/IYqn4Kt9Eizqd5kpsdNc7l/pTTidcCY3nQ82
                    CUGQrnwyVjZ/8wwbjtTL1TOismK3yA169WXD2yXGz15fR08lLCMqXEKotl82FxfX
                    6mw9EkLC3MAfIxuuWcaCl8/AHAUlrzrQOoVj9OkBcxwYeYH++KKYOGCaiuKMen9x
                    cjyCo05qmBQq4PWOoafDJMrdAcnzjc0tTW3DBTnJY7tR0qA/6iGVsdlHce6rMZtR
                    sb5m6XFgABCcg54yfrHIGgv1Te78aA==
                    -----END CERTIFICATE-----
                    """
      keyFile = """-----BEGIN PRIVATE KEY-----
                    MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQCfOpyTKchDSkK7
                    A5X6zCPE8HQLh11y6YJTEj1Zpk85tGOxewlUQMJfDSXWwSZwZit/r2htT91n137v
                    6QX5/Bg675SXftV6inRMTGDAg8tgMtYIZNZjnEZCdl0toexY7xowJhO/4nGGaa0c
                    4npP+ISwraU4z8DyMf+bTf2UfpaTF3vnRtuAqr3E37WQkvkWUjLgAhtLSpZoTsf8
                    Y6FDkDIhAPMrJkBPkaxLO2Ur7tUi2DCC2oxqx5h/+dpCImvsziaovJOKVU5ERQdQ
                    jXklEYBTkH9c1s0iOPRk4dFBrJpYHjKDlxYKxI7G5jCsQs/CLT4kzIydcy3eWMgb
                    fgE3g5dJAgMBAAECggEBAJb3QyxkGPNznQEizTgUBHXcZKnplbg1nNKWIIS3IbZc
                    SYkkVhwDqkXkeVO7nW5D8EoHTMMXTZv+vONJ09jII1D9hax9HHoqe0k7wR/F1TVH
                    /ZXHxVx79kFdZ8q0Exne2L193Iu5blk6KF+SetUHkwbQHC5ZC16V+PqOXzZDLxAV
                    XOXjv5nOK2bqffgZdjYmW3fiwARJ1y/2s6G2Pn3Uk9z3Hk259ZI6k1zE4mwmf62t
                    FQJfjDQ7Ej7N0lXwmSA/Kb7qTXQ/div+juGlEjKTZIZRJMcLOxaZMo93sU8T/Y6b
                    wWnsRvEZlZ9yy8Y/OWN3UxWjc6nx8DPI6sYnTBUNEmECgYEA0Co6Px6n4CFtmeT6
                    6vj/YNGY1vzrycqixtymaIJk7TmPXeLqL919Sro/jbI0B189tnm3DFLk458feRaA
                    Nw+Jl1vQo9IjutJ1ujNt7IrCtlP6smRpQ5fXQ9Ywk3IPtZiG9W+fxM1uHSYKoAh/
                    sAym38Ao5hPRoRJ0TJxvlkCJtz8CgYEAw9GdFa+CXcx9U8pZE1tV2J9gXjd0mr10
                    OKjpKA9ez8IOB9S96bNOiEwHzzsza+OFoou1DrIK0zoK2+XYZYJ+sWOU1pO4PYlj
                    vbRC5kFb2EvD5h47oIoAWqk3QOGewM+z6eWI9r4LTOgdiZ1r6c73jydGGtTBtcZq
                    oqgVOziHV3cCgYEAgtw495se77A0BU3ES1iyujuo+GXRGG7xe1AUsJhgOB06Okw+
                    6k4zyHljIIGJYaUG16KWZwHpcAqFL4rsmTCET4lhlMqyuWw6pA5WqR/mHdaXP/QN
                    lHds1PFeYiKPK5qyJLLCXmbCNULH5SjSlGqvsPNNAq4s1zQRtmOVZoA10uMCgYBw
                    laPrQ+2AYIHuGvj72BeTZI7TnK6+CQ85RGRJVAz/BmNqvTYzl5QfaRdUXoVCEp5S
                    xmBjApqWt1hm9c9lDZIqC0cX1/al6sgd2fZczFcyFN2dAOTneHCbr47FDvNniAO5
                    Bu+obbuxjALvjELvOr+63qm/43M8P8gvnNQYKHgoIQKBgQCQJrT3z27O/hRVUIGg
                    bwAmzwGExOC39uLHNWmOcVmw36oV592sVw1K9pRvPFVJp956T63lhcGv4t0ki2DE
                    gBN/V1jhkAw33qRDd5bbYmIutf6RO7HeewwFHpbaLw0FULe5UlsziE9AbTHmddqU
                    GTcyajXnqimXvSu6AC9XFA9s4g==
                    -----END PRIVATE KEY-----
                   """
  [entryPoints.api]
    address = ":8081"
  [entryPoints.ping]
    address = ":8082"

[api]
  entryPoint = "api"
  dashboard = true

[ping]
  entryPoint = "ping"

[retry]
  logLevel = "DEBUG"

[traefikLog]
  logLevel = "DEBUG"
  format   = "json"

[accessLog]
  logLevel = "DEBUG"
  format = "json"

[docker]
  endpoint = "unix:///var/run/docker.sock"
  domain = "traefik.dev.pbp"
  swarmmode = true
  watch = true
  exposedByDefault = false

上面的证书是在我的笔记本电脑上使用命令生成的： openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./backend.key -out ./backend.cert和 Common Name: *traefik.dev.pap。

我要公开的 2 个服务具有以下标签：

      labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_traefik"
      # Public segment
      - "traefik.public.protocol=h2c"
      - "traefik.public.port=50051"
      - "traefik.public.frontend.rule=Host:serviceA-grpc.traefik.dev.pbp"
      - "traefik.public.frontend.auth.forward.tls.insecureSkipVerify=true"
      - "traefik.public.backend.loadbalancer.method=drr"
      - "traefik.public.frontend.passHostHeader=true"

      labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_traefik"
      # Public segment
      - "traefik.public.protocol=http"
      - "traefik.public.port=8080"
      - "traefik.public.frontend.rule=Host:serviceB-gateway.traefik.dev.pbp"
      - "traefik.public.frontend.passHostHeader=true"
      - "traefik.public.backend.loadbalancer.method=drr"

第一个是我正在运行的 grpc 服务，第二个是 grpc 服务的 HTTP 网关。第二个按预期工作。除了证书不安全这一事实之外，没有什么奇怪的。但是，我发现让 Traefik 暴露 grpc 服务真的很难。如果我执行grpcurl -plaintext tag-service-grpc.traefik.dev.pbp:443 list，我会得到结果Failed to list services: rpc error: code = Unknown desc = Internal Server Error ，并且 Traefic 日志是

{"BackendAddr":"","BackendName":"Traefik","BackendURL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"ClientAddr":"x.x.x.x:64391","ClientHost":"x.x.x.x","ClientPort":"64391","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":500,"DownstreamStatusLine":"500 Internal Server Error","Duration":426232,"FrontendName":"Auth for frontend-public-tag-service-public","OriginContentSize":0,"OriginDuration":222656,"OriginStatus":500,"OriginStatusLine":"500 Internal Server Error","Overhead":203576,"RequestAddr":"serviceA-grpc.traefik.dev.pap:443","RequestContentSize":0,"RequestCount":14,"RequestHost":"serviceA-grpc.traefik.dev.pap","RequestLine":"POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0","RequestMethod":"POST","RequestPath":"/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","RequestPort":"443","RequestProtocol":"HTTP/2.0","RetryAttempts":0,"StartLocal":"2019-05-24T20:24:24.216358162Z","StartUTC":"2019-05-24T20:24:24.216358162Z","level":"info","msg":"","request_Content-Type":"application/grpc","request_Te":"trailers","request_User-Agent":"grpc-go/1.19.0","time":"2019-05-24T20:24:24Z"}
{"BackendAddr":"","BackendName":"Traefik","BackendURL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"ClientAddr":"x.x.x.x:64391","ClientHost":"x.x.x.x","ClientPort":"64391","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":500,"DownstreamStatusLine":"500 Internal Server Error","Duration":398352,"FrontendName":"Auth for frontend-public-tag-service-public","OriginContentSize":0,"OriginDuration":142857,"OriginStatus":500,"OriginStatusLine":"500 Internal Server Error","Overhead":255495,"RequestAddr":"serviceA-grpc.traefik.dev.pap:443","RequestContentSize":0,"RequestCount":15,"RequestHost":"serviceA-grpc.traefik.dev.pap","RequestLine":"POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0","RequestMethod":"POST","RequestPath":"/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","RequestPort":"443","RequestProtocol":"HTTP/2.0","RetryAttempts":0,"StartLocal":"2019-05-24T20:24:24.297632103Z","StartUTC":"2019-05-24T20:24:24.297632103Z","level":"info","msg":"","request_Content-Type":"application/grpc","request_Te":"trailers","request_User-Agent":"grpc-go/1.19.0","time":"2019-05-24T20:24:24Z"}

如果我执行grpcurl -plaintext docker-swarm-worker-01.dev.pap:50061 list，我会得到我可以调用的服务列表。有人能帮助我吗？

标签： dockerdocker-composedocker-swarmgrpctraefik

解决方案

I managed to make it work! It seems that the problem was the redirecting of http to https and the basic http authentication that I was using.

Traefik.toml config file now is simpler:

defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.http]
    address = ":80"
    compress = true
  [entryPoints.https]
    address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      certFile = "/certs/certfile.cert"
      keyFile = "/certs/keyfile.key"

  [entryPoints.api]
    address = ":8081"
  [entryPoints.ping]
    address = ":8082"

[api]
  entryPoint = "api"
  dashboard = true

[ping]
  entryPoint = "ping"

[retry]

[traefikLog]
  format   = "json"

[accessLog]
  format = "json"

[docker]
  endpoint = "unix:///var/run/docker.sock"
  domain = "traefik.dev.pbp"
  swarmmode = true
  watch = true
  exposedByDefault = false
  insecureSkipVerify = true

Now I can connect to my grpc service running on Docker swarm through Traefik!

推荐阅读