kubernetes - 如何让 hasura websocket 在我的本地 Kubernetes 集群上工作?
问题描述
我已经在 Windows 下建立了一个本地 K8s 集群,如下所示:
- 为桌面安装 docker
- 在 docker for desktop 中,启用 kubernetes
- 安装 nginx 入口控制器
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.29.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.29.0/deploy/static/provider/cloud-generic.yaml
- 将以下域添加到主机 (
C:\Windows\System32\drivers\etc\hosts
)
127.0.0.1 localhost api.shopozor
我在这里没有做任何特别的事情,我将所有内容都保留为默认设置。
然后,我使用以下 yamls 将 hasura 部署到我的集群(为简洁起见,我没有显示 postgres 部署):
---
# Source: api/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: api
labels:
app.kubernetes.io/name: api
helm.sh/chart: api-0.0.0
app.kubernetes.io/instance: api
app.kubernetes.io/version: "0.0"
app.kubernetes.io/managed-by: Helm
type: Opaque
data:
admin-secret: "c2VjcmV0"
---
# Source: api/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: api
labels:
app.kubernetes.io/name: api
helm.sh/chart: api-0.0.0
app.kubernetes.io/instance: api
app.kubernetes.io/version: "0.0"
app.kubernetes.io/managed-by: Helm
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
# TODO: we cannot use string port because devspace doesn't support it in its UI
# targetPort: http
protocol: TCP
name: http
selector:
app.kubernetes.io/name: api
app.kubernetes.io/instance: api
---
# Source: api/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
labels:
app.kubernetes.io/name: api
helm.sh/chart: api-0.0.0
app.kubernetes.io/instance: api
app.kubernetes.io/version: "0.0"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: api
app.kubernetes.io/instance: api
template:
metadata:
labels:
app.kubernetes.io/name: api
app.kubernetes.io/instance: api
spec:
serviceAccountName: api
securityContext:
{}
initContainers:
# App has to wait for the database to be online "depends_on" workaround
- name: wait-for-db
image: darthcabs/tiny-tools:1
args:
- /bin/bash
- -c
- >
set -x;
while [[ "$(nc -zv 'postgres' 5432 &> /dev/null; echo $?)" != 0 ]]; do
echo '.'
sleep 15;
done
containers:
- name: api
securityContext:
{}
image: shopozor/graphql-engine:EM5Aya
imagePullPolicy:
env:
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: shared-postgresql
key: postgresql-username
- name: POSTGRES_DATABASE
value: shopozor
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: shared-postgresql
key: postgresql-password
- name: POSTGRES_HOST
value: postgres
- name: POSTGRES_PORT
value: "5432"
- name: HASURA_GRAPHQL_SERVER_PORT
value: "8080"
- name: HASURA_GRAPHQL_ENABLE_CONSOLE
value: "true"
- name: HASURA_GRAPHQL_ENABLED_LOG_TYPES
value: startup, http-log, webhook-log, websocket-log, query-log
- name: HASURA_GRAPHQL_ENABLE_TELEMETRY
value: "false"
- name: HASURA_GRAPHQL_CORS_DOMAIN
value: "*"
- name: HASURA_GRAPHQL_DISABLE_CORS
value: "false"
- name: HASURA_GRAPHQL_UNAUTHORIZED_ROLE
value: incognito
- name: HASURA_GRAPHQL_ADMIN_SECRET
valueFrom:
secretKeyRef:
name: api
key: admin-secret
- name: HASURA_GRAPHQL_JWT_SECRET
value: "{\"type\": \"HS256\", \"key\": \"my-access-token-signing-key-secret\", \"audience\": [\"58640fbe-9a6c-11ea-bb37-0242ac130002\", \"6e707590-9a6c-11ea-bb37-0242ac130002\"], \"claims_namespace\": \"https://hasura.io/jwt/claims\", \"claims_format\": \"json\", \"issuer\": \"shopozor.com\" }"
- name: HASURA_GRAPHQL_DATABASE_URL
value: postgres://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):$(POSTGRES_PORT)/$(POSTGRES_DATABASE)
- name: FUNCTION_NAMESPACE
value: dev
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: http
readinessProbe:
httpGet:
path: /healthz
port: http
resources:
{}
---
# Source: api/templates/ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: api
labels:
app.kubernetes.io/name: api
helm.sh/chart: api-0.0.0
app.kubernetes.io/instance: api
app.kubernetes.io/version: "0.0"
app.kubernetes.io/managed-by: Helm
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: "api.shopozor"
http:
paths:
- path: /
backend:
serviceName: api
servicePort: 8080
现在,我有一个尝试使用 hasura 的 websocket 的 nuxt 前端。我以标准方式配置了 apollo
//---
// nuxt.config.js
[...]
// Give apollo module options
apollo: {
cookieAttributes: {
expires: 7
},
includeNodeModules: true,
authenticationType: 'Basic',
clientConfigs: {
default: '~/apollo/clientConfig.js'
}
},
[...]
//---
// apollo/clientConfig.js
import { InMemoryCache } from 'apollo-cache-inmemory'
export default function (context) {
return {
httpLinkOptions: {
uri: 'http://api.shopozor/v1/graphql',
credentials: 'same-origin'
},
cache: new InMemoryCache(),
wsEndpoint: 'ws://localhost:8080/v1/graphql'
}
}
请注意,我目前不需要特定的标题。我应该能够在没有授权令牌的情况下访问 websocket。
现在,当我启动我的应用程序时,websocket 连接会尝试初始化。如果我端口转发我的 hasura 服务,那么上面的配置就可以了。websocket 连接似乎工作正常。至少,哈苏拉日志显示
2020-07-16T06:49:59.937386882Z {"type":"websocket-log","timestamp":"2020-07-16T06:49:59.937+0000","level":"info","detail":{"event":{"type":"accepted"},"connection_info":{"websocket_id":"8437b784-1fce-4430-9ca9-a9e7517307f0","token_expiry":null,"msg":null},"user_vars":null}}
但是,如果我更改wsEndpoint
上述配置以使用进入我的 hasura 实例的入口,
wsEndpoint: 'ws://api.shopozor/v1/graphql'
然后它不再起作用了。相反,我不断得到一个404 Not Found
. 但是,我可以通过http://api.shopozor
. hasura 日志显示
2020-07-16T10:37:53.564657244Z {"type":"websocket-log","timestamp":"2020-07-16T10:37:53.564+0000","level":"error","detail":{"event":{"type":"rejected","detail":{"path":"$","error":"only '/v1/graphql', '/v1alpha1/graphql' are supported on websockets","code":"not-found"}},"connection_info":{"websocket_id":"5e031467-fb5c-460d-b2a5-11f1e21f22e7","token_expiry":null,"msg":null},"user_vars":null}}
所以我搜索了很多,找到了一些关于我应该在我的入口中使用的注释的信息,但没有任何效果。我在这里想念什么?我需要特定的 nginx 入口控制器配置吗?我是否需要将一些特殊注释传递给我的 hasura 入口?我需要做什么才能使其工作?
编辑
要回答这篇文章的问题,这是我在集群上应用的 hasura 入口:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: api
meta.helm.sh/release-namespace: dev
labels:
app.kubernetes.io/instance: api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: api
app.kubernetes.io/version: '0.0'
helm.sh/chart: api-0.0.0
name: api
spec:
rules:
- host: api.shopozor
http:
paths:
- backend:
serviceName: api
servicePort: 8080
path: /
编辑 2
对于我的 hasura 实例,使用以下入口
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: api
meta.helm.sh/release-namespace: dev
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
labels:
app.kubernetes.io/instance: api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: api
app.kubernetes.io/version: '0.0'
helm.sh/chart: api-0.0.0
name: api
spec:
rules:
- host: api.shopozor
http:
paths:
- backend:
serviceName: api
servicePort: 8080
path: /
前端应用程序仍然存在与 websockets 相同的问题。此外,它已经无法真正连接到hasura了。相反,我收到以下错误:
ERROR Network error: Unexpected token < in JSON at position 0 23:17:06
at new ApolloError (D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:92:26)
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:1588:34
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:2008:15
at Set.forEach (<anonymous>)
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:2006:26
at Map.forEach (<anonymous>)
at QueryManager.broadcastQueries (D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:2004:20)
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:1483:29
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Global error handler 23:17:06
ERROR Network error: Unexpected token < in JSON at position 0 23:17:06
at new ApolloError (D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:92:26)
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:1486:27
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
也
client.js?06a0:49 ApolloError: Network error: Unexpected token < in JSON at position 0
at new ApolloError (D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:92:26)
at D:\workspace\shopozor\services\node_modules\apollo-client\bundle.umd.js:1486:27
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
graphQLErrors: [],
networkError: SyntaxError [ServerParseError]: Unexpected token < in JSON at position 0
at JSON.parse (<anonymous>)
at D:\workspace\shopozor\services\node_modules\apollo-link-http-common\lib\index.js:35:25
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
name: 'ServerParseError',
response: Body {
url: 'http://api.shopozor/v1/graphql/',
status: 404,
statusText: 'Not Found',
headers: [Headers],
ok: false,
body: [PassThrough],
bodyUsed: true,
size: 0,
timeout: 0,
_raw: [Array],
_abort: false,
_bytes: 153
},
statusCode: 404,
bodyText: '<html>\r\n' +
'<head><title>404 Not Found</title></head>\r\n' +
'<body>\r\n' +
'<center><h1>404 Not Found</h1></center>\r\n' +
'<hr><center>nginx/1.17.8</center>\r\n' +
'</body>\r\n' +
'</html>\r\n'
},
message: 'Network error: Unexpected token < in JSON at position 0',
extraInfo: undefined
}
和
index.js?a6d6:111 OPTIONS http://api.shopozor/v1/graphql/ net::ERR_ABORTED 404 (Not Found)
和
Access to fetch at 'http://api.shopozor/v1/graphql/' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
没有新的注释
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
在我的 hasura 入口上,我的前端和 hasura 之间的连接工作正常,除了 websockets。
编辑 3
我尝试了以下两个入口,但没有成功:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: api
meta.helm.sh/release-namespace: dev
nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
labels:
app.kubernetes.io/instance: api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: api
app.kubernetes.io/version: '0.0'
helm.sh/chart: api-0.0.0
name: api
spec:
rules:
- host: api.shopozor
http:
paths:
- backend:
serviceName: api
servicePort: 8080
path: /
和
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: api
meta.helm.sh/release-namespace: dev
nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
labels:
app.kubernetes.io/instance: api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: api
app.kubernetes.io/version: '0.0'
helm.sh/chart: api-0.0.0
name: api
spec:
rules:
- host: api.shopozor
http:
paths:
- backend:
serviceName: api
servicePort: 8080
path: /
在后一种情况下,我只是得到错误
WebSocket connection to 'ws://api.shopozor/v1/graphql/' failed: Error during WebSocket handshake: Unexpected response code: 404
而 graphql 端点功能正常。在前一种情况下,我无法访问 hasura 实例上的任何内容,并且我得到了上面的 websocket 握手问题(所以没有 graphql 也没有 websocket 工作)。
编辑 4
使用我的 api 入口配置(没有任何额外的 nginx 注释,如上面:、、、nginx.ingress.kubernetes.io/proxy-read-timeout
) nginx.ingress.kubernetes.io/proxy-send-timeout
,nginx.ingress.kubernetes.io/configuration-snippet
如果我这样做:
curl -i -N -H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Origin: http://localhost:3000" \
-H "Host: api.shopozor" \
-H "Sec-Websocket-Version: 13" \
-H "Sec-WebSocket-Key: B8KgbaRLCMNCREjE5Kvg1w==" \
-H "Sec-WebSocket-Protocol: graphql-ws" \
-H "Accept-Encoding: gzip, deflate" \
-H "Accept-Language: en-US,en;q=0.9" \
-H "Cache-Control: no-cache" \
-H "Pragma: no-cache" \
-H "Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits" \
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.49 Safari/537.36" \
http://api.shopozor/v1/graphql
然后哈苏拉很高兴:
2020-07-28T07:42:28.903877263Z {"type":"websocket-log","timestamp":"2020-07-28T07:42:28.894+0000","level":"info","detail":{"event":{"type":"accepted"},"connection_info":{"websocket_id":"94243bde-41c4-42c8-8d8f-355c47a3492e","token_expiry":null,"msg":null},"user_vars":null}}
我上面 curl 中的标头与我的前端应用程序发送的标头完全相同。关于我做错了什么的任何线索?前端调用和此 curl 之间的区别在于,在前端我将 websocket 端点定义为ws://api.shopozor/v1/graphql
当我 curl 时http://api.shopozor/v1/graphql
。在 apollo vue 中,我无法wsEndpoint
将http://api.shopozor/v1/graphql
. 我得到一个错误。
解决方案
听起来您正在使用 nginx 入口控制器,并且开箱即用地支持 WebSocket。
你可以试试这个注释:
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
或/和其他注释,因为他们的文档 建议将其用于 WebSockets:
nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
注意:我前一阵子回答了一个类似的问题⌛⌚。
推荐阅读
- c# - asp.net:如何在我的应用程序中获取另一个页面的 URL?
- php - MySQL选择单元格值等于数字的行,其中非数字字符被删除
- java - 在php中生成带有位置的真值表
- java - new 运算符总是返回相同的引用
- reactjs - 多选大纲变体属性不起作用
- angular - 在组件之间使用带参数的回调函数
- javascript - 如果对象值在javascript中为真,则迭代数组提取并添加到新列表
- python - Tkinter:使用 TopLevel 窗口的存在作为条件
- apache-flink - Apache Flink 依次写入多个接收器
- python - Flask-SQL 中的关联对象在 shell 中表现,但在网站上创建“非 JSONable”错误