spring-boot - 设置 GOOGLE_APPLICATION_CREDENTIALS 后，Google 云存储为部署在 GKE 上的 Spring 应用程序返回 401

Question

我有一个在 Google Cloud Storage 中推送数据的 spring 应用程序。我生成了一个新的服务帐户 json 文件，并在我的 Windows 测试期间使用它。一切正常。

然后我将我的应用程序 dockerise 并尝试在 Kubernetes 上进行部署。为了将 dockerise 应用程序与云存储连接，我使用我的服务帐户 json 文件创建了一个秘密

kubectl create secret generic cloud-storage-credentials \
 --from-file=cloudstorage.json=cloud-storage-credentials.json

然后我在我的部署文件中挂载云存储文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: "{{ include "m-ebook.name" . }}-deployment"
  labels:
    app: {{ include "m-ebook.name" . }}
    tier: backend
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ include "m-ebook.name" . }}
      tier: backend
  template:
    metadata:
      labels:
        app: {{ include "m-ebook.name" . }}
        tier: backend
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          env:
            #Cloud storage
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: /var/secrets/google/cloudstorage.json
            #Cloud sql
            - name: DB_HOST
              value: 127.0.0.1:3306
            # These secrets are required to start the pod.
            # [START cloudsql_secrets]
            # The db name is set directly in the back end propeties files
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: password
        #Rabbitmq cookie erlang (needed to connect to rabbitmq)
            - name: RABBITMQ_ERLANG_COOKIE
              valueFrom:
                secretKeyRef:
                  name: rabbitmq
                  key: erlangCookie
          # [END cloudsql_secrets]
        # Change <INSTANCE_CONNECTION_NAME> here to include your GCP
        # project, the region of your Cloud SQL instance and the name
        # of your Cloud SQL instance. The format is
        # $PROJECT:$REGION:$INSTANCE
        # [START proxy_container]
        - name: cloudsql-proxy
          image: gcr.io/cloudsql-docker/gce-proxy:1.11
          command: ["/cloud_sql_proxy",
                  "-instances=gara-261618:europe-west1:gara-postgresql-server=tcp:3306",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
          # [START cloudsql_security_context]
          securityContext:
            runAsUser: 2  # non-root user
            allowPrivilegeEscalation: false
          # [END cloudsql_security_context]
          volumeMounts:
            - name: cloudsql-instance-credentials
              mountPath: /secrets/cloudsql
              readOnly: true
            - name: cloud-storage-credentials-volume
              mountPath: /var/secrets/google
              readOnly: true
        # [END proxy_container]
          ports:
            - name: http
              containerPort: {{ .Values.service.internalPort }}
              protocol: TCP
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
    {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
    {{- end }}
    {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
      imagePullSecrets: 
        - name: registry-gitlab-secrets
      # [START volumes]
      volumes:
        - name: cloudsql-instance-credentials
          secret:
            secretName: cloudsql-instance-credentials
        - name: cloud-storage-credentials-volume
          secret: #The name of the secret as defined in create secret generic cloud-storage-credentials
            secretName: cloud-storage-credentials
      # [END volumes]

编辑秘密是从谷歌生成的文件，看起来像：

{
  "type": "service_account",
  "project_id": "myproject-261618",
  "private_key_id": "3d1625a7428367cfb274251",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgk...Z7XSQik\nYWSPGLxNDlopi+DLDzzHvJtO\n-----END PRIVATE KEY-----\n",
  "client_email": "[e-mail]",
  "client_id": "1023295593410734556",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/name%40nameofmyproject.iam.gserviceaccount.com"
}

编辑

kubectl get secret cloud-storage-credentials -o yaml
apiVersion: v1
data:
  cloudstorage.json: ewogICJ0eXBlIjogInNlcnZpY2V...DUwOV9jZXJ0X3VybCI6ICJodHRwczovL3d3dy5nb29nbGVhcGlzLmNvbS9yb2JvdC92MS9tZXRhZGF0YS94NTA5L2dhcmEtY2xvdWQtc3RvcmFnZS1pbnQtZGV2JTQwZ2FyYS0yNjE2MTguaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iCn0K
kind: Secret
metadata:
  creationTimestamp: "2020-02-27T16:19:27Z"
  name: cloud-storage-credentials
  namespace: default
  resourceVersion: "23609"
  selfLink: /api/v1/namespaces/default/secrets/cloud-storage-credentials
  uid: ecb4821d-597c-...010af001ab
type: Opaque

编辑当我尝试读取代码中的属性并检查文件是否存在时，会读取属性，但不存在。

5    [http-nio-9103-exec-3] INFO  com.gara.mebooks.services.googlecloud.implement.GcloudStorageServiceImpl  - /var/secrets/google/cloudstorage.json
5    [http-nio-9103-exec-3] INFO  com.gara.mebooks.services.googlecloud.implement.GcloudStorageServiceImpl  - is the file exists ? false

我不知道为什么，但是当我调用我的服务时，我收到了来自 Google 的 401。请问我做错了什么？我进入容器内部，当我输入时，echo $GOOGLE_APPLICATION_CREDENTIALS我得到了/var/secrets/google/cloudstorage.json但是当我尝试时cat /var/secrets/google/cloudstorage.json，我得到了文件未找到异常。正常吗？

提前致谢

标签： spring-bootkubernetesgoogle-cloud-storagegoogle-kubernetes-enginekubernetes-helm

Answer 1

您的部署上有两个不同的容器。您在第一个容器中设置环境变量并在第二个容器中安装卷。

这是您的代码的外观。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: "{{ include "m-ebook.name" . }}-deployment"
  labels:
    app: {{ include "m-ebook.name" . }}
    tier: backend
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ include "m-ebook.name" . }}
      tier: backend
  template:
    metadata:
      labels:
        app: {{ include "m-ebook.name" . }}
        tier: backend
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          env:
            #Cloud storage
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: /var/secrets/google/cloudstorage.json
            #Cloud sql
            - name: DB_HOST
              value: 127.0.0.1:3306
            # These secrets are required to start the pod.
            # [START cloudsql_secrets]
            # The db name is set directly in the back end propeties files
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql-db-credentials
                  key: password
          volumeMounts:
            - name: cloud-storage-credentials-volume
              mountPath: /var/secrets/google
              readOnly: true
        #Rabbitmq cookie erlang (needed to connect to rabbitmq)
            - name: RABBITMQ_ERLANG_COOKIE
              valueFrom:
                secretKeyRef:
                  name: rabbitmq
                  key: erlangCookie
          # [END cloudsql_secrets]
        # Change <INSTANCE_CONNECTION_NAME> here to include your GCP
        # project, the region of your Cloud SQL instance and the name
        # of your Cloud SQL instance. The format is
        # $PROJECT:$REGION:$INSTANCE
        # [START proxy_container]
        - name: cloudsql-proxy
          image: gcr.io/cloudsql-docker/gce-proxy:1.11
          command: ["/cloud_sql_proxy",
                  "-instances=gara-261618:europe-west1:gara-postgresql-server=tcp:3306",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
          # [START cloudsql_security_context]
          securityContext:
            runAsUser: 2  # non-root user
            allowPrivilegeEscalation: false
          # [END cloudsql_security_context]
          volumeMounts:
            - name: cloudsql-instance-credentials
              mountPath: /secrets/cloudsql
              readOnly: true
        # [END proxy_container]
          ports:
            - name: http
              containerPort: {{ .Values.service.internalPort }}
              protocol: TCP
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
    {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
    {{- end }}
    {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
      imagePullSecrets: 
        - name: registry-gitlab-secrets
      # [START volumes]
      volumes:
        - name: cloudsql-instance-credentials
          secret:
            secretName: cloudsql-instance-credentials
        - name: cloud-storage-credentials-volume
          secret: #The name of the secret as defined in create secret generic cloud-storage-credentials
            secretName: cloud-storage-credentials
      # [END volumes]