google-cloud-platform - 通过 Terraform 安装 GKE 和 Anthos

Question

我正在尝试启动 GKE 集群，然后安装 Anthos，但我遇到了以下问题。

 module "gke" {
      source                      = "./gke_private_cluster"
      project_id                  = var.project_id
      name                        = var.cluster_name
      regional                    = false
      region                      = var.region
      zones                       = var.zones
      network                     = module.gcp-network.network_name
      subnetwork                  = module.gcp-network.subnets_names[0]
      ip_range_pods               = var.ip_range_pods
      ip_range_services           = var.ip_range_services
    }
    module "gke_node_pool11" {
      count                          = var.count_primary_np
      source                         = "./gke_node_pool"
      project_id                     = var.project_id
      gke_cluster_name               = module.gke.name
      node_pool_name                 = "${var.node_pool_name}-${count.index + 1}"
      region                         = var.region
      regional                       = var.regional
      zones                          = var.zones_01
      gke_cluster_min_master_version = var.gke_cluster_min_master_version
      image_type                     = var.image_type
      machine_type                   = var.machine_type_01[count.index]
      preemptible                    = false
    }
    module "gke_node_pool_preemptible22" {
      count                          = var.count_preemptible_np
      source                         = "./gke_node_pool"
      project_id                     = var.project_id
      gke_cluster_name               = module.gke.name
      node_pool_name                 = "${var.preemptible_node_pool_name}-${count.index + 1}"
      region                         = var.region
      regional                       = var.regional
      zones                          = var.zones_02
      gke_cluster_min_master_version = var.gke_cluster_min_master_version
      image_type                     = var.image_type
      machine_type                   = var.machine_type_02[count.index]
      preemptible                    = true
    }

module "hub-primary" {
  source                  = "terraform-google-modules/kubernetes-engine/google//modules/hub"
  project_id              = var.project_id
  cluster_name            = module.gke.name
  cluster_endpoint        = module.gke.endpoint
  location                = module.gke.location
  gke_hub_membership_name = "primary"
  gke_hub_sa_name         = "primary"
}
module "asm" {
  source                = "github.com/terraform-google-modules/terraform-google-kubernetes-engine//modules/asm"
  cluster_name          = module.gke.name
  cluster_endpoint      = module.gke.endpoint
  project_id            = var.project_id
  location              = module.gke.location
  enable_all            = true
  enable_cluster_roles  = true
  enable_cluster_labels = false
  enable_gcp_apis       = true
  enable_gcp_iam_roles  = false
  enable_gcp_components = true
  enable_registration   = false
  asm_version           = "1.10"
  managed_control_plane = false
  options               = ["envoy-access-log,egressgateways"]
  skip_validation       = false
  outdir                = "./${module.gke.name}-outdir-${var.asm_version}"
}

当我运行上面的 terraform 代码时，我遇到了以下错误：

dial tcp 35.244.8.53:443: connect: connection refused not found
│ Error: failed to install manifests: errors occurred during operation
│ 2021-09-06T08:23:21.219364 install_asm_1.10: [WARNING]: Failed, retrying...(1 of 5)
│ 2021-09-06T08:23:23.252989 install_asm_1.10: Running: './istio-1.10.4-asm.6/bin/istioctl install -f asm/istio/istio-operator.yaml -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml01 -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml01 --set
│ revision=asm-1104-6 -c asm_kubeconfig --skip-confirmation'
│ 2021-09-06T08:23:23.284655 install_asm_1.10: -------------
│ Error: Get "https://35.244.8.53/api?timeout=32s": dial tcp 35.244.8.53:443: connect: connection refused
│ 2021-09-06T08:23:23.516704 install_asm_1.10: [WARNING]: Failed, retrying...(2 of 5)
│ 2021-09-06T08:23:27.574012 install_asm_1.10: Running: './istio-1.10.4-asm.6/bin/istioctl install -f asm/istio/istio-operator.yaml -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml01 -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml01 --set
│ revision=asm-1104-6 -c asm_kubeconfig --skip-confirmation'
│ 2021-09-06T08:23:27.617173 install_asm_1.10: -------------
│ Error: Get "https://35.244.8.53/api?timeout=32s": dial tcp 35.244.8.53:443: connect: connection refused
│ 2021-09-06T08:23:27.836374 install_asm_1.10: [WARNING]: Failed, retrying...(3 of 5)
│ 2021-09-06T08:23:33.873300 install_asm_1.10: Running: './istio-1.10.4-asm.6/bin/istioctl install -f asm/istio/istio-operator.yaml -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml01 -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml01 --set
│ revision=asm-1104-6 -c asm_kubeconfig --skip-confirmation'
│ 2021-09-06T08:23:33.902169 install_asm_1.10: -------------
│ Error: Get "https://35.244.8.53/api?timeout=32s": dial tcp 35.244.8.53:443: connect: connection refused
│ 2021-09-06T08:23:34.117136 install_asm_1.10: [WARNING]: Failed, retrying...(4 of 5)
│ 2021-09-06T08:23:42.149546 install_asm_1.10: Running: './istio-1.10.4-asm.6/bin/istioctl install -f asm/istio/istio-operator.yaml -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-envoy-access-log.yaml01 -f
│ /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml00 -f /home/sanjay_m/terraform-gcp-3/santest-outdir-1.10/overlay-egressgateways.yaml01 --set
│ revision=asm-1104-6 -c asm_kubeconfig --skip-confirmation'
│ 2021-09-06T08:23:42.181575 install_asm_1.10: -------------
│ Error: Get "https://35.244.8.53/api?timeout=32s": dial tcp 35.244.8.53:443: connect: connection refused
│ 2021-09-06T08:23:42.398271 install_asm_1.10: [WARNING]: Failed, retrying...(5 of 5)
│ + cleanup
│ + rm -rf /tmp/kubectl_wrapper_24744_28603
│
╵
╷
│ Error: Error creating service account: Post "https://iam.googleapis.com/v1/projects/xxx/serviceAccounts?alt=json&prettyPrint=false": oauth2/google: incomplete token received from metadata
│
│   with module.hub-primary.google_service_account.gke_hub_sa[0],
│   on .terraform/modules/hub-primary/modules/hub/main.tf line 33, in resource "google_service_account" "gke_hub_sa":
│   33: resource "google_service_account" "gke_hub_sa" {

假设在集群完全完成之前端点还没有准备好，尝试depend_on为模块添加标志hub-primary，asm这会导致以下错误：

depend_on = [module.gke_node_pool11, module.gke_node_pool_preemptible22]

Error: Invalid count argument
│
│   on .terraform/modules/asm.asm_install/main.tf line 57, in resource "random_id" "cache":
│   57:   count = (! local.skip_download) ? 1 : 0
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use
│ the -target argument to first apply only the resources that the count depends on.

我如何让asmandhub-primary模块在集群创建时等待，因为如果我先拆分集群创建然后运行模块，则 terraform 脚本可以正常hub-primary工作asm？

有什么我错过的吗？

谢谢 ！

标签： google-cloud-platformterraformterraform-provider-gcpgoogle-anthos