terraform - Terraform：使用流量管理器端点部署 Kubernetes 服务

问题描述

我需要使用 terraform 在流量管理器中添加 Kubernetes 服务的解决方案，为此我需要为每个集群提供一个公共 IP 地址，但似乎 IP 是在部署后在不同的订阅下创建的。

尝试使用 azurerm_traffic_manager_endpoint 关于不同类型（如 azureEndpoints 和 nestedEndpoints），但似乎脚本失败并出现下面列出的相同错误。

下面是我要部署的脚本，我将分享错误：

错误：

创建/更新nestedEndpoints端点“vmap-tmep”（流量管理器配置文件“vmap-tm”/资源组“RG-TEST-TEST”）：trafficmanager.EndpointsClient#CreateOrUpdate：响应请求失败：StatusCode = 400 - 原始错误： autorest/azure：服务返回错误。Status=400 Code="BadRequest" Message="端点 'vmap-tmep' 的 'resourceTargetId' 属性无效或丢失。该属性只能为以下端点类型指定：AzureEndpoints、NestedEndpoints。您必须具有读取权限它所指的资源。”


# Traffic Manager Profile Resource
resource "azurerm_traffic_manager_profile" "tmp" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tm")
  resource_group_name = azurerm_resource_group.rg.name
  
  traffic_routing_method = "Weighted"

  dns_config {
    relative_name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tm-dns-test")
    ttl           = 100
  }

  monitor_config {
    protocol                     = "http"
    port                         = 80
    path                         = "/"
    interval_in_seconds          = 30
    timeout_in_seconds           = 9
    tolerated_number_of_failures = 3
  }
}

# Traffic Manager Endpoint Resource
resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-tmep")
  resource_group_name = azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "nestedEndpoints"
  weight              = 1000
  target_resource_id = azurerm_kubernetes_cluster.k8s1.id
}



################ K8S nodes pool location 1 ################

resource "azurerm_kubernetes_cluster" "k8s1" {
  name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-k8s")
  location            = var.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = "exampleaks1"

  service_principal {
    client_id = "bsdfsdfs3b"
    client_secret = "353sdfsdfsdfsdfsd9"
  }
    role_based_access_control {
      azure_active_directory {
        managed           = true
        admin_group_object_ids = [var.group_object_id]
        tenant_id         = var.tenant_id
        azure_rbac_enabled = true
      }
        enabled = true
    
    }   

  linux_profile {
    admin_username = var.adminusername

    ssh_key {
      key_data = "${file("${var.ssh_public_key}")}"
    }
  }
auto_scaler_profile {
           new_pod_scale_up_delay           = "5s" 
           scale_down_delay_after_delete    = "10s" 
           skip_nodes_with_local_storage    = false
        }
addon_profile {
            azure_policy{
        enabled                    = true
      }
}


 default_node_pool {
          enable_auto_scaling          = true
          max_count                    = 5
          max_pods                     = 30
          min_count                    = 1
          name                         = "default"
          only_critical_addons_enabled = false
          #orchestrator_version         = "1.20.7"
          vm_size                                = "Standard_D2_v2"
          os_disk_size_gb                    = 30
}
}

标签： terraformterraform-provider-azureazure-rm

解决方案

如前所述，您需要更改代码中的一些内容才能使用 AKS 的流量管理器。

您需要使用azureEndpoints而不是nestedEndpoints作为流量管理器端点类型。

因为目前有四种服务(Cloud Service ,App Service, App Service Slots and Public IP's)支持流量管理器。因此，您必须使用 AKS 正在使用的公共 IP。

您必须使用以下块：

resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = "ansumanaks-tmep"
  resource_group_name = data.azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "azureEndpoints"
  endpoint_status = "enabled"
  target_resource_id = (tolist(azurerm_kubernetes_cluster.k8s1.network_profile.0.load_balancer_profile.0.effective_outbound_ips)[0])
}

为了进行测试，我使用了以下 terraform 代码：

provider "azurerm" {
  features {}
}

data "azurerm_resource_group" "rg"{
    name="ansumantest"
}

# Traffic Manager Profile Resource
resource "azurerm_traffic_manager_profile" "tmp" {
  name = "ansumanaks-tm"
  resource_group_name = data.azurerm_resource_group.rg.name
  
  traffic_routing_method = "Priority"

  dns_config {
    relative_name = "ansumanaks-tm-dns-test"
    ttl           = 100
  }

  monitor_config {
    protocol                     = "http"
    port                         = 80
    path                         = "/"
    interval_in_seconds          = 30
    timeout_in_seconds           = 9
    tolerated_number_of_failures = 3
  }
}
resource "azurerm_public_ip" "example" {
  name                = "akspublicIP"
  resource_group_name = data.azurerm_resource_group.rg.name
  location            = data.azurerm_resource_group.rg.location
  sku = "Standard"
  allocation_method   = "Static"
  domain_name_label = "akstestregion"
}

# Traffic Manager Endpoint Resource
resource "azurerm_traffic_manager_endpoint" "tmep" {
  name = "ansumanaks-tmep"
  resource_group_name = data.azurerm_resource_group.rg.name
  profile_name        = azurerm_traffic_manager_profile.tmp.name
  type                = "azureEndpoints"
  endpoint_status = "enabled"
  target_resource_id = (tolist(azurerm_kubernetes_cluster.k8s1.network_profile.0.load_balancer_profile.0.effective_outbound_ips)[0])
}




################ K8S nodes pool location 1 ################

resource "azurerm_kubernetes_cluster" "k8s1" {
  name = "ansumanaks-k8s"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name = data.azurerm_resource_group.rg.name
  dns_prefix          = "exampleaks1"

  service_principal {
    client_id = "1dd6833b-xxxx-xxxx-xxxx-112c3fb4fb79"
    client_secret = "e997Q~ky5ZWHIxxxxxxxxxxxxxxxxxxxxxxx"
  }
    role_based_access_control {
      azure_active_directory {
        managed           = true
        tenant_id         = "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
        azure_rbac_enabled = true
      }
        enabled = true
    
    }   
    network_profile {
        network_plugin = "kubenet"
      load_balancer_profile {
        outbound_ip_address_ids= [azurerm_public_ip.example.id]
      }
    }

  linux_profile {
    admin_username = "ansuman"

    ssh_key {
      key_data = "${file("C:/Users/ansbal/public.pub")}"
    }
  }
auto_scaler_profile {
           new_pod_scale_up_delay           = "5s" 
           scale_down_delay_after_delete    = "10s" 
           skip_nodes_with_local_storage    = false
        }
addon_profile {
            azure_policy{
        enabled                    = true
      }
}


 default_node_pool {
          enable_auto_scaling          = true
          max_count                    = 5
          max_pods                     = 30
          min_count                    = 1
          name                         = "default"
          only_critical_addons_enabled = false
          #orchestrator_version        = "1.20.7"
          vm_size                      = "Standard_D2_v2"
          os_disk_size_gb              = 30
}
}

输出：

笔记：

我还创建了一个公共 IP，它将用于 AKS 负载均衡器，因为当我使用创建 AKS 时创建的默认公共 IP 进行测试时，它给了我如下错误，以解决我创建公共 IP 的问题。

admin_group_object_ids = [var.group_object_id]由于缺少权限，我已删除。您可以根据自己的要求使用它们。

terraform - Terraform：使用流量管理器端点部署 Kubernetes 服务

问题描述

解决方案

推荐阅读