windows - 将我的服务部署到 azure kubernetes 服务时,总是出现错误“部署 'xxx' 超出其进度期限”
问题描述
我是 AKS 的新手,我正在使用 AKS 设置 CI/CD。我创建了一个带有两个节点池的 azure kubernetes 服务,一个是默认的(nodepool1,带有 linux OS),另一个是我要部署到的 windows os 节点池(wdpool)。
我的 kubernetes 节点池: 我的 azure kubernetes 节点池快照
我看到我的存储库映像已成功推送: 'xap' 存储库已成功推送快照
kubeevents 错误如下,我应该如何解决?
Error: failed to start container "xap": Error response from daemon: hcsshim::CreateComputeSystem xap: The container operating system does not match the host operating system.
"
- 我的部署.yml
apiVersion : apps/v1beta1
kind: Deployment
metadata:
name: xap
namespace: kube-public
spec:
progressDeadlineSeconds: 900
replicas: 1
template:
metadata:
labels:
app: xap
spec:
nodeSelector:
agentpool: wdpool
beta.kubernetes.io/os: windows
containers:
- name: xap
image: xapcontainerregistry.azurecr.io/xap
ports:
- containerPort: 80
- 我的服务.yml
apiVersion: v1
kind: Service
metadata:
name: xap
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: xap
- 我在构建 yml 中的部署任务:
# Deploy to Kubernetes - Review app with Azure DevSpaces
# Build and push image to Azure Container Registry; Deploy to Azure Kuberentes Services
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- users/jasca/xapaks
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: '49c3dcce-5bc7-4ea4-90fe-4bb701a497e8'
imageRepository: 'xap'
containerRegistry: 'xapcontainerregistry.azurecr.io'
dockerfilePath: '**/Dockerfile'
tag: '$(Build.BuildId)'
imagePullSecret: 'xapcontainerregistry4671248d-auth'
# Agent VM image name
vmImageName: 'windows-latest'
# Azure Service connection
azureConnection: '4e998ed5-bbcb-4495-b19e-1ad6ca32bbe1'
# Name of the new namespace being created to deploy the PR changes.
k8sNamespaceForPR: '$(system.pullRequest.sourceBranch)'
buildConfiguration: 'Debug'
stages:
- stage: Build
displayName: Build WebContainerHost
jobs:
- job: Build
displayName: Build
pool:
# vmImage: $(vmImageName)
name: XAP_AKS_Pool
# name: XapOuterloopProductionOnly
# demands:
# - Agent.ComputerName -equals xapagent4
steps:
# ignore build tasks, succeed.
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
jobs:
- deployment: Deploy
condition: and(succeeded(), not(startsWith(variables['Build.SourceBranch'], 'refs/pull/')))
displayName: Deploy
pool:
vmImage: $(vmImageName)
environment: 'xapplatform.kube-public'
strategy:
runOnce:
deploy:
steps:
- task: KubernetesManifest@0
displayName: Create imagePullSecret
inputs:
action: createSecret
secretName: $(imagePullSecret)
dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
- task: KubernetesManifest@0
displayName: Deploy to Kubernetes cluster
inputs:
rolloutStatusTimeout: 1800
action: deploy
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yml
$(Pipeline.Workspace)/manifests/service.yml
imagePullSecrets: |
$(imagePullSecret)
containers: |
$(containerRegistry)/$(imageRepository):$(tag)
timeoutInMinutes: 40
- deployment: DeployPullRequest
displayName: Deploy Pull request
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/pull/'))
pool:
vmImage: $(vmImageName)
environment: 'xapplatform.$(k8sNamespaceForPR)'
strategy:
runOnce:
deploy:
steps:
- reviewApp: kube-public
- task: Kubernetes@1
displayName: 'Create a new namespace for the pull request'
inputs:
command: apply
useConfigurationFile: true
inline: '{ "kind": "Namespace", "apiVersion": "v1", "metadata": { "name": "$(k8sNamespaceForPR)" }}'
- task: KubernetesManifest@0
displayName: Create imagePullSecret
inputs:
action: createSecret
secretName: $(imagePullSecret)
namespace: $(k8sNamespaceForPR)
dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
- task: KubernetesManifest@0
displayName: Deploy to the new namespace in the Kubernetes cluster
inputs:
action: deploy
namespace: $(k8sNamespaceForPR)
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yml
$(Pipeline.Workspace)/manifests/service.yml
imagePullSecrets: |
$(imagePullSecret)
containers: |
$(containerRegistry)/$(imageRepository):$(tag)
- task: Kubernetes@1
name: get
displayName: 'Get services in the new namespace'
continueOnError: true
inputs:
command: get
namespace: $(k8sNamespaceForPR)
arguments: svc
outputFormat: jsonpath='http://{.items[0].status.loadBalancer.ingress[0].ip}:{.items[0].spec.ports[0].port}'
# Getting the IP of the deployed service and writing it to a variable for posing comment
- script: |
url="$(get.KubectlOutput)"
message="Your review app has been deployed"
if [ ! -z "$url" -a "$url" != "http://:" ]
then
message="${message} and is available at $url.<br><br>[Learn More](https://aka.ms/testwithreviewapps) about how to test and provide feedback for the app."
fi
echo "##vso[task.setvariable variable=GITHUB_COMMENT]$message"
- 我的 createPullSecret 任务成功:
##[section]Starting: Create imagePullSecret
==============================================================================
Task : Deploy to Kubernetes
Description : Use Kubernetes manifest files to deploy to clusters or even bake the manifest files to be used for deployments using Helm charts
Version : 0.165.4
Author : Microsoft Corporation
Help : https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes-manifest
==============================================================================
==============================================================================
Kubectl Client Version: v1.17.1
Kubectl Server Version: v1.14.8
==============================================================================
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe delete secret xapcontainerregistry4671248d-auth --namespace kube-public
secret "xapcontainerregistry4671248d-auth" deleted
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe create secret docker-registry xapcontainerregistry4671248d-auth --docker-username *** --docker-password *** --docker-server *** --docker-email ServicePrincipal@AzureRM --namespace kube-public
secret/xapcontainerregistry4671248d-auth created
##[section]Finishing: Create imagePullSecret
- 我部署到 Kubernetes 集群的任务日志:
##[section]Starting: Deploy to Kubernetes cluster
==============================================================================
Task : Deploy to Kubernetes
Description : Use Kubernetes manifest files to deploy to clusters or even bake the manifest files to be used for deployments using Helm charts
Version : 0.165.4
Author : Microsoft Corporation
Help : https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes-manifest
==============================================================================
==============================================================================
Kubectl Client Version: v1.17.1
Kubectl Server Version: v1.14.8
==============================================================================
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe apply -f d:\a\_temp\Deployment_xap_1583635691677,d:\a\_temp\Service_xap_1583635691677 --namespace kube-public
deployment.apps/xap configured
service/xap unchanged
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe rollout status Deployment/xap --timeout 1800s --namespace kube-public
error: deployment "xap" exceeded its progress deadline
Waiting for deployment "xap" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment spec update to be observed...
Waiting for deployment spec update to be observed...
Waiting for deployment "xap" rollout to finish: 1 old replicas are pending termination...
##[error]Error: error: deployment "xap" exceeded its progress deadline
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe describe Deployment xap --namespace kube-public
Name: xap
Namespace: kube-public
CreationTimestamp: Fri, 06 Mar 2020 14:16:37 +0000
Labels: app=xap
Annotations: deployment.kubernetes.io/revision: 15
kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"apps/v1beta1","kind":"Deployment","metadata":{"annotations":{},"name":"xap","namespace":"kube-public"},"spec":{"progressDea...
Selector: app=xap
Replicas: 1 desired | 1 updated | 2 total | 0 available | 2 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=xap
Containers:
xap:
Image: ***/xap
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available False MinimumReplicasUnavailable
Progressing False ProgressDeadlineExceeded
OldReplicaSets: xap-5b95494dd7 (1/1 replicas created)
NewReplicaSet: xap-79cc65ddf9 (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 36m deployment-controller Scaled down replica set xap-668784656f to 0
Normal ScalingReplicaSet 36m deployment-controller Scaled up replica set xap-694857dcd8 to 1
Normal ScalingReplicaSet 36m deployment-controller Scaled down replica set xap-67fd46c7b5 to 0
Normal ScalingReplicaSet 36m deployment-controller Scaled up replica set xap-79cc65ddf9 to 1
Normal ScalingReplicaSet 10m deployment-controller Scaled down replica set xap-694857dcd8 to 0
Normal ScalingReplicaSet 10m deployment-controller Scaled up replica set xap-5b95494dd7 to 1
For more information, go to https://msasg.visualstudio.com/Bing_and_IPG/_environments/75/providers/kubernetes/12
[command]C:\ProgramData\Chocolatey\bin\kubectl.exe get service/xap -o json --namespace kube-public
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"name\":\"xap\",\"namespace\":\"kube-public\"},\"spec\":{\"ports\":[{\"port\":80}],\"selector\":{\"app\":\"xap\"},\"type\":\"LoadBalancer\"}}\n"
},
"creationTimestamp": "2020-03-06T14:16:38Z",
"name": "xap",
"namespace": "kube-public",
"resourceVersion": "1743607",
"selfLink": "/api/v1/namespaces/kube-public/services/xap",
"uid": "17d862ed-5fb5-11ea-ba44-82c49be4b313"
},
"spec": {
"clusterIP": "10.0.76.15",
"externalTrafficPolicy": "Cluster",
"ports": [
{
"nodePort": 30742,
"port": 80,
"protocol": "TCP",
"targetPort": 80
}
],
"selector": {
"app": "xap"
},
"sessionAffinity": "None",
"type": "LoadBalancer"
},
"status": {
"loadBalancer": {
"ingress": [
{
"ip": "52.139.245.51"
}
]
}
}
}
service xap external IP is 52.139.245.51
##[error]Rollout status check failed.
##[section]Finishing: Deploy to Kubernetes cluster
解决方案
根据@Arghya-Sadhu 的建议,我检查了 Azure 日志中的日志,即“KubeEvent”表,它说
failed to start container "xap": Error response from daemon: hcsshim::CreateComputeSystem xap: The container operating system does not match the host operating system.
上述错误是由于我的 docker building agent 使用的是 Windows Server 2016,而 azure 容器系统是 Windows Server 2019,不匹配...我将我的 building agent 机器操作系统升级到 Windows Server 2019,问题已解决并且成功部署到 Kubernetes。
推荐阅读
- magento - 为什么要在 Magento2 的每个插件拦截器中使用 $subject?
- bash - 在 httpd.conf 中更新 ServerLimit 的 Bash 方式
- c# - FileStream 损坏的 zip - SignalR Stream
- gatsby - Gatsby TypeError:createNodeId 不是函数
- java - 在 Java/Python 中识别安全/认证的 pdf 文件
- jquery - 如何在网页中将普通的 HTML Select 元素与 Material Select 并排使用?
- python - 尝试使用键绑定时出现 Tkinter/Canvas 类型错误
- python - 浮士德:TypeError:produce()得到了一个意外的关键字参数'timestamp'
- html - 单击时更改按钮的值,但文件未下载
- flutter - Flutter:将对象从流转换为另一个对象并在未来的构建器中使用它?