时间戳

首页 > 解决方案 > Kubernetes Deployment 动态端口转发

docker - Kubernetes Deployment 动态端口转发

问题描述

我正在将 Docker 映像从 Docker 移动到 K8s 部署。我有自动缩放规则，所以它从 5 开始，但可以到 12。K8s 上的 Docker 映像完美地开始，前面有一个 k8s 服务来集群部署。

现在每个容器都有自己的 JVM，它有一个 Prometheus 应用程序来检索其统计信息。在 Docker 中，这没有问题，因为提供 Prometheus 信息的端口是动态创建的，起始端口为 8000，因此 docker-compose.yml 会根据启动的镜像数量将端口增加 1。

问题是我在 K8s [deployment].yml 文件中找不到如何执行此操作。因为部署 pod 是动态的，我原以为会有一些方法来设置起始 HOST 端口，以根据启动的容器数量增加。

也许我以错误的方式看待这个问题，所以任何澄清都会有所帮助，同时会继续在谷歌上搜索有关此类事情的任何信息。

标签： dockerkuberneteskubernetes-pod

解决方案

好吧，在阅读了这么多之后，我得出的结论是，K8s 不负责为 Docker Image 打开端口或在某个奇怪的端口上为您的应用程序提供入口，这不是它的责任。K8s Deployment 只是部署您请求的 Pod。您可以在 DEPLOYMENT -> SPEC -> CONTAINERS -> PORTS 上设置 Ports 选项，这就像 Docker 一样仅提供信息。但这允许您使用可用的 Prometheus 端口对所有 PODS（容器）进行 JSONPath 查询。这将允许您在 Prometheus.yaml 文件中重建“目标”值。现在有了这些目标，Grafana 就可以使用它们来创建仪表板。

就是这样，很简单。我把事情复杂化了，因为我不明白。我包含一个我快速编写的脚本，以便您自行承担风险使用某些东西。

顺便说一句，我交替使用 Pod 和 Container。

  #!/usr/bin/env bash
  #set -x

  _MyappPrometheusPort=8055
  _finalIpsPortArray=()
  _prometheusyamlFile=prometheus.yml

  cd /docker/images/prometheus


  #######################################################################################################################################################
  #One container on the K8s System is weave and it holds the subnet we need to validate against.
  #weave-net-lwzrk                            2/2     Running   8 (7d3h ago)   9d    192.168.2.16   accl-ffm-srv-006   <none>           <none>
  _weavenet=$(kubectl get pod -n kube-system -o wide | grep weave | cut -d ' ' -f1 )
  echo "_weavenet: $_weavenet"

  #The default subnet is the one that lets us know the conntainer is part of kubernetes network.
  #          Range: 10.32.0.0/12
  #  DefaultSubnet: 10.32.0.0/12
  _subnet=$( kubectl exec -n kube-system $_weavenet -c weave -- /home/weave/weave --local status | sed -En "s/^(.*)(DefaultSubnet:\s)(.*)?/\3/p" )
  echo "_subnet: $_subnet"
  _cidr2=$( echo "$_subnet" | cut -d '/' -f2 )
  echo "_cidr2: /$_cidr2"



  #######################################################################################################################################################
  #This is an array of the currently monitored containers that prometheus was sstarted with.
  #We will remove any containers form the array that fit the K8s Weavenet subnet with the myapp prometheus port.
  _targetLineFound_array=($( egrep '^\s{1,20}-\s{0,5}targets\s{0,5}:\s{0,5}\[.*\]' $_prometheusyamlFile | sed -En "s/(.*-\stargets:\s\[)(.*)(\]).*/\2/p"  | tr "," "\n"))
  for index in "${_targetLineFound_array[@]}"
  do
      _ip="${index//\'/$''}"
      _ipTocheck=$( echo $_ip | cut -d ':' -f1 )
      _portTocheck=$( echo $_ip | cut -d ':' -f2 )
      #We need to check if the IP is within the subnet mask attained from K8s.
      #The port must also be the prometheus port in case some other port is used also for Prometheus.
      #This means the IP should be removed since we will put the list of IPs from
      #K8s currently in production by Deployment/AutoScale rules.
      #Network:   10.32.0.0/12
      _isIpWithinSubnet=$( ipcalc $_ipTocheck/$_cidr2 | sed -En "s/^(.*)(Network:\s+)([0-9]{1}[0-9]?[0-9]?\.[0-9]{1}[0-9]?[0-9]?\.[0-9]{1}[0-9]?[0-9]?\.[0-9]{1}[0-9]?[0-9]?)(\/[0-9]{1}[0-9]{1}.*)?/\3/p" )
      if [[ "$_isIpWithinSubnet/$_cidr2" == "$_subnet" && "$_portTocheck" == "$_MyappPrometheusPort" ]]; then
          echo "IP managed by K8s will be deleted: _isIpWithinSubnet: ($_ip) $_isIpWithinSubnet"
      else
          _finalIpsPortArray+=("$_ip")
      fi
  done



  #######################################################################################################################################################
  #This is an array of the current running myapp App containers with a prometheus port that is available.
  #From this list we will add them to the prometheus file to be available for Grafana monitoring.
  readarray -t _currentK8sIpsArr < <( kubectl get pods --all-namespaces --chunk-size=0 -o json | jq '.items[] | select(.spec.containers[].ports != null) | select(.spec.containers[].ports[].containerPort == '$_MyappPrometheusPort' ) | .status.podIP' )
  for index in "${!_currentK8sIpsArr[@]}"
  do
      _addIPToMonitoring=${_currentK8sIpsArr[index]//\"/$''}
      echo "IP Managed by K8s as myapp app with prometheus currently running will be added to monitoring: $_addIPToMonitoring"
      _finalIpsPortArray+=("$_addIPToMonitoring:$_MyappPrometheusPort")
  done



  ######################################################################################################################################################
  #we need to recreate this string and sed it into the file
  #- targets: ['192.168.2.13:3201', '192.168.2.13:3202', '10.32.0.7:8055', '10.32.0.8:8055']

  _finalPrometheusTargetString="- targets: ["
  i=0

  # Iterate the loop to read and print each array element
  for index in "${!_finalIpsPortArray[@]}"
  do
      ((i=i+1))
      _finalPrometheusTargetString="$_finalPrometheusTargetString '${_finalIpsPortArray[index]}'"
      if [[ $i != ${#_finalIpsPortArray[@]}  ]]; then
           _finalPrometheusTargetString="$_finalPrometheusTargetString,"
      fi
  done

  _finalPrometheusTargetString="$_finalPrometheusTargetString]"

  echo "$_finalPrometheusTargetString"

  sed -i -E "s/(.*)-\stargets:\s\[.*\]/\1$_finalPrometheusTargetString/" ./$_prometheusyamlFile

  docker-compose down
  sleep 4
  docker-compose up -d

  echo "All changes were made. Exiting"

  exit 0

推荐阅读