flask - Kubernetes 上没有持久的 Prometheus 指标
问题描述
我正在从 Kubernetes 上托管的 uwsgi 应用程序收集 Prometheus 指标,删除 pod 后指标不会保留。Prometheus 服务器托管在同一个 kubernetes 集群上,我为它分配了一个持久存储。
即使在它们被删除后,我如何保留 Pod 中的指标?
Prometheus 部署 yaml:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: prometheus
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
app: prometheus
spec:
containers:
- name: prometheus
image: prom/prometheus
args:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus/"
- "--storage.tsdb.retention=2200h"
ports:
- containerPort: 9090
volumeMounts:
- name: prometheus-config-volume
mountPath: /etc/prometheus/
- name: prometheus-storage-volume
mountPath: /prometheus/
volumes:
- name: prometheus-config-volume
configMap:
defaultMode: 420
name: prometheus-server-conf
- name: prometheus-storage-volume
persistentVolumeClaim:
claimName: azurefile
---
apiVersion: v1
kind: Service
metadata:
labels:
app: prometheus
name: prometheus
spec:
type: LoadBalancer
loadBalancerIP: ...
ports:
- port: 80
protocol: TCP
targetPort: 9090
selector:
app: prometheus
应用部署 yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-app
spec:
replicas: 2
selector:
matchLabels:
app: api-app
template:
metadata:
labels:
app: api-app
spec:
containers:
- name: nginx
image: nginx
lifecycle:
preStop:
exec:
command: ["/usr/sbin/nginx","-s","quit"]
ports:
- containerPort: 80
protocol: TCP
resources:
limits:
cpu: 50m
memory: 100Mi
requests:
cpu: 10m
memory: 50Mi
volumeMounts:
- name: app-api
mountPath: /var/run/app
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: api-app
image: azurecr.io/app_api_se:opencv
workingDir: /app
command: ["/usr/local/bin/uwsgi"]
args:
- "--die-on-term"
- "--manage-script-name"
- "--mount=/=api:app_dispatch"
- "--socket=/var/run/app/uwsgi.sock"
- "--chmod-socket=777"
- "--pyargv=se"
- "--metrics-dir=/storage"
- "--metrics-dir-restore"
resources:
requests:
cpu: 150m
memory: 1Gi
volumeMounts:
- name: app-api
mountPath: /var/run/app
- name: storage
mountPath: /storage
volumes:
- name: app-api
emptyDir: {}
- name: storage
persistentVolumeClaim:
claimName: app-storage
- name: nginx-conf
configMap:
name: app
tolerations:
- key: "sku"
operator: "Equal"
value: "test"
effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
labels:
app: api-app
name: api-app
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: api-app
解决方案
您的问题在于用于部署 Prometheus 的控制器类型错误。
在这种情况下,部署控制器是错误的选择(它适用于无状态应用程序,不需要在重新调度的 Pod 之间维护任何持久性标识符 - 例如持久性数据)。
如果您需要跨 Pod(重新)调度的数据(Prometheus 抓取的指标)的持久性,您应该切换到StatefulSet kind* 。
*这是 Prometheus 默认使用prometheus-operator部署的方式。
推荐阅读
- winapi - CCM_DPISCALE 通用控制消息
- excel - 带循环的单元格的相对参考
- python - async Python 真的是异步的吗?
- angular - 错误类型错误:“值未定义”角度 5 httpclient
- r - 如何使用 df2(t)/df2(t-1) 的比率轻松填写 df1(at time t) 的缺失值?
- android - 字符串在 Android 中总是返回 null
- android - 键盘弹出时按钮不向上滚动
- scala - 多个scala数据框concat
- pdf - 堆叠 PDF 图像:单页输出
- c# - 我正在尝试使用变量在 C# 中初始化 DateTime