kubernetes - Difference between NFS-PV, hostPath-PV on NFS and hostPath mount in deployment

问题描述

I have a Kubernetes cluster setup (on-premise), that has an NFS share (my-nfs.internal.tld) mounted to /exports/backup on each node to create backups there.

Now I'm setting up my logging stack and I wanted to make the data persistent. So I figured I could start by storing the indices on the NFS.

Now I found three different ways to achieve this:

NFS-PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: logging-data
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  nfs:
    server: my-nfs.internal.tld
    path: /path/to/exports/backup/logging-data/

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: logging-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: logging-data
  resources:
    requests:
      storage: 10Gi

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      ...
      volumes:
        - name: logging-data-volume
          persistentVolumeClaim:
            claimName: logging-data-pvc

This would, of course, require, that my cluster gets access to the NFS (instead of only the nodes as it is currently setup).

hostPath-PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: logging-data
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /exports/backup/logging-data/

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: logging-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: logging-data
  resources:
    requests:
      storage: 10Gi

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      ...
      volumes:
        - name: logging-data-volume
          persistentVolumeClaim:
            claimName: logging-data-pvc

hostPath mount in deployment

As the nfs is mounted to all my nodes, I could also just use the host path directly in the deployment without pinning anything.

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      ...
      volumes:
        - name: logging-data-volume
          hostPath:
            path: /exports/backup/logging-data
            type: DirectoryOrCreate

So my question is: Is there really any difference between these three? I'm pretty sure all three work. I tested the second and third already. I was not yet able to test the first though (in this specific setup at least). Especially the second and third solutions seem very similar to me. The second makes it easier to re-use deployment files on multiple clusters, I think, as you can use persistent volumes of different types without changing the volumes part of the deployment. But is there any difference beyond that? Performance maybe? Or is one of them deprecated and will be removed soon?

I found a tutorial mentioning, that the hostPath-PV only works on single-node clusters. But I'm sure it does also works in my case here. Maybe the comment was about: "On multi-node clusters the data changes when deployed to different nodes."

From reading to a lot of documentation and How-To's I understand, that the first one is the preferred solution. I would probably also go for it as it is the one easiest replicated to a cloud setup. But I do not really understand why this is preferred to the other two.

Thanks in advance for your input on the matter!

标签： kubernetesnfspersistent-volumes

解决方案

NFS确实是首选解决方案：

nfs卷允许将现有的 NFS（网络文件系统）共享挂载到 Pod 中。与emptyDir删除 Pod 时删除的不同，nfs卷的内容被保留并且卷只是被卸载。这意味着 NFS 卷可以预先填充数据，并且可以在 pod 之间共享数据。NFS 可以由多个写入器同时挂载。

因此，NFS 之所以有用有两个原因：

数据是持久的。
它可以同时从多个 Pod 访问，并且可以在 Pod 之间共享数据。

有关详细信息，请参阅 NFS示例。

而hostPath：

卷hostPath将文件或目录从主机节点的文件系统安装到您的 Pod 中。

由于节点上的文件不同，具有相同配置（例如从 PodTemplate 创建）的 Pod 在不同节点上的行为可能不同

在底层主机上创建的文件或目录只能由 root 写入。您要么需要在特权容器中以 root 身份运行进程，要么修改主机上的文件权限以便能够写入hostPath卷

hostPath不推荐，原因如下：

您不直接控制您的 pod 将在哪个节点上运行，因此您不能保证 pod 将实际调度在具有数据量的节点上。
您将集群暴露在安全威胁之下。
如果一个节点出现故障，您需要将该 pod 安排在其他节点上，您的本地配置卷将不可用。