时间戳

首页 > 解决方案 > docker compose up 无法挂载卷

docker - docker compose up 无法挂载卷

问题描述

运行时出现这些错误docker-compose up

错误：对于airflow_webserver_1 无法启动服务webserver：OCI 运行时创建失败：container_linux.go:346：启动容器进程导致“process_linux.go:449：容器初始化导致\”rootfs_linux.go:58：安装\\“/srv/airflow /dags\\" to rootfs \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged\\" at \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged/usr/local/airflow/dags\ \" 导致 \\"not a directory\\"\""：未知：您是否尝试将目录挂载到文件上（反之亦然）？检查指定的主机路径是否存在并且是预期的类型

错误：对于网络服务器无法启动服务网络服务器：OCI 运行时创建失败：container_linux.go:346：启动容器进程导致“process_linux.go:449：容器初始化导致\”rootfs_linux.go:58：安装\\“/srv/airflow /dags\\" 到 rootfs \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a

我看过类似的帖子说我应该检查挂载的目录是否应该存在并且是目录（而不是文件）等等......

我在我的主机上检查了路径/src/airflow/dags存在并包含几个文件（我以前容器运行的 dags 的代码）。

我怎样才能摆脱这个错误，为什么会发生这种情况？我不介意从头开始再次启动容器，丢失以前的卷内容。

这是 docker-compose 内容：

版本：'2.1' 服务：postgres：图像：postgres:9.6 重启：除非停止环境：- POSTGRES_USER=airflow - POSTGRES_PASSWORD=airflow - POSTGRES_DB=airflow

webserver:
    build: airflow-server
    restart: unless-stopped
    depends_on:
        - postgres
    environment:
        - LOAD_EX=n
        - EXECUTOR=Local
        - VIRTUAL_HOST=airflow.agatha.com
        - VIRTUAL_PORT=8080
    volumes:
        - /srv/airflow/dags:/usr/local/airflow/dags
        # Uncomment to include custom plugins
        # - /srv/airflow/plugins:/usr/local/airflow/plugins
    ports:
        - "8075:8080"
    command: webserver
    healthcheck:
        test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
        interval: 30s
        timeout: 30s
        retries: 3
    logging:
        driver: json-file
        options:
            max-size: 50m

网络：默认：外部：名称：nginx-proxy

（该airflow-server文件夹包含一个自定义 Dockerfile 以复制一些配置文件）

这是 Dockerfile

FROM puckel/docker-airflow
USER root

COPY proxy.conf /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update && apt-get install -y gnupg apt-transport-https procps

RUN mkdir /usr/local/airflow/ssh
COPY id_rsa_airflow_user /usr/local/airflow/ssh/id_rsa_airflow_user
RUN mkdir /usr/local/airflow/sparkScripts
COPY FileUnion.py /usr/local/airflow/sparkScripts
COPY ETLDag.py /usr/local/airflow/dags

USER airflow

完整的命令行输出：

> :~/docker/airflow$ sudo docker-compose up WARNING: The Docker Engine
> you're using is running in swarm mode.
> 
> Compose does not use swarm mode to deploy services to multiple nodes
> in a swarm. All containers will be scheduled on the current node.
> 
> To deploy your application across the swarm, use `docker stack
> deploy`.
> 
> Starting airflow_postgres_1 ... done Recreating airflow_webserver_1
> ... error
> 
> ERROR: for airflow_webserver_1  Cannot start service webserver: OCI
> runtime create failed: container_linux.go:348: starting container
> process caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\\"/srv/airflow/dags\\\" to rootfs
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\\"
> at
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\\"
> caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type
> 
> ERROR: for webserver  Cannot start service webserver: OCI runtime
> create failed: container_linux.go:348: starting container process
> caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\\"/srv/airflow/dags\\\" to rootfs
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\\"
> at
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\\"
> caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type ERROR: Encountered errors while
> bringing up the project.

以及airflow-server目录内容（有些文件没有被使用）

~/docker/airflow$ ls -l airflow-server/
total 326788
-rw-rw-r-- 1 user user 5008 mar 25 12:16 Dockerfile
-rw-rw-r-- 1 user user 7302 mar 24 15:37 ETLDagCreator.py
-rw-rw-r-- 1 user user 5900 mar 24 16:16 ETLDag.py
-rw-rw-r-- 1 user user 2564 mar 24 15:19 FileUnion.py
-rw-r--r-- 1 user user 334559382 mar 18 11:08 hadoop-3.1.1.tar.gz
drwxrwxr-x 2 user user 4096 mar 19 10:42 hadoop-client-conf
-rwxrwxr-x 1 user user 2026 mar 18 10:40 hbase.service.keytab
-rwxrwxr-x 1 user user 132 mar 18 10:39 hdfs.headless.keytab
-rwxrwxr-x 1 user user 2002 mar 18 10:38 hive.service.keytab
-rw-rw-r-- 1 user user 625 mar 19 16:02 hosts
-rwxrwxr-x 1 user user 1679 mar 20 10:17 id_rsa_airflow_user
-rw-rw-r-- 1 user user 529 mar 18 10:33 krb5.conf
-rw-rw-r-- 1 user user 101 mar 18 10:45 proxy.conf
-rw-rw-r-- 1 user user 58 mar 18 16:47 resolv.conf

谢谢

标签： dockerdocker-composedocker-volume

解决方案

如错误所示，您正在尝试将目录挂载到文件上。这不是 Linux 中的有效绑定挂载，因此 docker 无法创建容器。该文件是在您的图像中创建的：

COPY ETLDag.py /usr/local/airflow/dags

如果您想将其复制到名为的目录中，则dags需要一个斜杠：

COPY ETLDag.py /usr/local/airflow/dags/

您尝试在文件上挂载的目录来自您的卷挂载：

volumes:
    - /srv/airflow/dags:/usr/local/airflow/dags

正如您在评论中显示的那样，它是一个目录：

$ ls -ld /srv/airflow/dags
drwxr-xr-x 2 root root 4096 mar 25 12:45 /srv/airflow/dags

这些都需要是文件或都是目录。你不能混合它们。

推荐阅读