首页 > 解决方案 > 尽管有重启策略,Docker restartmanger 仍会阻止重启

问题描述

我有一个 docker 容器,尽管设置了restart=unless-stopped策略,但它喜欢在不重新启动的情况下关闭。

其他容器在我没有任何问题的同一主机上运行(具有相似的启动配置参数)。主机是一个有点不稳定的网络上的集群中的一个节点,并且容器是节点网络的频繁用户(与主节点交谈)所以我并不惊讶它会定期失败,但我希望它能够自行重启.

这是由于重新启动管理器。显示docker inspect State.error的消息显然来自 docker 而不是我的容器。日志显示:

... time="2019-09-21T02:06:31.969473802Z" level=error msg="restartmanger wait error: Could not attach to network cqr3v2jode1boqh2yofqrh7bx: context deadline exceeded"

因此,似乎 - 偶尔 - 当容器重新启动时,网络已关闭并且管理器决定停止重新启动。问题变成了如何覆盖这种行为。

docker info

Client:
 Debug Mode: false

Server:
 Containers: 4
  Running: 2
  Paused: 0
  Stopped: 2
 Images: 43
 Server Version: 19.03.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: wgn64s7lx9jvgw36gtlu0dsou
  Is Manager: false
  Node Address: 10.0.0.2
  Manager Addresses:
   10.0.0.1:2377
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.66-v7+
 Operating System: Raspbian GNU/Linux 9 (stretch)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 874.5MiB
 Name: sensors-2
 ID: NTRC:WPLS:GH2P:ZTLM:EDAN:H7HB:HGP6:6G6A:3YVW:T2I7:TVJU:XV3N
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

docker inspect以下是非重启容器上的相关位。请注意,它已经重新启动了几次,由于网络错误而退出,MaximumRetryCount 为 0(我假设它是无限的)。最近它没有持续很长时间......但我的理解unless-stopped是docker会继续重新启动容器,尽管它会增加重新启动之间的延迟。

[
    {
        "Id": "fa7c59dfa38f25c70d4c1293db27965c2e76af950fa19a2097b4ce63e1af2be4",
        "Created": "2019-06-24T05:25:10.792698029Z",
        "Path": "/srv/bin/weather_collector_server",
        "Args": [
            "/etc/config.ini"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 1,
            "Error": "Could not attach to network cqr3v2jode1boqh2yofqrh7bx: context deadline exceeded",
            "StartedAt": "2019-09-21T03:56:40.911764904Z",
            "FinishedAt": "2019-09-21T03:58:07.234852939Z"
        },
        "Image": "sha256:ee0e5023f37917f074dd0bf03dca328833eafd117fe69041203533768a196789",
        "ResolvConfPath": "/var/lib/docker/containers/fa7c59dfa38f25c70d4c1293db27965c2e76af950fa19a2097b4ce63e1af2be4/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/fa7c59dfa38f25c70d4c1293db27965c2e76af950fa19a2097b4ce63e1af2be4/hostname",
        "HostsPath": "/var/lib/docker/containers/fa7c59dfa38f25c70d4c1293db27965c2e76af950fa19a2097b4ce63e1af2be4/hosts",
        "LogPath": "",
        "Name": "/weather_collector_server",
        "RestartCount": 3,
        "Driver": "overlay2",
        "Platform": "linux",
        ...
        "HostConfig": {
           ...
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            ...
        ],

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "0e901219511bb618d66943a12af1e09d8bbcb78ca4caa0bad88880f21d843c55",
            ...
            "Networks": {
                "hostnet": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "fa7c59dfa38f"
                    ],
                    "NetworkID": "cqr3v2jode1boqh2yofqrh7bx",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": {}
                }
            }
        }
    }
]

标签: docker

解决方案


推荐阅读