时间戳

首页 > 解决方案 > docker stack/swarm 中的弹性

docker - docker stack/swarm 中的弹性

问题描述

我有两个节点

[ra@speechanalytics-test ~]$ docker node ls
ID                            HOSTNAME                  STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
mlwwmkdlzbv0zlapqe1veq3uq     speechanalytics-preprod   Ready               Active                                  18.09.3
se717p88485s22s715rdir9x2 *   speechanalytics-test      Ready               Active              Leader              18.09.3

我正在尝试在堆栈中运行具有弹性的容器。这是我的docker-compose.yml文件

version: '3.4'
services:
  elastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata:/usr/share/elasticsearch/data
    deploy:
      placement:
        constraints:
          - node.hostname==speechanalytics-preprod

volumes:
  esdata:
    driver: local

从 docker stack 开始后

docker stack deploy preprod -c docker-compose.yml

容器在 20 秒内崩溃

docker service logs preprod_elastic 
...
   | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   | OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0
   | [2019-04-03T16:41:30,044][WARN ][o.e.b.JNANatives         ] [unknown] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
   | [2019-04-03T16:41:30,049][WARN ][o.e.b.JNANatives         ] [unknown] This can result in part of the JVM being swapped out.
   | [2019-04-03T16:41:30,049][WARN ][o.e.b.JNANatives         ] [unknown] Increase RLIMIT_MEMLOCK, soft limit: 16777216, hard limit: 16777216
   | [2019-04-03T16:41:30,050][WARN ][o.e.b.JNANatives         ] [unknown] These can be adjusted by modifying /etc/security/limits.conf, for example:
   | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   |     # allow user 'elasticsearch' mlockall
   | OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0
   |     elasticsearch soft memlock unlimited
   | [2019-04-03T16:41:02,949][WARN ][o.e.b.JNANatives         ] [unknown] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
   |     elasticsearch hard memlock unlimited
   | [2019-04-03T16:41:02,954][WARN ][o.e.b.JNANatives         ] [unknown] This can result in part of the JVM being swapped out.
   | [2019-04-03T16:41:30,050][WARN ][o.e.b.JNANatives         ] [unknown] If you are logged in interactively, you will have to re-login for the new limits to take effect.
   | [2019-04-03T16:41:02,954][WARN ][o.e.b.JNANatives         ] [unknown] Increase RLIMIT_MEMLOCK, soft limit: 16777216, hard limit: 16777216
preprod

在我拥有的两个节点上

ra@speechanalytics-preprod:~$ sysctl vm.max_map_count
vm.max_map_count = 262144

任何想法如何解决？

标签： dockerelasticsearchdocker-swarm

解决方案

您在 Elasticsearch 中看到的 memlock 错误不是使用 Docker 独有的常见问题，而是在 Elasticsearch 被告知锁定其内存但无法这样做时发生。您可以通过从文件中删除以下环境变量来规避错误docker-compose.yml：

- bootstrap.memory_lock=true

Memlock可以与 Docker Swarm 模式一起使用，但有一些注意事项。

并非所有适用于docker-compose(Docker Compose) 的选项都适用于docker stack deploy(Docker Swarm Mode)，反之亦然，尽管它们共享 docker-compose YAML 语法。一个这样的选项是ulimits:，当与 docker stack deploy 一起使用时，将被忽略并显示警告消息，如下所示：

Ignoring unsupported options: ulimits

我的猜测是，对于您的docker-compose.yml文件，Elasticsearch 在 . 下运行良好docker-compose up，但在docker stack deploy.

使用 Docker Swarm 模式，默认情况下，您定义的 Elasticsearch 实例将无法使用 memlock。目前，尚未正式支持为 docker swarm 服务设置 ulimits。不过，有一些方法可以解决这个问题。

如果主机是Ubuntu，则可以跨 docker 服务启用无限制的 memlock（请参阅此处和此处）。这可以通过以下命令来实现：

echo -e "[Service]\nLimitMEMLOCK=infinity" | SYSTEMD_EDITOR=tee systemctl edit docker.service
systemctl daemon-reload
systemctl restart docker

然而，将 memlock 设置为无穷大并非没有缺点，正如 Elastic 自己在此处所阐明的那样。

根据我的测试，该解决方案适用于 Docker 18.06，但不适用于 18.09。考虑到 Elasticsearch 无法启动的不一致性和可能性，更好的选择是在 Swarm 上部署时不要将 memlock 与 Elasticsearch 一起使用。相反，您可以选择Elasticsearch 文档中提到的任何其他方法来获得类似的结果。

推荐阅读