时间戳

首页 > 解决方案 > 将 Celery 与 Django 一起使用时，文件描述符超出了 select() 的范围

python - 将 Celery 与 Django 一起使用时，文件描述符超出了 select() 的范围

问题描述

我正在使用 Django、Celery 和 Channels（带有 redis 后端）来处理基于 Dajngo 的后端中的任务。最近，随着事情的扩大，我面临的问题是：

ValueError('filedescriptor out of range in select()',)

Traceback (most recent call last):
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/asgiref/sync.py", line 46, in __call__
    loop.run_until_complete(self.main_wrap(args, kwargs, call_result))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 471, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1415, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/gevent/monkey.py", line 831, in _select
    return select.select(*args, **kwargs)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/gevent/select.py", line 145, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/celery/app/trace.py", line 648, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/cbt/backend/cbproj/tasks/tasks.py", line 1095, in start_new_subgame_timer
    add_users_to_subgame(game, game_type)
  File "/home/cbt/backend/cbproj/tasks/tasks.py", line 1405, in add_users_to_subgame
    async_to_sync(group_send_empty_subgame_audio_game_response)(game, game_type)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/asgiref/sync.py", line 50, in __call__
    loop.run_until_complete(loop.shutdown_asyncgens())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 471, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1415, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/gevent/monkey.py", line 831, in _select
    return select.select(*args, **kwargs)
  File "/home/cbt/backend/venv/lib/python3.6/site-packages/gevent/select.py", line 145, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()

我正在使用以下软件包和python 3.6

此外，当我使用async_to_syncdjango 频道库提供的功能时，也会发生这种情况（但并非每次都如此）。有解决办法吗？

我打开文件的限制是 20000，我绝对没有达到这个限制。

标签： pythonpython-3.xdjangocelery

解决方案

根据您的错误日志，我猜monkey.patch_all您的项目中使用了，如下所示：

from gevent import monkey
monkey.patch_all()

在旧的 gevent 版本（<1.5.0）中，monkey.patch_all()设置DefaultSelector = SelectSelector. 当 fileno > 1024 时，SelectSelector 将引发此类 ValueError。您可以检查此问题并将 gevent 升级到新版本（1.5.0+），以便解决此问题。

Gevent 的 monkey.patchDefaultSelector = PollSelector在新版本中设置。

推荐阅读