时间戳

首页 > 解决方案 > OSError：[Errno 24] python 导入时打开的文件过多

linux - OSError：[Errno 24] python 导入时打开的文件过多

问题描述

我有一个从 c 代码调用 python 的测试环境：

system("flows_parser.py ....");

运行了很长时间后，我每次都会收到此错误：

Traceback (most recent call last):
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/flows_parser.py", line 32, in <module>
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/../../shared/proto_steps/common/generated_py/proto_steps_common_steps_defines_pb2.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.5/dist-packages/google/protobuf/reflection.py'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/lib/python3/dist-packages/apport/__init__.py'

Original exception was:
Traceback (most recent call last):
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/flows_parser.py", line 32, in <module>
  File "/homes/fw/antvai01/work/cc312r2/cc_validation/hta/flows_parser/../../shared/proto_steps/common/generated_py/proto_steps_common_steps_defines_pb2.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 766, in get_code
  File "<frozen importlib._bootstrap_external>", line 818, in get_data
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.5/dist-packages/google/protobuf/reflection.py'

flow_parser.py proto_steps_common_steps_defines_pb2 的导入中的第 32 行 proto_steps_common_steps_defines_pb2.py 第 9 行也是反射的导入。

我在 Ubuntu 16.4 输出：sysctl -A | grep shm

kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
sysctl: reading key "net.ipv6.conf.all.stable_secret"
sysctl: reading key "net.ipv6.conf.default.stable_secret"
sysctl: reading key "net.ipv6.conf.enp0s3.stable_secret"
sysctl: reading key "net.ipv6.conf.enp0s8.stable_secret"
sysctl: reading key "net.ipv6.conf.lo.stable_secret"
vm.hugetlb_shm_group = 0

所以 shmmax 应该不是问题。有人可以帮忙吗？

标签： linuxpython-3.x

解决方案

谢谢里卡多布兰科！它是来自 c 代码的描述符韭菜，与 python 无关。我创建了管道，完成后没有关闭它们。我找到了描述符韭菜并插入它，现在它工作正常。

推荐阅读