python - Multiprocessing Module Error: `if __name__ == '__main__':
问题描述
I'm trying to run a code in multiprocessing mode using all the threads available to my CPU (MacBook Pro 16 inch 2019) and am getting an error I don't get running the exact same code on a linux environment for my windows computer.
I have never seen this error before. I tried uninstalling and reinstalling the multiprocessing
module, but that didn't help. What is causing this issue? Again, the exact same code works on a Linux environment.
pool = Pool(processes = ncpus)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 212, in __init__
self._repopulate_pool()
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
w.start()
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "/Users/aeglick/opt/anaconda3/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
解决方案
You have to run your Pool inside the block __main__
:
from multiprocessing import Pool
def f(x):
return x*x
if __name__ == '__main__': # <- you have to execute your Pool from here
with Pool(5) as p:
print(p.map(f, [1, 2, 3]))
So you can't launch a multiprocess
code from a Python/IPython console.
Read this: https://docs.python.org/3/library/multiprocessing.html#multiprocessing-programming
推荐阅读
- html - 水平导航栏的元素不顶部对齐
- android - 您可以通过蓝牙实时流式传输相机吗?
- html - 具有最大宽度的 Div,但是当内容换行时,适合内容
- c++ - 在 C++ 中,return 语句可以在 for 循环内的 catch 块内吗?
- vue.js - @nuxtjs/google-gtag @nuxtjs/gtm @nuxtjs/google-analytics 和 vue-gtag 有什么区别,GA4 用什么?
- android - Android Https 在发布版本中不起作用
- database - 有什么方法可以将外部数据库与 wordpress elementor 连接并将不同页面中的表单数据存储到不同的表中?
- qt - Qml 一侧带边框的圆角
- c# - 如何访问 Program.cs 中的 Blazor WASM 会话存储?
- c# - 无法链接两个方法C#