python - Multiprocessing ProcessPoolExecutor - 在 Windows 命令行中运行时出现 BrokenProcessPool 错误
问题描述
我正在使用 executor.map 并将结果存储在“结果”中。
以下代码在 Jupiter notebook 上没有任何问题。但是,在 Windows 命令提示符下执行 python 脚本时它会崩溃。
rop_test.py
print ('start of process')
import concurrent.futures
from funcfile import func
if __name__ == '__main__':
with concurrent.futures.ProcessPoolExecutor() as executor:
results = executor.map(func, varlist1,varlist2)
global master_list
master_list=list(results)
print(master_list)
#do something with master_list
函数文件.py
def func(var1,var2):
var3=var1+var2
return var3
错误
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="__mp_main__")
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Administrator\notebooks\rop_test.py", line 640, in <module>
print (master_list)
NameError: name 'master_list' is not defined
Finished in 0.0 second(s)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="__mp_main__")
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Administrator\notebooks\rop_test.py", line 640, in <module>
print (master_list)
NameError: name 'master_list' is not defined
Traceback (most recent call last):
File "rop_test.py", line 634, in <module>
master_list=list(results)
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\process.py", line 483, in _chain_from_iterable_of_lists
for element in iterable:
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\_base.py", line 598, in result_iterator
yield fs.pop().result()
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
我尝试将它放在主命令中,添加等待,添加 [var for var in results]、list(results) 等,但无济于事。
注意我在 python 3.6
解决方案
推荐阅读
- javascript - moment.js 无法将实际月份转换为其他格式
- android - 从 react-native-static-server 启动 StaticServer 时出错:未定义不是对象(评估“this.staticServer.start”)
- linux - sh shell 将子shell 重定向到文件,找不到正确的语法
- java - Playframework [Java]:通过代码填充选择选项
- tensorflow - 如何从张量流中的列表中制作所有“唯一”对?
- android - git remote add 命令:如何在 .git/config 文件中编辑远程标签?
- scala - 火花数据帧是否懒惰地加载镶木地板数据?
- php - 按帖子 ID 从帖子中删除标签和分类
- javascript - 如果我们只知道 className 的一部分,如何删除 className?
- javascript - Scrollreveal.js 使用 afterreveal 调用函数