python - Python中的多级多处理

Question

我意识到我的问题非常广泛，但我也在寻找广泛的答案:)

我有 2 个功能：outer和inner.

import traceback
import multiprocessing

def inner():
    return "blah"

def outer():
    success: bool = True
    process_pool: multiprocessing.Pool = multiprocessing.Pool(XXX)
    try:
        process_pool.map(inner)
        process_pool.close()
    except Exception as ex:
        print(ex)
        print(traceback.print_exc())
        success = False
        process_pool.terminate()
    except KeyboardInterrupt as ex:
        print(ex)
        success = False
        process_pool.terminate()
    finally:
        process_pool.join()
    return success

我现在想创建一个space执行outer方式相同的函数outerexecutes inner。

def space():
    success: bool = True
    process_pool: multiprocessing.Pool = multiprocessing.Pool(YYY)
    try:
        process_pool.map(outer)
        process_pool.close()
    except Exception as ex:
        print(ex)
        print(traceback.print_exc())
        success = False
        process_pool.terminate()
    except KeyboardInterrupt as ex:
        print(ex)
        success = False
        process_pool.terminate()
    finally:
        process_pool.join()
    return success

假设我有 100 个 CPU 可供使用。如何设置该space功能？在 XXX 和 YYY 之间拆分（每个多处理函数中的 CPU 数量）？这样做是否明智？

标签： pythonmultithreadingmultiprocessing