时间戳

首页 > 解决方案 > python ProcessPoolExecutor在函数中不起作用

python - python ProcessPoolExecutor在函数中不起作用

问题描述

python ProcessPoolExecutor 在命令行中工作，但在添加到函数后不运行

它是这样工作的

from concurrent import futures

def multi_process(func, paras, threads):
    with futures.ProcessPoolExecutor(max_workers=threads) as pool:
        res = pool.map(func, paras, chunksize=threads)
    return list(res)
p = multi_process(func,paras,threads)

但根本不工作，如下所示

def upper(paras,threads):
    def func:
        some func
    def multi_process(func, paras, threads):
        with futures.ProcessPoolExecutor(max_workers=threads) as pool:
            res = pool.map(func, paras, chunksize=threads)
        return list(res)
    p = multi_process(func,paras,threads)
    return p
p = upper(paras,threads)

没有警告或错误但很长时间没有任何响应。

标签： pythonmultiprocessing

解决方案

你确实得到了一个错误。它的。

AttributeError: Can't pickle local object 'upper.<locals>.func'.

原因是多处理工作需要在全局级别定义函数。

要实现您想要的，您可以执行以下操作：

from concurrent import futures

# Has to be a global function
def func(para):
    print(para)


def upper(paras,threads):
    # This cannot be a local function.
    #def func(para):
    #    print(para)
    def multi_process(func, paras, threads):
        with futures.ProcessPoolExecutor(max_workers=threads) as pool:
            res = pool.map(func, paras, chunksize=threads)
        return list(res)
    p = multi_process(func, paras, threads)
    return p

paras = [1, 2, 3]
threads = 3
p = upper(paras,threads)

推荐阅读