python - 使用 Pool() 进行 Python 多处理

Question

我似乎无法让它工作，我正在尝试使用多处理来为列表的所有组合调用我的评估函数；

import itertools
from multiprocessing import Pool

def combinations(inputs):
    pool = Pool()
    combi = list(itertools.combinations(inputs, 5))
    outputs = pool.map(evaluate, combi)
    return outputs

def evaluate(input):
    <do stuff>
    return output

a = [[2,2],[4,3],[5,1],[6,3],[7,2],[12,4],[12,2]]
b = combinations(a)

如果我将其放入 for 循环并一次处理每个循环，则此方法有效。这只是 pool.map() 以及如何传递我无法弄清楚的每个单独的列表项。

标签： pythonmultiprocessing

Answer 1

以下是 21 种组合的原始列表：

import itertools
from multiprocessing import Pool
import os

def combinations(inputs):
    combi = list(itertools.combinations(inputs, 5))
    pool = Pool(len(combi))
    outputs = pool.map(evaluate, combi)
    return outputs


def evaluate(input):
    return ['processed by {0}'.format(os.getpid()), input]


a = [[2,2],[4,3],[5,1],[6,3],[7,2],[12,4],[12,2]]
b = combinations(a)
for i in b:
    print(i)

输出：

['processed by 5185', ([2, 2], [4, 3], [5, 1], [6, 3], [7, 2])]
['processed by 5186', ([2, 2], [4, 3], [5, 1], [6, 3], [12, 4])]
['processed by 5187', ([2, 2], [4, 3], [5, 1], [6, 3], [12, 2])]
['processed by 5188', ([2, 2], [4, 3], [5, 1], [7, 2], [12, 4])]
['processed by 5189', ([2, 2], [4, 3], [5, 1], [7, 2], [12, 2])]
['processed by 5190', ([2, 2], [4, 3], [5, 1], [12, 4], [12, 2])]
['processed by 5191', ([2, 2], [4, 3], [6, 3], [7, 2], [12, 4])]
['processed by 5192', ([2, 2], [4, 3], [6, 3], [7, 2], [12, 2])]
['processed by 5193', ([2, 2], [4, 3], [6, 3], [12, 4], [12, 2])]
['processed by 5194', ([2, 2], [4, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5195', ([2, 2], [5, 1], [6, 3], [7, 2], [12, 4])]
['processed by 5196', ([2, 2], [5, 1], [6, 3], [7, 2], [12, 2])]
['processed by 5197', ([2, 2], [5, 1], [6, 3], [12, 4], [12, 2])]
['processed by 5198', ([2, 2], [5, 1], [7, 2], [12, 4], [12, 2])]
['processed by 5199', ([2, 2], [6, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5200', ([4, 3], [5, 1], [6, 3], [7, 2], [12, 4])]
['processed by 5201', ([4, 3], [5, 1], [6, 3], [7, 2], [12, 2])]
['processed by 5202', ([4, 3], [5, 1], [6, 3], [12, 4], [12, 2])]
['processed by 5203', ([4, 3], [5, 1], [7, 2], [12, 4], [12, 2])]
['processed by 5204', ([4, 3], [6, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5205', ([5, 1], [6, 3], [7, 2], [12, 4], [12, 2])]