python - 用 joblib Parallel 填充数组

Question

from joblib import Parallel, delayed
import numpy as np

n = 10
l = np.random.randn(n,50)
a = np.zeros(n)

def myfun(i):
    for k in range(5):
       a[i] = np.correlate(l[0],l[i])[0]

Parallel(n_jobs=6,max_nbytes='50M')(map(delayed(myfun), range(n)))

>> a
  array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

为什么数组a没有被填充？

标签： pythonmultiprocessingjoblib

Answer 1

正在修改的内部数组与a该函数外部的数组不同。我不知道 Parallel 调用是做什么的，但它似乎在复制数组并且您的函数对副本进行操作。也许它克隆了整个全球环境。myfun()aa

这是您的代码的简化版本，它使我所说的更清楚（我尝试添加global调用希望可以解决它，但不是）：

from joblib import Parallel, delayed
import numpy as np

n = 10
a = np.zeros(n)
a[1] = 33

def myfun(i):
    global a
    a[2] = i
    print(a)

Parallel(n_jobs=6,max_nbytes='50M')(map(delayed(myfun), range(n)))

print()
print(a)

结果：

[ 0. 33.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  1.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  2.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  3.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  4.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  5.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  6.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  7.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  8.  0.  0.  0.  0.  0.  0.  0.]
[ 0. 33.  9.  0.  0.  0.  0.  0.  0.  0.]

[ 0. 33.  0.  0.  0.  0.  0.  0.  0.  0.]