python - 线程一次运行时无法减少两个函数的运行时间

Question

我有两个函数f1，f2它们在这两个函数内的循环中增加一个整数特定次数。

我用两种方式调用这些函数。

1) 一个接一个，即先f1接再接f2。2）创建一个线程t1来运行函数f1和线程t2来运行函数f2。

在下面的代码中，我已经尝试了这两种方法。

from threading import Thread
import time
import datetime
from queue import Queue

def f1(a):
    for i in range(1,100000000):
        a+=1
    return a

def f2(a):
    for i in range(1,100000000):
        a+=1
    return a
if __name__ == '__main__':

    que1 = Queue()
    que2 = Queue()

    # t2 = Thread(target=f1(a),name='t2')
    a = 0
    s_t = time.time()
    print('Value of a, before calling function f1: ',a)
    a=f1(a)
    print('Value of a, after calling function f1: ',a)
    a = 0
    print('Value of a, before calling function f2: ',a)
    a=f2(a)
    print('Value of a, after calling function f2: ',a)
    print('Time taken without threads: ',datetime.timedelta(seconds=time.time()-s_t))

    s_t = time.time()
    a = 0
    print('Value of a, before calling function f1 through thread t1: ',a)

    t1 = Thread(target=lambda q, arg1: q.put(f1(arg1)), args=(que1,a),name = 't1')
    print('Value of a, before calling function f2 through thread t2: ',a)

    t2 = Thread(target=lambda q, arg1: q.put(f2(arg1)), args=(que2,a),name = 't2')

    t1.start()
    t2.start()
    t1.join()
    print('Value of a, after calling function f1 through thread t1: ',que1.get())
    t2.join()
    print('Value of a, after calling function f2 through thread t2: ',que2.get())
    print('Time taken with threads: ',datetime.timedelta(seconds=time.time()-s_t))

预期线程比一个接一个地调用函数更快地完成工作，但这里不是这种情况。

这是输出

Value of a, before calling function f1:  0
Value of a, after calling function f1:  99999999
Value of a, before calling function f2:  0
Value of a, after calling function f2:  99999999
Time taken without threads:  0:00:07.623239
Value of a, before calling function f1 through thread t1:  0
Value of a, before calling function f2 through thread t2:  0
Value of a, after calling function f1 through thread t1:  99999999
Value of a, after calling function f2 through thread t2:  99999999
Time taken with threads:  0:00:27.274876

出了什么问题？

标签： pythonmultithreadingparallel-processing

Answer 1

在python中，由于GIL(Global Interpreter Lock) ，一次只能运行一个线程。什么是 GIL？. 因此，在 python 中为 cpu 密集型操作运行线程并不是很有用。但是线程非常适合 I/O。我希望，我澄清了:)

假设python3，您可以使用ProcessPoolExecutorfrom concurrent.futureslike，

$ cat cpuintense.py
import time
from concurrent.futures import ProcessPoolExecutor


def f1(a):
    for i in range(1,100000000):
        a+=1
    return a

def f2(a):
    for i in range(1,100000000):
        a+=1
    return a

def run_in_sequence(a):
    start = time.time()
    f1(a)
    f2(a)
    end = time.time()
    print(f'[Sequential] Took {end-start} seconds')

def run_in_parallel(a):
    with ProcessPoolExecutor(max_workers=2) as pool:
        start = time.time()
        fut1 = pool.submit(f1, a)
        fut2 = pool.submit(f2, a)
        for fut in (fut1, fut2):
            print(fut.result())
        end = time.time()
        print(f'[Parallel] Took {end-start} seconds')


if __name__ == '__main__':
    a = 0
    run_in_sequence(a)
    run_in_parallel(a)

输出：

$ python3 cpuintense.py
[Sequential] Took 6.838468790054321 seconds
99999999
99999999
[Parallel] Took 3.488879919052124 seconds

注意：if __name__ == '__main__'窗户需要防护装置。从文档来看，原因是，

由于 Windows 缺少 os.fork() 它有一些额外的限制：

安全导入主模块

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

For example, under Windows running the following module would fail with a RuntimeError:

from multiprocessing import Process

def foo():
    print 'hello'

p = Process(target=foo)
p.start()

Instead one should protect the “entry point” of the program by using if __name__ == '__main__': as follows:

from multiprocessing import Process, freeze_support

def foo():
    print 'hello'

if __name__ == '__main__':
    freeze_support()
    p = Process(target=foo)
    p.start()

(The freeze_support() line can be omitted if the program will be run normally instead of frozen.)

This allows the newly spawned Python interpreter to safely import the module and then run the module’s foo() function.

Similar restrictions apply if a pool or manager is created in the main module.