首页 > 解决方案 > 通过多处理队列 Python 3.6 发送套接字

问题描述

我正在尝试实现一个通用的“超时”功能,它允许我发送一个要运行的功能,如果它在一定时间后没有完成,就杀死它。这是当前的实现:

from multiprocessing import Process, Queue
import multiprocessing as mp
mp.allow_connection_pickling()
from fn.monad import Full, Empty
import traceback


def timeout(timeout, func, args=()):
    '''
    Calls function, and if it times out returns an Empty()
    :param timeout: int | The amount of time to wait for the function
    :param func: () => Any | The function to call (must take no arguments)
    :param queue: Queue | The multiprocessing queue to put the result into
    :return Option | Full(Result) if we get the result, Empty() if it times out
    '''


    queue = Queue()
    p = Process(target=_helper_func, args=(func, queue, args,))
    p.daemon = True
    p.start()
    p.join(timeout)
    if p.is_alive() or queue.empty():
        p.terminate()
        return Empty()
    else:
        out = queue.get()
        # if 'rebuild_handle' in dir(out):
        #   out.rebuild_handle()
        return Full(out)

def _helper_func(func, queue, args):
    try:
        func(*args, queue)
    except Exception as e:
        pass

该函数必须将“返回值”放入多处理队列。但是,当运行超时并将套接字放入队列时,我收到以下错误。

Traceback (most recent call last):
  File "socket_test.py", line 27, in <module>
    print(q.get())
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 239, in _rebuild_socket
    fd = df.detach()
  File "/usr/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/usr/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 487, in Client
    c = SocketClient(address)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
    s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused

我已经尝试过各种以前的堆栈溢出帖子,例如:Python3 Windows multiprocessing passing socket to process

如果您知道此问题的解决方案,请告诉我,因为它给我的代码带来了巨大的压力。谢谢!

标签: pythonpython-3.xsocketsmultiprocessing

解决方案


推荐阅读