python - 使用 atexit 终止线程时脚本卡在退出时
问题描述
我在 python 3.7.4 上玩线程,我想用它atexit
来注册一个清理函数,它将(干净地)终止线程。
例如:
# example.py
import threading
import queue
import atexit
import sys
Terminate = object()
class Worker(threading.Thread):
def __init__(self):
super().__init__()
self.queue = queue.Queue()
def send_message(self, m):
self.queue.put_nowait(m)
def run(self):
while True:
m = self.queue.get()
if m is Terminate:
break
else:
print("Received message: ", m)
def shutdown_threads(threads):
for t in threads:
print(f"Terminating thread {t}")
t.send_message(Terminate)
for t in threads:
print(f"Joining on thread {t}")
t.join()
else:
print("All threads terminated")
if __name__ == "__main__":
threads = [
Worker()
for _ in range(5)
]
atexit.register(shutdown_threads, threads)
for t in threads:
t.start()
for t in threads:
t.send_message("Hello")
#t.send_message(Terminate)
sys.exit(0)
但是,似乎与atexit
回调中的线程和队列交互会使用一些内部关闭例程创建死锁:
$ python example.py
Received message: Hello
Received message: Hello
Received message: Hello
Received message: Hello
Received message: Hello
^CException ignored in: <module 'threading' from '/usr/lib64/python3.7/threading.py'>
Traceback (most recent call last):
File "/usr/lib64/python3.7/threading.py", line 1308, in _shutdown
lock.acquire()
KeyboardInterrupt
Terminating thread <Worker(Thread-1, started 140612492904192)>
Terminating thread <Worker(Thread-2, started 140612484511488)>
Terminating thread <Worker(Thread-3, started 140612476118784)>
Terminating thread <Worker(Thread-4, started 140612263212800)>
Terminating thread <Worker(Thread-5, started 140612254820096)>
Joining on thread <Worker(Thread-1, stopped 140612492904192)>
Joining on thread <Worker(Thread-2, stopped 140612484511488)>
Joining on thread <Worker(Thread-3, stopped 140612476118784)>
Joining on thread <Worker(Thread-4, stopped 140612263212800)>
Joining on thread <Worker(Thread-5, stopped 140612254820096)>
All threads terminated
(这KeyboardInterrupt
是我使用ctrl-c
的,因为该过程似乎无限期地挂起)。
但是,如果我Terminate
在退出之前发送消息(取消注释之后的行t.send_message("Hello")
),程序不会挂起并正常终止:
$ python example.py
Received message: Hello
Received message: Hello
Received message: Hello
Received message: Hello
Received message: Hello
Terminating thread <Worker(Thread-1, stopped 140516051592960)>
Terminating thread <Worker(Thread-2, stopped 140516043200256)>
Terminating thread <Worker(Thread-3, stopped 140515961992960)>
Terminating thread <Worker(Thread-4, stopped 140515953600256)>
Terminating thread <Worker(Thread-5, stopped 140515945207552)>
Joining on thread <Worker(Thread-1, stopped 140516051592960)>
Joining on thread <Worker(Thread-2, stopped 140516043200256)>
Joining on thread <Worker(Thread-3, stopped 140515961992960)>
Joining on thread <Worker(Thread-4, stopped 140515953600256)>
Joining on thread <Worker(Thread-5, stopped 140515945207552)>
All threads terminated
这就引出了一个问题,threading._shutdown
相对于处理程序,该例程何时执行atexit
?在处理程序中与线程交互是否有意义atexit
?
解决方案
您可以使用一个守护线程来要求您的非守护线程优雅地进行清理。例如,如果您使用的是启动非守护线程的第三方库,则必须更改该库或执行以下操作:
import threading
def monitor_thread():
main_thread = threading.main_thread()
main_thread.join()
send_signal_to_non_daemon_thread_to_gracefully_shutdown()
monitor = threading.Thread(target=monitor_thread)
monitor.daemon = True
monitor.start()
start_non_daemon_thread()
把它放在原始海报代码的上下文中(注意我们不需要 atexit 函数,因为在所有非守护线程停止之前不会调用它):
if __name__ == "__main__":
threads = [
Worker()
for _ in range(5)
]
for t in threads:
t.start()
for t in threads:
t.send_message("Hello")
#t.send_message(Terminate)
def monitor_thread():
main_thread = threading.main_thread()
main_thread.join()
shutdown_threads(threads)
monitor = threading.Thread(target=monitor_thread)
monitor.daemon = True
monitor.start()
推荐阅读
- windows - windows - 为什么firefox只信任certutil安装的证书?
- jhipster - Jhipster 与 jdk-11
- javascript - 用户在游戏客户端中导航到站点从未知站点获取 HTML 响应
- ios - +[NSManagedObjectModel mergeModelFromBundles::forStoreMetadata:] 总是返回 nil
- ruby - 用一个期望测试自定义异常?
- c++ - 如何在保持多态性的同时将成员函数添加到需要它的继承类而不影响其他兄弟类?
- java - 为什么 BufferedWriter 拒绝写入文件?
- javascript - 从字符串中删除 html 元素和新行?
- c++ - 字符串声明中的分段
- javascript - 获取详细信息后动态检查附加到 DOM 的输入框