python - Unclear behavior with exception during asyncio task clean-up

Question

I try to implement a graceful shutdown when a SIGTERM signal is received.

Specifically I want to cleanly handle the case when the tasks do generate an exception during clean-up.

I currently observe a behavior which I cannot make sense of.

Consider the following code.

• It installs callbacks (via loop.add_signal_handler) which raise an exception when a signal is received. This works as expected.
• Tasks are cleaned up in the 'finally:' clause in the task. This also works. Application is terminated and all clean-up parts of the tasks is run, no exception is printed.
• However when there is an exception during clean-up, behavior is strange. I observe an unhandled triple nested exception, even though there should be none.

Can anyone explain this behavior?

Or, even better, make a suggestion to handle things in a better way?

class GracefulExit(SystemExit):
    pass

async def async_main():

    logging.basicConfig(level=logging.INFO)

    try:
        import signal
        
        loop = asyncio.get_event_loop()
        
        def raise_graceful_exit():
            logging.info('SIGINT or SIGTERM received')
            raise GracefulExit()

        loop.add_signal_handler(signal.SIGINT, raise_graceful_exit)
        loop.add_signal_handler(signal.SIGTERM, raise_graceful_exit)
        loop.add_signal_handler(signal.SIGABRT, raise_graceful_exit)
        
        logging.info('Installed SIGINT & SIGTERM handlers')
    except NotImplementedError:  
        # add_signal_handler is not implemented on Windows
        pass

    async def some_task():
        logging.info('enter: some task')    
        try:    
            while True:
                await asyncio.sleep(1)
                logging.info('some_task')
        finally:
            logging.info('cleanup start: some task')
            await asyncio.sleep(3)
            raise ValueError()   # <--- exception during clean-up
            logging.info('cleanup end: some task')
    
    task = asyncio.create_task(some_task())
                         
    try:
        await asyncio.Event().wait()    
    finally:
        logging.info("Main cleanup (-> finally)")
        try:
            await task   # <--this catches the exception during clean-up, but upon termination it is still there
        except ValueError:
            logging.info("some_task generated ValueError")

asyncio.run( async_main() )

This is the output: showing the triple nested exception, which in my opinion, should not be there. This is because I do catch the exception in the line with await task which is guarded with try, except (which also works).

signal_test    | INFO:root:Installed SIGINT & SIGTERM handlers
signal_test    | INFO:root:enter: some task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:some_task
signal_test    | INFO:root:SIGINT or SIGTERM received
signal_test    | INFO:root:Main cleanup (-> finally)
signal_test    | INFO:root:cleanup start: some task
signal_test    | INFO:root:some_task generated ValueError
signal_test    | ERROR:asyncio:unhandled exception during asyncio.run() shutdown
signal_test    | task: <Task finished coro=<async_main.<locals>.some_task() done, defined at /app/run_script.py:88> exception=ValueError()>
signal_test    | Traceback (most recent call last):
signal_test    |   File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run
signal_test    |     return loop.run_until_complete(main)
signal_test    |   File "/usr/local/lib/python3.7/asyncio/base_events.py", line 574, in run_until_complete
signal_test    |     self.run_forever()
signal_test    |   File "/usr/local/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
signal_test    |     self._run_once()
signal_test    |   File "/usr/local/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
signal_test    |     handle._run()
signal_test    |   File "/usr/local/lib/python3.7/asyncio/events.py", line 88, in _run
signal_test    |     self._context.run(self._callback, *self._args)
signal_test    |   File "/app/run_script.py", line 21, in raise_graceful_exit
signal_test    |     raise GracefulExit()
signal_test    | GracefulExit
signal_test    |
signal_test    | During handling of the above exception, another exception occurred:
signal_test    |
signal_test    | Traceback (most recent call last):
signal_test    |   File "/app/run_script.py", line 92, in some_task
signal_test    |     await asyncio.sleep(1)
signal_test    |   File "/usr/local/lib/python3.7/asyncio/tasks.py", line 595, in sleep
signal_test    |     return await future
signal_test    | concurrent.futures._base.CancelledError
signal_test    |
signal_test    | During handling of the above exception, another exception occurred:
signal_test    |
signal_test    | Traceback (most recent call last):
signal_test    |   File "/app/run_script.py", line 107, in async_main
signal_test    |     await task
signal_test    |   File "/app/run_script.py", line 97, in some_task
signal_test    |     raise ValueError()
signal_test    | ValueError
signal_test exited with code 0

标签： pythonpython-asyncio