[Async-sig] Cancelling a coroutine from a signal handler?

Mark E. Haase mehaase at gmail.com
Tue Apr 24 17:25:46 EDT 2018

I am trying to understand some unexpected behavior in asyncio. My goal is to
use a custom signal handler to cleanly unwind an asyncio program that has
different tasks running. Here's a simplified test case:

     1  import asyncio, logging, random, signal, sys
     3  logging.basicConfig(level=logging.DEBUG)
     4  logger = logging.getLogger()
     6  async def main():
     7      try:
     8          await asyncio.Event().wait()
     9      except asyncio.CancelledError:
    10          logger.info('cancelled main()')
    11          # cleanup logic
    13  def handle_sigint(signal, frame):
    14      global sigint_count
    15      global main_coro
    16      sigint_count += 1
    17      if sigint_count == 1:
    18          logger.warn('Received interrupt: shutting down...')
    19          main_coro.throw(asyncio.CancelledError())
    20          # missing event loop logic?
    21      else:
    22          logger.warn('Received 2nd interrupt: exiting!')
    23          main_coro.throw(SystemExit(1))
    25  sigint_count = 0
    26  signal.signal(signal.SIGINT, handle_sigint)
    27  loop = asyncio.get_event_loop()
    28  main_coro = main()
    29  try:
    30      loop.run_until_complete(main_coro)
    31  except StopIteration:
    32      logger.info('run_until_complete() finished')

The main() function is a placeholder that represents some long running task,
e.g. a server that is waiting for new connections. The handle_sigint()
is supposed to attempt to cancel main() so that it can gracefully exit, but
it receives a second interrupt, then the process exits immediately.

Here's an example running the program and then typing Ctrl+C.

    $ python test.py
    DEBUG:asyncio:Using selector: EpollSelector
    ^CWARNING:root:Received interrupt: shutting down...
    INFO:root:cancelled main()
    INFO:root:run_until_complete() finished

This works as I expect it to. Of course my cleanup logic (line 10) isn't
actually doing anything. In a real server, I might want to send goodbye
to connected clients. To mock this behavior, I'll modify line 11:

    11          await asyncio.sleep(0)

Surprisingly, now my cleanup code hangs:

    $ python test.py
    DEBUG:asyncio:Using selector: EpollSelector
    ^CWARNING:root:Received interrupt: shutting down...
    INFO:root:cancelled main()
    ^CWARNING:root:Received 2nd interrupt: exiting!

Notice that the program doesn't exit after the first interrupt. It enters
exception handler and appears to hang on the await expression on line 11. I
to interrupt it a second time, which throws SystemExit instead.

I puzzled over this for quite some time until I realized that I can force
to resume by changing line 20:

    20          main_coro.send(None)

With this change, the interrupt causes the cleanup logic to run to
and the program exits normally. Of course, if I add a second await

    11          await asyncio.sleep(0); await asyncio.sleep(0)

Then I also have to step twice:

    20          main_coro.send(None); main_coro.send(None)

My mental model of how the event loop works is pretty poor, but I roughly
understand that the event loop is responsible for driving coroutines. It
here that the event loop has stopped driving my main() coroutine, and so the
only way to force it to complete is to call send() from my code.

Can somebody explain *why* the event loop is not driving my coroutine? Is
this a
bug or am I missing something conceptually?

More broadly, handling KeyboardInterrupt in async code seems very tricky,
but I also cannot figure out how to make this interrupt approach work. Is
of these better than the other? What is the best practice here? Would it be
terrible to add `while True: main_coro.send(None)` to my signal handler?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20180424/114295c9/attachment.html>

More information about the Async-sig mailing list