[New-bugs-announce] [issue41623] What's the by-design behavior when os.fork() is invoked in an asyncio loop?

[simin lin]

New submission from simin lin <linsm08 at gmail.com>:

I have the same quesion in stackoverflow. Please refer to
https://stackoverflow.com/questions/63558555/whats-the-by-design-behavior-when-os-fork-is-invoked-in-an-asyncio-loop
to get a better format.


Does asyncio work with os.fork()?

Code Snippet 1:

import asyncio
import os

import aiohttp


async def main():
    url = "https://google.com"
    pid = os.fork()
    if pid == 0:
        # child process
        print("in child process")
        await fetch(url)
        print("in child process done")
    else:
        print("in parent process")
        await asyncio.sleep(20)
        print("in parent process done")


async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

if __name__ == "__main__":
    asyncio.run(main())
Code above works fine.

Code Snippet 2:

import asyncio
import os

import aiohttp


async def main():
    url = "https://google.com"
    pid = os.fork()
    if pid == 0:
        # child process
        print("in child process")
        await asyncio.sleep(10)                  # different with code snippet 1
        # await fetch(url)
        print("in child process done")
    else:
        print("in parent process")
        await asyncio.sleep(20)
        print("in parent process done")


async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

if __name__ == "__main__":
    asyncio.run(main())
Code above will raise following exception:

Traceback (most recent call last):
  File "fork_sleep.py", line 28, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "fork_sleep.py", line 13, in main
    await asyncio.sleep(10)                  # different with code snippet 1
  File "/usr/lib/python3.8/asyncio/tasks.py", line 637, in sleep
    loop = events.get_running_loop()
RuntimeError: no running event loop
The reason for the "no running event loop" exception is that function get_running_loop compare the os.getpid() and the pid saved in loop. When they are different, the exception above is raised.

Please refer to the following code from cpython source code.

def get_running_loop():
    """Return the running event loop.  Raise a RuntimeError if there is none.

    This function is thread-specific.
    """
    # NOTE: this function is implemented in C (see _asynciomodule.c)
    loop = _get_running_loop()
    if loop is None:
        raise RuntimeError('no running event loop')
    return loop


def _get_running_loop():
    """Return the running event loop or None.

    This is a low-level function intended to be used by event loops.
    This function is thread-specific.
    """
    # NOTE: this function is implemented in C (see _asynciomodule.c)
    running_loop, pid = _running_loop.loop_pid
    if running_loop is not None and pid == os.getpid():
        return running_loop
So it seems that asyncio event loop works fine in child process if you don't touch the function get_running_loop. My question is, what is the by-design behavior? Why the author check the pid in function _get_running_loop? And what is the solution if you encounter the "no running event loop" in child process.

----------
components: asyncio
messages: 375839
nosy: asvetlov, linsm08, yselivanov
priority: normal
severity: normal
status: open
title: What's the by-design behavior when os.fork() is invoked in an asyncio loop?
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue41623>
_______________________________________