[New-bugs-announce] [issue36636] Inner exception is not being raised using asyncio.gather

[Drew Budwin]

New submission from Drew Budwin <drew at budw.in>:

Using Python 3.7, I am trying to catch an exception and re-raise it by following an example I found on StackOverflow (https://stackoverflow.com/a/6246394/1595510). While the example does work, it doesn't seem to work for all situations. Below I have two asynchronous Python scripts that try to re-raise exceptions. The first example works, it will print both the inner and outer exception.

import asyncio

class Foo:
    async def throw_exception(self):
        raise Exception("This is the inner exception")

    async def do_the_thing(self):
        try:
            await self.throw_exception()
        except Exception as e:
            raise Exception("This is the outer exception") from e

async def run():
    await Foo().do_the_thing()

def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(run())

if __name__ == "__main__":
    main()

Running this will correctly output the following exception stack trace:

$ py test.py
Traceback (most recent call last):
  File "test.py", line 9, in do_the_thing
    await self.throw_exception()
  File "test.py", line 5, in throw_exception
    raise Exception("This is the inner exception")
Exception: This is the inner exception

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    main()
  File "test.py", line 18, in main
    loop.run_until_complete(run())
  File "C:\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "test.py", line 14, in run
    await Foo().do_the_thing()
  File "test.py", line 11, in do_the_thing
    raise Exception("This is the outer exception") from e
Exception: This is the outer exception

However, in my next Python script, I have multiple tasks that I queue up that I want to get a similar exception stack trace from. Essentially, I except the above stack trace to be printed 3 times (once for each task in the following script). The only difference between the above and below scripts is the run() function.

import asyncio

class Foo:
    async def throw_exception(self):
        raise Exception("This is the inner exception")

    async def do_the_thing(self):
        try:
            await self.throw_exception()
        except Exception as e:
            raise Exception("This is the outer exception") from e

async def run():
    tasks = []

    foo = Foo()

    tasks.append(asyncio.create_task(foo.do_the_thing()))
    tasks.append(asyncio.create_task(foo.do_the_thing()))
    tasks.append(asyncio.create_task(foo.do_the_thing()))

    results = await asyncio.gather(*tasks, return_exceptions=True)

    for result in results:
        if isinstance(result, Exception):
            print(f"Unexpected exception: {result}")

def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(run())

if __name__ == "__main__":
    main()

The above code snippet produces the disappointingly short exceptions lacking stack traces.

$ py test.py
Unexpected exception: This is the outer exception
Unexpected exception: This is the outer exception
Unexpected exception: This is the outer exception

If I change return_exceptions to be False, I will get the exceptions and stack trace printed out once and then execution stops and the remaining two tasks are cancelled. The output is identical to the output from the first script. The downside of this approach is, I want to continue processing tasks even when exceptions are encountered and then display all the exceptions at the end when all the tasks are completed.

----------
components: asyncio
messages: 340297
nosy: Drew Budwin, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Inner exception is not being raised using asyncio.gather
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36636>
_______________________________________