[New-bugs-announce] [issue30556] asyncio.wait very slow with FIRST_COMPLETED

[Samuel Colvin]

New submission from Samuel Colvin:

This is best described the script at https://gist.github.com/samuelcolvin/00f01793c118bf9aafae886ffbc81a58.

Basically, completing a set of tasks using asyncio.wait(...return_when=asyncio.FIRST_COMPLETED) in a loop to wait for them to finish is much much slower than completing the same set of tasks using just asyncio.wait(pending_tasks, loop=loop, return_when=asyncio.ALL_COMPLETED).

In my example, the ALL_COMPLETED case takes ~5s, but the FIRST_COMPLETED case takes 18 - 47s.

The screenshot on the gist shows network usage with a very long tail, in other words very few tasks are taking a long time to complete.

This example uses aiohttp get a url as the test case but I'm pretty sure this isn't an issue with aiohttp.

I tried the same thing with uvloop but got broadly the same result.

----------
components: asyncio
files: asyncio_wait_test.py
messages: 295045
nosy: samuelcolvin, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait very slow with FIRST_COMPLETED
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file46920/asyncio_wait_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30556>
_______________________________________