[New-bugs-announce] [issue20566] asyncio as_completed() thrashes adding and removing callbacks
report at bugs.python.org
Sat Feb 8 21:18:14 CET 2014
New submission from Glenn Langford:
In asyncio, tasks.py as_completed() appears to trigger adding and removing callbacks multiple times for the pending set of futures, each time a single future completes.
For example, to wait for 5 futures which complete at different times:
- as_completed() uses _wait()
- _wait() will add 5 callbacks, one for each future
- when one future completes, a callback is triggered and all 5 callbacks are removed; this is because _wait() was called with FIRST_COMPLETED
- for the 4 remaining futures - 4 callbacks have to be added back again
- when the next future completes, the 4 callbacks are removed
The worst case is if as_completed() is called to wait for all of a larger number of futures, which complete at different times. For example, with 100 futures worst case, ~10,000 callback adds and removes would be performed.
(I am very new to the asyncio code, so I don't have a patch to offer at this point).
components: Library (Lib)
nosy: glangford, gvanrossum, haypo, pitrou
title: asyncio as_completed() thrashes adding and removing callbacks
versions: Python 3.4, Python 3.5
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce