(I had posted this in the "more general 'for' loop" thread, but this really is a different idea from that.) Initialising several coroutines at once still doesn't seem clear/clean to me. Here is what I would like. values = awaiting [awaitable, awaitable, ...] a, b, ... = awaiting (awaitable, awaitable, ...) This doesn't have the issues of order because a list of values is returned with the same order of the awaitables. But the awaitables are scheduled in parallel. A regular for loop could still do these in order, but would pause when it gets to a values that hasn't returned/resolved yet. That would probably be expected. for x in awaiting [awaitable, awaitable, ...]: print(x) X is printed in the order of the awaitables. Awaiting sets would be different... they are unordered. So we could use a set and get the items that become available as they become available... for x in awaiting {awaitable, awaitable, ...}: print(x) x would print in an arbitrary order, but that would be what I would expect here. The body could have await calls in it, and so it could cooperate along with the awaiting set of awaitablers. Of course if the for body is only a few statements, that probably wouldn't make much difference. This seems like it's both explicit and simple to think about. It also seems like it might not be that hard to do, I think most of the parts are already worked out. One option is to allow await to work with iterables in this way. But the awaiting keyword would make the code clearer and error messages nicer. The last piece of the puzzle is how to specify the current coroutine manager/runner. import asyncio with asyncio.coroutine_loop(): main() That seems simple enough. It prettty much abstracts out all the coroutine specific stuff to three keyword. async, await, and awaiting. Are async for and async with needed if we have awaiting? Can they be impelented in terms of awaiting? import asyncio async def factorial(name, number): f = 1 for i in range(2, number+1): print("Task %s: Compute factorial(%s)..." % (name, i)) await yielding() f *= i print("Task %s: factorial(%s) = %s" % (name, number, f)) with asyncio.coroutine_loop(): awaiting [ factorial("A", 2), factorial("B", 3), factorial("C", 4)] Compared to the example in asyncio docs... import asyncio @asyncio.coroutine def factorial(name, number): f = 1 for i in range(2, number+1): print("Task %s: Compute factorial(%s)..." % (name, i)) yield from asyncio.sleep(1) f *= i print("Task %s: factorial(%s) = %s" % (name, number, f)) loop = asyncio.get_event_loop() tasks = [ asyncio.async(factorial("A", 2)), asyncio.async(factorial("B", 3)), asyncio.async(factorial("C", 4))] loop.run_until_complete(asyncio.wait(tasks)) loop.close() Cheers, Ron
There are three modes in which you can await multiple coroutines: - iterate over results as they become ready - await till all are done - await till any is done For example C# has helper functions WhenAll and WhenAny for that: await Task.WhenAll(tasks_list); await Task.WhenAny(tasks_list); I can imagine the set of three functions being exposed to user to control waiting for multiple coroutines: asynctools.as_done() # returns asynchronous iterator for iterating over the results of coroutines as they complete asynctools.all_done() # returns a future aggregating results from the given coroutine objects, which awaited returns list of results (like asyncio.gather()) asynctools.any_done() # returns a future, which awaited returns result of first completed coroutine Example: from asynctools import as_done, all_done, any_done corobj0 = async_sql_query("SELECT...") corobj1 = async_memcached_get("someid") corobj2 = async_http_get("http://python.org") # ------------------------------------------------ # Iterate over results as coroutines complete # using async iterator await for result in as_done([corobj0, corobj1, corobj2]): print(result) # ------------------------------------------------ # Await for results of all coroutines # using async iterator results = [] await for result in as_done([corobj0, corobj1, corobj2]): results.append(result) # or using shorthand all_done() results = await all_done([corobj0, corobj1, corobj2]) # ------------------------------------------------ # Await for a result of first completed coroutine # using async iterator await for result in as_done([corobj0, corobj1, corobj2]): first_result = result break # or using shorthand any_done() first_result = await any_done([corobj0, corobj1, corobj2]) Piotr
The asyncio package already has this functionality; check out wait() (it has various options), as_completed(), gather(). On Sat, May 2, 2015 at 2:24 PM, Piotr Jurkiewicz < piotr.jerzy.jurkiewicz@gmail.com> wrote:
There are three modes in which you can await multiple coroutines: - iterate over results as they become ready - await till all are done - await till any is done
For example C# has helper functions WhenAll and WhenAny for that:
await Task.WhenAll(tasks_list); await Task.WhenAny(tasks_list);
I can imagine the set of three functions being exposed to user to control waiting for multiple coroutines:
asynctools.as_done() # returns asynchronous iterator for iterating over the results of coroutines as they complete
asynctools.all_done() # returns a future aggregating results from the given coroutine objects, which awaited returns list of results (like asyncio.gather())
asynctools.any_done() # returns a future, which awaited returns result of first completed coroutine
Example:
from asynctools import as_done, all_done, any_done
corobj0 = async_sql_query("SELECT...") corobj1 = async_memcached_get("someid") corobj2 = async_http_get("http://python.org")
# ------------------------------------------------
# Iterate over results as coroutines complete # using async iterator
await for result in as_done([corobj0, corobj1, corobj2]): print(result)
# ------------------------------------------------
# Await for results of all coroutines # using async iterator
results = [] await for result in as_done([corobj0, corobj1, corobj2]): results.append(result)
# or using shorthand all_done()
results = await all_done([corobj0, corobj1, corobj2])
# ------------------------------------------------
# Await for a result of first completed coroutine # using async iterator
await for result in as_done([corobj0, corobj1, corobj2]): first_result = result break
# or using shorthand any_done()
first_result = await any_done([corobj0, corobj1, corobj2])
Piotr
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
I know that. But the problem with wait() is that it returns Tasks, not their results directly. So user has to unpack them manually. Furthermore, after introduction of `await`, its name will become problematic. It will reassembles `await` too much and can cause a confusion. Its usage would result in an awkward 'await wait()'. There is a function gather(*coros_or_futures) which returns results list directly, like the function all_done() I proposed. But there is no function gather_any(*coros_or_futures), to return just a result of the first done coroutine. (One can achieve it with wait(return_when=FIRST_COMPLETED) but as mentioned before, it does not return a result directly, so there is no symmetry with gather()) Function as_completed() returns indeed an iterator over the futures as they complete, but it is not compatible with the 'async for' protocol proposed in PEP 492. So new function has to be created anyway. Therefore I deliberately placed these functions in a new asynctools module, not in the asyncio module: to emphasize that they are supposed to be used with the new-style coroutines, proposed in PEP 492. I wanted to achieve simplicity (by returning results directly) and symmetry (all_done()/any_done()). Piotr On 2015-05-02 23:29, Guido van Rossum wrote:
The asyncio package already has this functionality; check out wait() (it has various options), as_completed(), gather().
You can try to place these in a separate module, but in the end they still depend on asyncio. You'll find out why when you try to implement any of them. Don't dismiss the effort that went into asyncio too lightly. On Sat, May 2, 2015 at 3:18 PM, Piotr Jurkiewicz < piotr.jerzy.jurkiewicz@gmail.com> wrote:
I know that. But the problem with wait() is that it returns Tasks, not their results directly. So user has to unpack them manually.
Furthermore, after introduction of `await`, its name will become problematic. It will reassembles `await` too much and can cause a confusion. Its usage would result in an awkward 'await wait()'.
There is a function gather(*coros_or_futures) which returns results list directly, like the function all_done() I proposed.
But there is no function gather_any(*coros_or_futures), to return just a result of the first done coroutine. (One can achieve it with wait(return_when=FIRST_COMPLETED) but as mentioned before, it does not return a result directly, so there is no symmetry with gather())
Function as_completed() returns indeed an iterator over the futures as they complete, but it is not compatible with the 'async for' protocol proposed in PEP 492. So new function has to be created anyway.
Therefore I deliberately placed these functions in a new asynctools module, not in the asyncio module: to emphasize that they are supposed to be used with the new-style coroutines, proposed in PEP 492.
I wanted to achieve simplicity (by returning results directly) and symmetry (all_done()/any_done()).
Piotr
On 2015-05-02 23:29, Guido van Rossum wrote:
The asyncio package already has this functionality; check out wait() (it has various options), as_completed(), gather().
-- --Guido van Rossum (python.org/~guido)
participants (3)
-
Guido van Rossum -
Piotr Jurkiewicz -
Ron Adam