[Python-ideas] awaiting ... was Re: More general "for" loop handling

[Ron Adam]

On 04/30/2015 09:07 PM, Yury Selivanov wrote:
> On 2015-04-30 9:02 PM, Ethan Furman wrote:
>> On 04/30, Yury Selivanov wrote:
>>> On 2015-04-30 8:35 PM, Steven D'Aprano wrote:
>>>> I don't think it guarantees ordering in the sense I'm referring to. It
>>>> guarantees that the returned result will be [f(a), f(b), f(c), ...] in
>>>> that order, but not that f(a) will be calculated before f(b), which is
>>>> calculated before f(c), ... and so on. That's the point of parallelism:
>>>> if f(a) takes a long time to complete, another worker may have completed
>>>> f(b) in the meantime.
>>> This is an *excellent* point.
>> So, PEP 492 asynch for also guarantees that the loop runs in order, one at
>> a time, with one loop finishing before the next one starts?
>>
>> *sigh*
>>
>> How disappointing.

> No.  Nothing prevents you from scheduling asynchronous
> parallel computation, or prefetching more data.  Since
> __anext__ is an awaitable you can do that.
>
> Steven's point is that Todd's proposal isn't that
> straightforward to apply.

Initialising several coroutines at once still doesn't seem clear/clean to 
me.  Or maybe I'm just not getting that part yet.

Here is what I would like. :-)

values = awaiting [awaitable, awaitable, ...]
a, b, ... = awaiting (awaitable, awaitable, ...)

This doesn't have the issues of order because a list of values is returned 
with the same order of the awaitables.  But the awaitables are scheduled in 
parallel.

A regular for loop could still do these in order, but would pause when it 
gets to a values that haven't returned/resolved yet.  That would probably 
be expected.

Awaiting sets would be different... they are unordered.  So we can use a 
set and get the items that become available as they become available...

     for x in awaiting {awaitable, awaitable, ...}:
         print(x)

x would print in an arbitrary order, but that would be what I would expect 
here. :-)

The body could have await calls in it, and so it could cooperate along with 
the awaiting set.  Of course if it's only a few statements, that probably 
wouldn't make much difference.

This seems like it's both explicit and simple to think about.  It also 
seems like it might not be that hard to do, I think most of the parts are 
already worked out.

One option is to allow await to work with iterables in this way.  But the 
awaiting keyword would make the code clearer and error messages nicer.

Cheers,
    Ron