[Python-Dev] async/await in Python; v2

[Arnaud Delobelle]

On 25 April 2015 at 22:02, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
[...]
> On 2015-04-25 4:47 PM, Arnaud Delobelle wrote:
[...]
>> 1.  About the 'async for' construct.  Each iteration will create a new
>> coroutine object (the one returned by Cursor.__anext__()) and it seems
>> to me that it can be wasteful.  In the example given of an 'aiterable'
>> Cursor class, probably a large number of rows will fill the cursor
>> buffer in one call of cursor._prefetch().  However each row that is
>> iterated over will involve the creation execution of a new coroutine
>> object.  It seems to me that what is desirable in that case is that
>> all the buffered rows will be iterated over as in a plain for loop.
>
>
> I agree that creating a new coroutine object is a little bit
> wasteful.
>
> However, the proposed iteration protocol was designed to:
>
> 1. Resemble already existing __iter__/__next__/StopIteration
> protocol;
>
> 2. Pave the road to introduce coroutine-generators in the
> future.

Do you mean that __aiter__() would return a 'coroutine-generator'?
I'm not sure what such an object is but if it is a suspendable
generator in the same way that a coroutine is a suspendable function,
then this is a strong argument to make the __aiter__() magic method a
coroutine rather than a plain function. I.e. __aiter__() would return
either an 'aiterator' or a 'coroutine generator object'.  I think this
could be mentioned in the section 'Why "__aiter__" is a coroutine'
[1].

> We could, in theory, design the protocol to make __anext__
> awaitable return a regular iterators (and raise
> StopAsyncIteration at the end) to make things more
> efficient, but that would complicate the protocol
> tremendously, and make it very hard to program and debug.
>
> My opinion is that this has to be addressed in 3.6 with
> coroutine-generators if there is enough interest from
> Python users.

True, but to me this is bound to happen.  I feel like the semantics of
__anext__() is tied to the behaviour of this yet to be defined
coroutine generator object and that if it turns out that the natural
bevaviour of a coroutine generator is not consistent with the
semantics of __anext__() then it would be a shame.  I must say I have
no evidence that this will happen!

>>
>> 2.  I think the semantics of the new coroutine objects could be
>> defined more clearly in the PEP.  Of course they are pretty obvious
>> when you know that the coroutines are meant to replace
>> asyncio.coroutine as described in [1].  I understand that this PEP is
>> mostly for the benefit of asyncio, hence mainly of interest of people
>> who know it.  However I think it would be good for it to be more
>> self-contained.  I have often read a PEP as an introduction to a new
>> feature of Python.  I feel that if I was not familiar with yield from
>> and asyncio I would not be able to understand this PEP, even though
>> potentially one could use the new constructs without knowing anything
>> about them.
>>
>
> I agree. I plan to update the PEP with some new
> semantics (prohibit passing coroutine-objects to iter(),
> tuple() and other builtins, as well as using them in
> 'for .. in coro()' loops).  I'll add a section with
> a more detailed explanation of coroutine-objects.

Great! Thanks,

-- 
Arnaud

PS: there's a slight asymmetry in the terminology between coroutines
and generators. 'Generator functions' are to 'generators' what
'coroutines' are to 'coroutine objects', which makes it difficult to
what one is talking about when referring to a 'coroutine generator'.


[1] https://www.python.org/dev/peps/pep-0492/#id52