[Python-ideas] PEP 525: Asynchronous Generators

[Yury Selivanov]

On 2016-08-05 4:51 PM, Sven R. Kunze wrote:
> On 04.08.2016 17:17, Yury Selivanov wrote:
>>
>>> From your answer to Stefan, I get the impression that the reason why 
>>> we actual need all those a* methods (basically a duplication of the 
>>> existing gen protocol), is the fact that normal generators can be 
>>> converted to coroutines. That means, 'yield' still can be used in 
>>> both ways.
>>>
>>> So, it's a technical symptom of the backwards-compatibility rather 
>>> than something that cannot be avoided by design. Is this correct?
>>>
>>
>> async/await in Python is implemented on top of the generator 
>> protocol.  Any 'await' is either awaiting on a coroutine or on a 
>> Future-like object.  Future-like objects are defined by implementing 
>> the __await__ method, which should return a generator.
>>
>> So coroutines and generators are very intimately tied to each other, 
>> and that's *by design*.
>>
>> Any coroutine that iterates over an asynchronous generator uses the 
>> generator protocol behind the scenes.  So we have to multiplex the 
>> async generaotor's "yields" into the generator protocol in such a 
>> way, that it stays isolated, and does not interfere with the "yields" 
>> that drive async/await.
>
> Yes, that is how I understand it as well. So, all this complexity 
> stems from the intertwining of generators and coroutines.
>
> I am wondering if loosening the tie between the two could make it all 
> simpler; even for you. (Not right now, but maybe later.)

Unfortunately, the only way to "loosen the tie" is to re-implement 
generator protocol for coroutines (make them a separate thing). Besides 
breaking backwards compatibility with the existing Python code that uses 
@coroutine decorator and 'yield from' syntax, it will also introduce a 
ton of work.

>
>
>
>>
>>> If it's correct, would you think it would make sense to get rid of 
>>> the a* in a later iteration of the async capabilities of Python? So, 
>>> just using the normal generator protocol again?
>>
>> Because async generators will contain 'await' expressions, we have to 
>> have a* methods (although we can name them without the "a" prefix, 
>> but I believe that would be confusing for many users).
>>
>
> I think that is related to Stefan's question here. I cannot speak for 
> him but it seems to me the confusion is actually the other way around.
>
> Coming from a database background, I prefer redundant-free data 
> representation. The "a" prefix encodes the information "that's an 
> asynchronous generator" which is repeated several times and thus is 
> redundant.
>
> Another example: when writing code for generators and checking for the 
> "send" method, I would then need to check for the "asend" method as 
> well. So, I would need to touch that code although it even might not 
> have been necessary in the first place.
>
> I don't want to talk you out of the "a" prefix, but I get the feeling 
> that the information "that's asynchronous" should be provided as a 
> separate attribute. I believe it would reduce confusion, simplify 
> duck-typing and flatten the learning curve. :)


Thing is you can't write one piece of code that will accept any type of 
generator (sync or async).

* send, throw, close, __next__ methods for sync generators, they are 
synchronous.

* send, throw, close, __next__ methods for async generators, they 
*require* to use 'await' on them.  There is no way to make them 
"synchronous", because you have awaits in async generators.

Because of the above, duck-typing simply isn't possible.

The prefix is there to make people aware that this is a completely 
different API, even though it looks similar.

Thank you,
Yury