[Python-ideas] async/await in Python

[Yury Selivanov]

Hi David,

Thanks a lot for feedback; answers below:

On 2015-04-17 6:24 PM, David Wilson wrote:
> On Fri, Apr 17, 2015 at 02:58:58PM -0400, Yury Selivanov wrote:
>
>> I believe that PEPs 380 and 3156 were a major breakthrough for Python
>> 3, and I hope that this PEP can be another big step.  Who knows, maybe
>> it will be one of the reasons to drive people's interest towards
>> Python 3.
> An offer like this would assuredly and irrevocably propel me from my
> "Python 3 conservative" position and well into the "Python 3 zealot"
> stratosphere :)
Such a relief to hear that!

>
> A great deal of thought has obviously gone into your proposal, so I
> expect you already have answers to these:
>
>
>      Regarding introduction of async/await keywords, could e.g. the
> tokenizer change be avoided by moving the 'async' keyword after the
> parameter list? like "def foo(x) async:". That would seem parseable
> without introducing any new token, since the parser could be made to
> error out if any identifier except "async" appears, similarly instead of
> introducing an 'await' keyword, 'yield await' might be possible. (Please
> forgive me if my parsing ignorance is showing through)

I'm not entirely sure that it's possible to modify tokenizer
to understand "def() async:"

For instance, for the this code:

"async = 1; print = 2; lst[async:print]"

we'll need to do a hell of a look-ahead.  Or do some smart
stack tracking.. Better just to either use the proposed mechanism
or __future__ import.

>
>
>      Further regarding the introduction of the await keyword, within an
> async function did you consider the possibility of making suspension of
> the coroutine implicit?
That would imply implementing mechanisms similar to what
greenlets module does, or stackless python.  My understanding
is that is never going to happen, for better or worse.

>
>  From the perspective of the interpreter, objects exposing __await__
> might grow a .tp_await slot which could trivially be tested for NULL
> during e.g. the CALL_FUNCTION opcode epilog, though I guess this stuff
> strays quite far from well understod async/await designs from other
> languages (e.g. C#).
I think this is an optimization (or better C api?) that we can
consider later.

>
> I quite like the idea of folding the mechanism out of the user's sight
> as far as practical, though I don't know if that's possible without
> introducing ambiguities.
>
>
>      Is there a need or desire for "async lambda ...:"? Notably, it's
> possible though fairly useless to define a lambda generator:
> "lambda x: (yield from x)" works today.

If we use __future__ import it should be possible to modify
grammar to parse that.  Although I don't like the syntax.

>
>
>      Generally the magic to avoid backwards compatibility issues seem
> less desirable than simply including a new __future__ mode, though the
> stated reasons for the trickery are good. Somewhat contradicting my last
> point, __future__ serves as very useful documentation for a file's
> implementation.
I'm not opposed to have __future__ imports. If most people
think that they are better, we'll update the proposal and
reference implementation.

>
> Regarding the problematic module import example, that might easily be
> solved by simple file rename and introducing a stub module for old code
> like:
>
>      async_impl.py:
>          # Guts of old async.py
>
>      async.py:
>          from .async_impl import *  # NOQA
>
> Similarly for object attributes and suchlike, properties might suffice.
Agree!

>
> Thanks for proposing this! Off to buypopcorn, very much looking forward
> to seeing how the thread plays out. ;)

Thanks a lot again!

Yury