[Python-ideas] async/await in Python

[Chris Angelico]

On Sat, Apr 18, 2015 at 4:58 AM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> The following new syntax is used to declare a coroutine::
>
>     async def read_data(db):
>         pass
>
> Key properties of coroutines:
>
> * Coroutines are always generators, even if they do not contain ``await``
>   expressions.

Do you mean that a coroutine is a special form of generator, or that a
function declared with "async" is always a coroutine even if it
doesn't contain an "await"? You're separating coroutines from
generators in many ways, so maybe I'm just misreading this.

> Asynchronous Context Managers and "async with"
> ----------------------------------------------
>
> An *asynchronous context manager* is a context manager that is able to
> suspend
> execution in its *enter* and *exit* methods.

A regular 'with' block guarantees that subsequent code won't execute
until __exit__ completes. I presume the same guarantee is applied to
an 'async with' block, and thus it can be used only inside a
coroutine?

(Edit: Oops, missed where that's said further down - SyntaxError to
use it outside a coroutine. Might still be worth clarifying the
guarantee somewhere, otherwise ignore me.)

> New Syntax
> ''''''''''
>
> A new statement for iterating through asynchronous iterators is proposed::
>
>     async for TARGET in ITER:
>         BLOCK
>     else:
>         BLOCK2
>
> which is semantically equivalent to::
>
>     iter = (ITER)
>     iter = await type(iter).__aiter__(iter)
>     running = True
>     while running:
>         try:
>             TARGET = await type(iter).__anext__(iter)
>         except StopAsyncIteration:
>             running = False
>         else:
>             BLOCK
>     else:
>         BLOCK2

(Not sure why you don't just use "break" instead of "running = False"?
Maybe I'm blind to some distinction here.)

> Why StopAsyncIteration?
> '''''''''''''''''''''''
>
> Coroutines are still based on generators internally.  So, before PEP 479,
> there
> was no fundamental difference between
>
> ::
>
>     def g1():
>         yield from fut
>         return 'spam'
>
> and
>
> ::
>
>     def g2():
>         yield from fut
>         raise StopIteration('spam')
>
> And since PEP 479 is accepted and enabled by default for coroutines, the
> following example will have its ``StopIteration`` wrapped into a
> ``RuntimeError``
>
> ::
>
>     async def a1():
>         await fut
>         raise StopIteration('spam')
>
> The only way to tell the outside code that the iteration has ended is to
> raise
> something other than ``StopIteration``.  Therefore, a new built-in exception
> class ``StopAsyncIteration`` was added.
>
> Moreover, with semantics from PEP 479, all ``StopIteration`` exceptions
> raised
> in coroutines are wrapped in ``RuntimeError``.

Can't a coroutine simply use 'return'? I'm not hugely familiar with
them, but I'm not understanding here why you need a completely
separate exception - and, more importantly, why you wouldn't have the
same consideration of "StopAsyncIteration leakage". It ought to be
possible for the coroutine to return, StopIteration be synthesized,
and the __anext__ function to bubble that out - exactly the way it
does for generators today.

> Glossary
> ========
>
> :Coroutine:
>     A coroutine function, or just "coroutine", is declared with ``async
> def``.
>     It uses ``await`` and ``return value``; see `New Coroutine Declaration
>     Syntax`_ for details.

(Using this definition to justify my statement above)

> Why not a __future__ import
> ---------------------------
>
> ``__future__`` imports are inconvenient and easy to forget to add. Also,
> they
> are enabled for the whole source file.  Consider that there is a big project
> with a popular module named "async.py".  With future imports it is required
> to
> either import it using ``__import__()`` or ``importlib.import_module()``
> calls,
> or to rename the module.  The proposed approach makes it possible to
> continue
> using old code and modules without a hassle, while coming up with a
> migration
> plan for future python versions.

The clash would occur only in modules that use the __future__ import,
wouldn't it? And those modules are going to have to be fixed by
version X.Y anyway - the whole point of the __future__ import is to
detect that.

> Why magic methods start with "a"
> --------------------------------
>
> New asynchronous magic methods ``__aiter__``, ``__anext__``, ``__aenter__``,
> and ``__aexit__`` all start with the same prefix "a".  An alternative
> proposal
> is to use "async" prefix, so that ``__aiter__`` becomes ``__async_iter__``.
> However, to align new magic methods with the existing ones, such as
> ``__radd__`` and ``__iadd__`` it was decided to use a shorter version.

I support the short names, with the possible exception that
"__await__" now looks like "asynchronous wait", which may or may not
be desirable.

> Comprehensions
> --------------
>
> For the sake of restricting the broadness of this PEP there is no new syntax
> for asynchronous comprehensions.  This should be considered in a separate
> PEP,
> if there is a strong demand for this feature.

Likewise asynchronous lambda functions (or maybe I just missed it).


Overall comment: The changes appear to be infecting a lot of code.
Will this suddenly be something that everyone has to think about, or
can it still be ignored in projects that don't use it? There'll be a
few object types that would benefit from becoming async context
managers as well as regular context managers (I would expect, for
instance, that the built-in file type would allow its auto-closing to
be asynchronous), but if a third-party module completely ignores this
proposal, everything should continue to work, right?

I'm seeing the potential for a lot of code duplication here, but
possibly once things settle into real-world usage, a few easy
decorators will help out with that (like the total_ordering decorator
for handling __gt__, __le__, etc etc).

+1 on the proposal as a whole.

ChrisA