[Python-ideas] async/await in Python

[Victor Stinner]

Hi,

Yury Selivanov <yselivanov.ml at ...> writes:
> Here's my proposal to add async/await in Python.

As a contributor to asyncio, I'm a strong supporter of this PEP. IMO it
should land into Python 3.5 to confirm that Python promotes asynchronous
programming (and is the best language for async programming? :-)) and
accelerate the adoption of asyncio "everywhere".

The first time Yury shared with me his idea of new keywords (at Pycon
Montreal 2014), I understood that it was just syntax sugar. In fact, it adds
also "async for" and "async with" which adds new features. They are almost
required to make asyncio usage easier. "async for" would help database ORMs
or the aiofiles project (run file I/O in threads). "async with" helps also ORMs.

I already reviewed this PEP before Yury sent it to this list, so sorry, I
have no longer major issue to report :-)

> Debugging Features
> ------------------
> (...)
> With this proposal, coroutines is a native, distinct from generators,
> concept.  A new method ``set_async_wrapper`` is added to the ``sys`` module,

I suggest to rename it to set_coroutine_wrapper() to be more consistent with
the PEP. By the way, asyncio has a private "CoroWrapper" class (coroutine
wrapper). It is the class used in debug mode to emit warnings described in
this section.

I also suggest to add get_coroutine_wrapper(), similar to
sys.get/set_trace(). It is required for a framework to check if the wrapper
is correctly setup (and no concurrent framework installed its own wrapper?).

> List of functions and methods
> =============================
> 
> ================= ======================================= =================
> Method            Can contain                              Can't contain
> ================= ======================================= =================
> async def func    await, return value                      yield, yield from
> async def __a*__  await, return value                      yield, yield from
> def __a*__        return Future-like                       await
> def __await__     yield, yield from, return iterable       await
> generator         yield, yield from, return value          await
> ================= ======================================= =================

You may add asyncio coroutines, which are special (using yield is invalid),
for the comparison:

can contain: yield from, return value
can't contain: yield, await

> Backwards Compatibility
> -----------------------
> 
> The only backwards incompatible change is an extra argument ``is_async`` to
> ``FunctionDef`` AST node.  But since it is a documented fact that the 
> structure
> of AST nodes is an implementation detail and subject to change, this 
> should not
> be considered a serious issue.

The PEP breaks the backward compatibility in Python 3.7 when async and await
will become keywords. You may mention it.


> Why not a __future__ import
> ---------------------------
> 
> ``__future__`` imports are inconvenient and easy to forget to add. (...)

I understand that you describe a situation where "async def ..." would raise
a SyntaxError when the __future__ is missing.

I think that an *optional* __future__ would help projects to be prepared for
async & await keywords.

I mean, you can add "from __future__ import async" to get async & await as
keywords in the current file, as we did before fore "from __future__ import
with_statement".

Also, I'm not opposed to require "from __future__ import async" to get the
new feature. I don't expect async/await to popup in *all* files of a
project, so changes are limited. I would avoid border case issues in the
parser (issues described above: when async/await is used in parameters or
one a single line).

Nowadays, there are more and more linters. For example, in the OpenStack
project, an issue detected by static analyzers ("hacking" based on flake8)
must be fixed by the author of a patch, otherwise the patch cannot be
merged. It's just mandatory. So maybe __future__ is less important than it
was when with was added in Python 2.6?

> Why magic methods start with "a"
> --------------------------------
> 
> New asynchronous magic methods ``__aiter__``, ``__anext__``, ``__aenter__``,
> and ``__aexit__`` all start with the same prefix "a".  An alternative 
> proposal
> is to use "async" prefix, so that ``__aiter__`` becomes ``__async_iter__``.

I prefer __async_iter__ over __aiter__ (and __async_next/enter/exit__), but
I can survive to write __aiter__. __async_iter__ is more consistent with
"async" prefix added to functions (async def).


> However, to align new magic methods with the existing ones, such as
> ``__radd__`` and ``__iadd__`` it was decided to use a shorter version.

I don't think that past mistakes justify new mistakes :-D

Victor