[Python-checkins] peps: pep-0492: Trim lines at 72
yury.selivanov
python-checkins at python.org
Tue Apr 21 02:23:50 CEST 2015
https://hg.python.org/peps/rev/646dc160b8fe
changeset: 5779:646dc160b8fe
user: Yury Selivanov <yselivanov at sprymix.com>
date: Mon Apr 20 20:23:45 2015 -0400
summary:
pep-0492: Trim lines at 72
files:
pep-0492.txt | 639 ++++++++++++++++++++------------------
1 files changed, 338 insertions(+), 301 deletions(-)
diff --git a/pep-0492.txt b/pep-0492.txt
--- a/pep-0492.txt
+++ b/pep-0492.txt
@@ -14,58 +14,59 @@
========
This PEP introduces new syntax for coroutines, asynchronous ``with``
-statements and ``for`` loops. The main motivation behind this proposal is to
-streamline writing and maintaining asynchronous code, as well as to simplify
-previously hard to implement code patterns.
+statements and ``for`` loops. The main motivation behind this proposal
+is to streamline writing and maintaining asynchronous code, as well as
+to simplify previously hard to implement code patterns.
Rationale and Goals
===================
-Current Python supports implementing coroutines via generators (PEP 342),
-further enhanced by the ``yield from`` syntax introduced in PEP 380.
-This approach has a number of shortcomings:
+Current Python supports implementing coroutines via generators (PEP
+342), further enhanced by the ``yield from`` syntax introduced in PEP
+380. This approach has a number of shortcomings:
-* it is easy to confuse coroutines with regular generators, since they share
- the same syntax; async libraries often attempt to alleviate this by using
- decorators (e.g. ``@asyncio.coroutine`` [1]_);
+* it is easy to confuse coroutines with regular generators, since they
+ share the same syntax; async libraries often attempt to alleviate
+ this by using decorators (e.g. ``@asyncio.coroutine`` [1]_);
-* it is not possible to natively define a coroutine which has no ``yield``
- or ``yield from`` statements, again requiring the use of decorators to
- fix potential refactoring issues;
+* it is not possible to natively define a coroutine which has no
+ ``yield`` or ``yield from`` statements, again requiring the use of
+ decorators to fix potential refactoring issues;
-* support for asynchronous calls is limited to expressions where ``yield`` is
- allowed syntactically, limiting the usefulness of syntactic features, such
- as ``with`` and ``for`` statements.
+* support for asynchronous calls is limited to expressions where
+ ``yield`` is allowed syntactically, limiting the usefulness of
+ syntactic features, such as ``with`` and ``for`` statements.
-This proposal makes coroutines a native Python language feature, and clearly
-separates them from generators. This removes generator/coroutine ambiguity,
-and makes it possible to reliably define coroutines without reliance on a
-specific library. This also enables linters and IDEs to improve static code
-analysis and refactoring.
+This proposal makes coroutines a native Python language feature, and
+clearly separates them from generators. This removes
+generator/coroutine ambiguity, and makes it possible to reliably define
+coroutines without reliance on a specific library. This also enables
+linters and IDEs to improve static code analysis and refactoring.
-Native coroutines and the associated new syntax features make it possible
-to define context manager and iteration protocols in asynchronous terms.
-As shown later in this proposal, the new ``async with`` statement lets Python
-programs perform asynchronous calls when entering and exiting a runtime
-context, and the new ``async for`` statement makes it possible to perform
-asynchronous calls in iterators.
+Native coroutines and the associated new syntax features make it
+possible to define context manager and iteration protocols in
+asynchronous terms. As shown later in this proposal, the new ``async
+with`` statement lets Python programs perform asynchronous calls when
+entering and exiting a runtime context, and the new ``async for``
+statement makes it possible to perform asynchronous calls in iterators.
Specification
=============
-This proposal introduces new syntax and semantics to enhance coroutine support
-in Python, it does not change the internal implementation of coroutines, which
-are still based on generators.
+This proposal introduces new syntax and semantics to enhance coroutine
+support in Python, it does not change the internal implementation of
+coroutines, which are still based on generators.
It is strongly suggested that the reader understands how coroutines are
-implemented in Python (PEP 342 and PEP 380). It is also recommended to read
-PEP 3156 (asyncio framework) and PEP 3152 (Cofunctions).
+implemented in Python (PEP 342 and PEP 380). It is also recommended to
+read PEP 3156 (asyncio framework) and PEP 3152 (Cofunctions).
-From this point in this document we use the word *coroutine* to refer to
-functions declared using the new syntax. *generator-based coroutine* is used
-where necessary to refer to coroutines that are based on generator syntax.
+From this point in this document we use the word *coroutine* to refer
+to functions declared using the new syntax. *generator-based
+coroutine* is used where necessary to refer to coroutines that are
+based on generator syntax.
New Coroutine Declaration Syntax
@@ -78,30 +79,31 @@
Key properties of coroutines:
-* ``async def`` functions are always coroutines, even if they do not contain
- ``await`` expressions.
+* ``async def`` functions are always coroutines, even if they do not
+ contain ``await`` expressions.
-* It is a ``SyntaxError`` to have ``yield`` or ``yield from`` expressions in
- an ``async`` function.
+* It is a ``SyntaxError`` to have ``yield`` or ``yield from``
+ expressions in an ``async`` function.
-* Internally, a new code object flag - ``CO_COROUTINE`` - is introduced to
- enable runtime detection of coroutines (and migrating existing code). All
- coroutines have both ``CO_COROUTINE`` and ``CO_GENERATOR`` flags set.
+* Internally, a new code object flag - ``CO_COROUTINE`` - is introduced
+ to enable runtime detection of coroutines (and migrating existing
+ code). All coroutines have both ``CO_COROUTINE`` and ``CO_GENERATOR``
+ flags set.
-* Regular generators, when called, return a *generator object*; similarly,
- coroutines return a *coroutine object*.
+* Regular generators, when called, return a *generator object*;
+ similarly, coroutines return a *coroutine object*.
-* ``StopIteration`` exceptions are not propagated out of coroutines, and are
- replaced with a ``RuntimeError``. For regular generators such behavior
- requires a future import (see PEP 479).
+* ``StopIteration`` exceptions are not propagated out of coroutines,
+ and are replaced with a ``RuntimeError``. For regular generators
+ such behavior requires a future import (see PEP 479).
types.coroutine()
-----------------
-A new function ``coroutine(gen)`` is added to the ``types`` module. It applies
-``CO_COROUTINE`` flag to the passed generator-function's code object,
-making it to return a *coroutine object* when called.
+A new function ``coroutine(gen)`` is added to the ``types`` module. It
+applies ``CO_COROUTINE`` flag to the passed generator-function's code
+object, making it to return a *coroutine object* when called.
This feature enables an easy upgrade path for existing libraries.
@@ -109,42 +111,45 @@
Await Expression
----------------
-The following new ``await`` expression is used to obtain a result of coroutine
-execution::
+The following new ``await`` expression is used to obtain a result of
+coroutine execution::
async def read_data(db):
data = await db.fetch('SELECT ...')
...
-``await``, similarly to ``yield from``, suspends execution of ``read_data``
-coroutine until ``db.fetch`` *awaitable* completes and returns the result
-data.
+``await``, similarly to ``yield from``, suspends execution of
+``read_data`` coroutine until ``db.fetch`` *awaitable* completes and
+returns the result data.
-It uses the ``yield from`` implementation with an extra step of validating its
-argument. ``await`` only accepts an *awaitable*, which can be one of:
+It uses the ``yield from`` implementation with an extra step of
+validating its argument. ``await`` only accepts an *awaitable*, which
+can be one of:
-* A *coroutine object* returned from a *coroutine* or a generator decorated
- with ``types.coroutine()``.
+* A *coroutine object* returned from a *coroutine* or a generator
+ decorated with ``types.coroutine()``.
* An object with an ``__await__`` method returning an iterator.
Any ``yield from`` chain of calls ends with a ``yield``. This is a
- fundamental mechanism of how *Futures* are implemented. Since, internally,
- coroutines are a special kind of generators, every ``await`` is suspended by
- a ``yield`` somewhere down the chain of ``await`` calls (please refer to PEP
- 3156 for a detailed explanation.)
+ fundamental mechanism of how *Futures* are implemented. Since,
+ internally, coroutines are a special kind of generators, every
+ ``await`` is suspended by a ``yield`` somewhere down the chain of
+ ``await`` calls (please refer to PEP 3156 for a detailed
+ explanation.)
To enable this behavior for coroutines, a new magic method called
- ``__await__`` is added. In asyncio, for instance, to enable Future objects
- in ``await`` statements, the only change is to add ``__await__ = __iter__``
- line to ``asyncio.Future`` class.
+ ``__await__`` is added. In asyncio, for instance, to enable Future
+ objects in ``await`` statements, the only change is to add
+ ``__await__ = __iter__`` line to ``asyncio.Future`` class.
- Objects with ``__await__`` method are called *Future-like* objects in the
- rest of this PEP.
+ Objects with ``__await__`` method are called *Future-like* objects in
+ the rest of this PEP.
- Also, please note that ``__aiter__`` method (see its definition below) cannot
- be used for this purpose. It is a different protocol, and would be like
- using ``__iter__`` instead of ``__call__`` for regular callables.
+ Also, please note that ``__aiter__`` method (see its definition
+ below) cannot be used for this purpose. It is a different protocol,
+ and would be like using ``__iter__`` instead of ``__call__`` for
+ regular callables.
It is a ``SyntaxError`` to use ``await`` outside of a coroutine.
@@ -152,12 +157,12 @@
Asynchronous Context Managers and "async with"
----------------------------------------------
-An *asynchronous context manager* is a context manager that is able to suspend
-execution in its *enter* and *exit* methods.
+An *asynchronous context manager* is a context manager that is able to
+suspend execution in its *enter* and *exit* methods.
-To make this possible, a new protocol for asynchronous context managers is
-proposed. Two new magic methods are added: ``__aenter__`` and ``__aexit__``.
-Both must return an *awaitable*.
+To make this possible, a new protocol for asynchronous context managers
+is proposed. Two new magic methods are added: ``__aenter__`` and
+``__aexit__``. Both must return an *awaitable*.
An example of an asynchronous context manager::
@@ -200,19 +205,19 @@
await aexit(mgr, None, None, None)
-As with regular ``with`` statements, it is possible to specify multiple context
-managers in a single ``async with`` statement.
+As with regular ``with`` statements, it is possible to specify multiple
+context managers in a single ``async with`` statement.
-It is an error to pass a regular context manager without ``__aenter__`` and
-``__aexit__`` methods to ``async with``. It is a ``SyntaxError`` to use
-``async with`` outside of a coroutine.
+It is an error to pass a regular context manager without ``__aenter__``
+and ``__aexit__`` methods to ``async with``. It is a ``SyntaxError``
+to use ``async with`` outside of a coroutine.
Example
'''''''
-With asynchronous context managers it is easy to implement proper database
-transaction managers for coroutines::
+With asynchronous context managers it is easy to implement proper
+database transaction managers for coroutines::
async def commit(session, data):
...
@@ -236,15 +241,16 @@
Asynchronous Iterators and "async for"
--------------------------------------
-An *asynchronous iterable* is able to call asynchronous code in its *iter*
-implementation, and *asynchronous iterator* can call asynchronous code in its
-*next* method. To support asynchronous iteration:
+An *asynchronous iterable* is able to call asynchronous code in its
+*iter* implementation, and *asynchronous iterator* can call
+asynchronous code in its *next* method. To support asynchronous
+iteration:
-1. An object must implement an ``__aiter__`` method returning an *awaitable*
- resulting in an *asynchronous iterator object*.
+1. An object must implement an ``__aiter__`` method returning an
+ *awaitable* resulting in an *asynchronous iterator object*.
-2. An *asynchronous iterator object* must implement an ``__anext__`` method
- returning an *awaitable*.
+2. An *asynchronous iterator object* must implement an ``__anext__``
+ method returning an *awaitable*.
3. To stop iteration ``__anext__`` must raise a ``StopAsyncIteration``
exception.
@@ -269,7 +275,8 @@
New Syntax
''''''''''
-A new statement for iterating through asynchronous iterators is proposed::
+A new statement for iterating through asynchronous iterators is
+proposed::
async for TARGET in ITER:
BLOCK
@@ -292,19 +299,19 @@
BLOCK2
-It is an error to pass a regular iterable without ``__aiter__`` method to
-``async for``. It is a ``SyntaxError`` to use ``async for`` outside of a
-coroutine.
+It is an error to pass a regular iterable without ``__aiter__`` method
+to ``async for``. It is a ``SyntaxError`` to use ``async for`` outside
+of a coroutine.
-As for with regular ``for`` statement, ``async for`` has an optional ``else``
-clause.
+As for with regular ``for`` statement, ``async for`` has an optional
+``else`` clause.
Example 1
'''''''''
-With asynchronous iteration protocol it is possible to asynchronously buffer
-data during iteration::
+With asynchronous iteration protocol it is possible to asynchronously
+buffer data during iteration::
async for data in cursor:
...
@@ -351,9 +358,10 @@
Example 2
'''''''''
-The following is a utility class that transforms a regular iterable to an
-asynchronous one. While this is not a very useful thing to do, the code
-illustrates the relationship between regular and asynchronous iterators.
+The following is a utility class that transforms a regular iterable to
+an asynchronous one. While this is not a very useful thing to do, the
+code illustrates the relationship between regular and asynchronous
+iterators.
::
@@ -378,8 +386,8 @@
Why StopAsyncIteration?
'''''''''''''''''''''''
-Coroutines are still based on generators internally. So, before PEP 479, there
-was no fundamental difference between
+Coroutines are still based on generators internally. So, before PEP
+479, there was no fundamental difference between
::
@@ -395,8 +403,8 @@
yield from fut
raise StopIteration('spam')
-And since PEP 479 is accepted and enabled by default for coroutines, the
-following example will have its ``StopIteration`` wrapped into a
+And since PEP 479 is accepted and enabled by default for coroutines,
+the following example will have its ``StopIteration`` wrapped into a
``RuntimeError``
::
@@ -405,46 +413,48 @@
await fut
raise StopIteration('spam')
-The only way to tell the outside code that the iteration has ended is to raise
-something other than ``StopIteration``. Therefore, a new built-in exception
-class ``StopAsyncIteration`` was added.
+The only way to tell the outside code that the iteration has ended is
+to raise something other than ``StopIteration``. Therefore, a new
+built-in exception class ``StopAsyncIteration`` was added.
-Moreover, with semantics from PEP 479, all ``StopIteration`` exceptions raised
-in coroutines are wrapped in ``RuntimeError``.
+Moreover, with semantics from PEP 479, all ``StopIteration`` exceptions
+raised in coroutines are wrapped in ``RuntimeError``.
Debugging Features
------------------
-One of the most frequent mistakes that people make when using generators as
-coroutines is forgetting to use ``yield from``::
+One of the most frequent mistakes that people make when using
+generators as coroutines is forgetting to use ``yield from``::
@asyncio.coroutine
def useful():
asyncio.sleep(1) # this will do noting without 'yield from'
-For debugging this kind of mistakes there is a special debug mode in asyncio,
-in which ``@coroutine`` decorator wraps all functions with a special object
-with a destructor logging a warning. Whenever a wrapped generator gets garbage
-collected, a detailed logging message is generated with information about where
-exactly the decorator function was defined, stack trace of where it was
-collected, etc. Wrapper object also provides a convenient ``__repr__``
-function with detailed information about the generator.
+For debugging this kind of mistakes there is a special debug mode in
+asyncio, in which ``@coroutine`` decorator wraps all functions with a
+special object with a destructor logging a warning. Whenever a wrapped
+generator gets garbage collected, a detailed logging message is
+generated with information about where exactly the decorator function
+was defined, stack trace of where it was collected, etc. Wrapper
+object also provides a convenient ``__repr__`` function with detailed
+information about the generator.
-The only problem is how to enable these debug capabilities. Since debug
-facilities should be a no-op in production mode, ``@coroutine`` decorator makes
-the decision of whether to wrap or not to wrap based on an OS environment
-variable ``PYTHONASYNCIODEBUG``. This way it is possible to run asyncio
-programs with asyncio's own functions instrumented. ``EventLoop.set_debug``, a
-different debug facility, has no impact on ``@coroutine`` decorator's behavior.
+The only problem is how to enable these debug capabilities. Since
+debug facilities should be a no-op in production mode, ``@coroutine``
+decorator makes the decision of whether to wrap or not to wrap based on
+an OS environment variable ``PYTHONASYNCIODEBUG``. This way it is
+possible to run asyncio programs with asyncio's own functions
+instrumented. ``EventLoop.set_debug``, a different debug facility, has
+no impact on ``@coroutine`` decorator's behavior.
-With this proposal, coroutines is a native, distinct from generators, concept.
-New methods ``set_coroutine_wrapper`` and ``get_coroutine_wrapper`` are added
-to the ``sys`` module, with which frameworks can provide advanced debugging
-facilities.
+With this proposal, coroutines is a native, distinct from generators,
+concept. New methods ``set_coroutine_wrapper`` and
+``get_coroutine_wrapper`` are added to the ``sys`` module, with which
+frameworks can provide advanced debugging facilities.
-It is also important to make coroutines as fast and efficient as possible,
-therefore there are no debug features enabled by default.
+It is also important to make coroutines as fast and efficient as
+possible, therefore there are no debug features enabled by default.
Example::
@@ -465,44 +475,47 @@
# previously set wrapper
assert not isinstance(debug_me(), AsyncDebugWrapper)
-If ``sys.set_coroutine_wrapper()`` is called twice, the new wrapper replaces
-the previous wrapper. ``sys.set_coroutine_wrapper(None)`` unsets the wrapper.
+If ``sys.set_coroutine_wrapper()`` is called twice, the new wrapper
+replaces the previous wrapper. ``sys.set_coroutine_wrapper(None)``
+unsets the wrapper.
Glossary
========
:Coroutine:
- A coroutine function, or just "coroutine", is declared with ``async def``.
- It uses ``await`` and ``return value``; see `New Coroutine Declaration
- Syntax`_ for details.
+ A coroutine function, or just "coroutine", is declared with ``async
+ def``. It uses ``await`` and ``return value``; see `New Coroutine
+ Declaration Syntax`_ for details.
:Coroutine object:
- Returned from a coroutine function. See `Await Expression`_ for details.
+ Returned from a coroutine function. See `Await Expression`_ for
+ details.
:Future-like object:
- An object with an ``__await__`` method. Can be consumed by an ``await``
- expression in a coroutine. A coroutine waiting for a Future-like object is
- suspended until the Future-like object's ``__await__`` completes, and
- returns the result. See `Await Expression`_ for details.
+ An object with an ``__await__`` method. Can be consumed by an
+ ``await`` expression in a coroutine. A coroutine waiting for a
+ Future-like object is suspended until the Future-like object's
+ ``__await__`` completes, and returns the result. See `Await
+ Expression`_ for details.
:Awaitable:
- A *Future-like* object or a *coroutine object*. See `Await Expression`_
- for details.
+ A *Future-like* object or a *coroutine object*. See `Await
+ Expression`_ for details.
:Generator-based coroutine:
Coroutines based in generator syntax. Most common example is
``@asyncio.coroutine``.
:Asynchronous context manager:
- An asynchronous context manager has ``__aenter__`` and ``__aexit__`` methods
- and can be used with ``async with``. See
- `Asynchronous Context Managers and "async with"`_ for details.
+ An asynchronous context manager has ``__aenter__`` and ``__aexit__``
+ methods and can be used with ``async with``. See `Asynchronous
+ Context Managers and "async with"`_ for details.
:Asynchronous iterable:
- An object with an ``__aiter__`` method, which must return an *asynchronous
- iterator* object. Can be used with ``async for``. See
- `Asynchronous Iterators and "async for"`_ for details.
+ An object with an ``__aiter__`` method, which must return an
+ *asynchronous iterator* object. Can be used with ``async for``.
+ See `Asynchronous Iterators and "async for"`_ for details.
:Asynchronous iterator:
An asynchronous iterator has an ``__anext__`` method. See
@@ -512,15 +525,15 @@
List of functions and methods
=============================
-================= ======================================= =================
-Method Can contain Can't contain
-================= ======================================= =================
-async def func await, return value yield, yield from
-async def __a*__ await, return value yield, yield from
-def __a*__ return awaitable await
-def __await__ yield, yield from, return iterable await
-generator yield, yield from, return value await
-================= ======================================= =================
+================= =================================== =================
+Method Can contain Can't contain
+================= =================================== =================
+async def func await, return value yield, yield from
+async def __a*__ await, return value yield, yield from
+def __a*__ return awaitable await
+def __await__ yield, yield from, return iterable await
+generator yield, yield from, return value await
+================= =================================== =================
Where:
@@ -529,32 +542,38 @@
* "async def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``,
``__aexit__`` defined with the ``async`` keyword;
-* "def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``, ``__aexit__``
- defined without the ``async`` keyword, must return an *awaitable*;
+* "def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``,
+ ``__aexit__`` defined without the ``async`` keyword, must return an
+ *awaitable*;
-* "def __await__": ``__await__`` method to implement *Future-like* objects;
+* "def __await__": ``__await__`` method to implement *Future-like*
+ objects;
-* generator: a "regular" generator, function defined with ``def`` and which
- contains a least one ``yield`` or ``yield from`` expression.
+* generator: a "regular" generator, function defined with ``def`` and
+ which contains a least one ``yield`` or ``yield from`` expression.
Transition Plan
===============
-To avoid backwards compatibility issues with ``async`` and ``await`` keywords,
-it was decided to modify ``tokenizer.c`` in such a way, that it:
+To avoid backwards compatibility issues with ``async`` and ``await``
+keywords, it was decided to modify ``tokenizer.c`` in such a way, that
+it:
-* recognizes ``async def`` name tokens combination (start of a coroutine);
+* recognizes ``async def`` name tokens combination (start of a
+ coroutine);
* keeps track of regular functions and coroutines;
* replaces ``'async'`` token with ``ASYNC`` and ``'await'`` token with
``AWAIT`` when in the process of yielding tokens for coroutines.
-This approach allows for seamless combination of new syntax features (all of
-them available only in ``async`` functions) with any existing code.
+This approach allows for seamless combination of new syntax features
+(all of them available only in ``async`` functions) with any existing
+code.
-An example of having "async def" and "async" attribute in one piece of code::
+An example of having "async def" and "async" attribute in one piece of
+code::
class Spam:
async = 42
@@ -584,16 +603,18 @@
async_stmt: ASYNC (funcdef | with_stmt | for_stmt)
- compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt | with_stmt
- | funcdef | classdef | decorated | async_stmt)
+ compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt |
+ with_stmt | funcdef | classdef | decorated |
+ async_stmt)
atom: ('(' [yield_expr|await_expr|testlist_comp] ')' |
'[' [testlist_comp] ']' |
'{' [dictorsetmaker] '}' |
NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False’)
- expr_stmt: testlist_star_expr (augassign (yield_expr|await_expr|testlist) |
- ('=' (yield_expr|await_expr|testlist_star_expr))*)
+ expr_stmt: testlist_star_expr
+ (augassign (yield_expr|await_expr|testlist) |
+ ('=' (yield_expr|await_expr|testlist_star_expr))*)
Transition Period Shortcomings
@@ -601,9 +622,9 @@
There is just one.
-Until ``async`` and ``await`` are not proper keywords, it is not possible (or
-at least very hard) to fix ``tokenizer.c`` to recognize them on the **same
-line** with ``def`` keyword::
+Until ``async`` and ``await`` are not proper keywords, it is not
+possible (or at least very hard) to fix ``tokenizer.c`` to recognize
+them on the **same line** with ``def`` keyword::
# async and await will always be parsed as variables
@@ -613,11 +634,11 @@
async def foo(): return (await fut) # 2
-Since ``await`` and ``async`` in such cases are parsed as ``NAME`` tokens, a
-``SyntaxError`` will be raised.
+Since ``await`` and ``async`` in such cases are parsed as ``NAME``
+tokens, a ``SyntaxError`` will be raised.
-To workaround these issues, the above examples can be easily rewritten to a
-more readable form::
+To workaround these issues, the above examples can be easily rewritten
+to a more readable form::
async def outer(): # 1
a_default = await fut
@@ -627,34 +648,35 @@
async def foo(): # 2
return (await fut)
-This limitation will go away as soon as ``async`` and ``await`` ate proper
-keywords. Or if it's decided to use a future import for this PEP.
+This limitation will go away as soon as ``async`` and ``await`` ate
+proper keywords. Or if it's decided to use a future import for this
+PEP.
Deprecation Plans
-----------------
-``async`` and ``await`` names will be softly deprecated in CPython 3.5 and 3.6.
-In 3.7 we will transform them to proper keywords. Making ``async`` and
-``await`` proper keywords before 3.7 might make it harder for people to port
-their code to Python 3.
+``async`` and ``await`` names will be softly deprecated in CPython 3.5
+and 3.6. In 3.7 we will transform them to proper keywords. Making
+``async`` and ``await`` proper keywords before 3.7 might make it harder
+for people to port their code to Python 3.
asyncio
-------
-``asyncio`` module was adapted and tested to work with coroutines and new
-statements. Backwards compatibility is 100% preserved.
+``asyncio`` module was adapted and tested to work with coroutines and
+new statements. Backwards compatibility is 100% preserved.
The required changes are mainly:
-1. Modify ``@asyncio.coroutine`` decorator to use new ``types.coroutine()``
- function.
+1. Modify ``@asyncio.coroutine`` decorator to use new
+ ``types.coroutine()`` function.
2. Add ``__await__ = __iter__`` line to ``asyncio.Future`` class.
-3. Add ``ensure_task()`` as an alias for ``async()`` function. Deprecate
- ``async()`` function.
+3. Add ``ensure_task()`` as an alias for ``async()`` function.
+ Deprecate ``async()`` function.
Design Considerations
@@ -666,15 +688,16 @@
PEP 3152 by Gregory Ewing proposes a different mechanism for coroutines
(called "cofunctions"). Some key points:
-1. A new keyword ``codef`` to declare a *cofunction*. *Cofunction* is always a
- generator, even if there is no ``cocall`` expressions inside it. Maps to
- ``async def`` in this proposal.
+1. A new keyword ``codef`` to declare a *cofunction*. *Cofunction* is
+ always a generator, even if there is no ``cocall`` expressions
+ inside it. Maps to ``async def`` in this proposal.
-2. A new keyword ``cocall`` to call a *cofunction*. Can only be used inside a
- *cofunction*. Maps to ``await`` in this proposal (with some differences,
- see below.)
+2. A new keyword ``cocall`` to call a *cofunction*. Can only be used
+ inside a *cofunction*. Maps to ``await`` in this proposal (with
+ some differences, see below.)
-3. It is not possible to call a *cofunction* without a ``cocall`` keyword.
+3. It is not possible to call a *cofunction* without a ``cocall``
+ keyword.
4. ``cocall`` grammatically requires parentheses after it::
@@ -687,54 +710,59 @@
Differences from this proposal:
-1. There is no equivalent of ``__cocall__`` in this PEP, which is called and
- its result is passed to ``yield from`` in the ``cocall`` expression.
- ``await`` keyword expects an *awaitable* object, validates the type, and
- executes ``yield from`` on it. Although, ``__await__`` method is similar to
- ``__cocall__``, but is only used to define *Future-like* objects.
+1. There is no equivalent of ``__cocall__`` in this PEP, which is
+ called and its result is passed to ``yield from`` in the ``cocall``
+ expression. ``await`` keyword expects an *awaitable* object,
+ validates the type, and executes ``yield from`` on it. Although,
+ ``__await__`` method is similar to ``__cocall__``, but is only used
+ to define *Future-like* objects.
-2. ``await`` is defined in almost the same way as ``yield from`` in the grammar
- (it is later enforced that ``await`` can only be inside ``async def``). It
- is possible to simply write ``await future``, whereas ``cocall`` always
- requires parentheses.
+2. ``await`` is defined in almost the same way as ``yield from`` in the
+ grammar (it is later enforced that ``await`` can only be inside
+ ``async def``). It is possible to simply write ``await future``,
+ whereas ``cocall`` always requires parentheses.
3. To make asyncio work with PEP 3152 it would be required to modify
- ``@asyncio.coroutine`` decorator to wrap all functions in an object with a
- ``__cocall__`` method. To call *cofunctions* from existing generator-based
- coroutines it would be required to use ``costart`` built-in. In this
- proposal ``@asyncio.coroutine`` simply sets ``CO_COROUTINE`` on the wrapped
- function's code object and everything works automatically.
+ ``@asyncio.coroutine`` decorator to wrap all functions in an object
+ with a ``__cocall__`` method. To call *cofunctions* from existing
+ generator-based coroutines it would be required to use ``costart``
+ built-in. In this proposal ``@asyncio.coroutine`` simply sets
+ ``CO_COROUTINE`` on the wrapped function's code object and
+ everything works automatically.
-4. Since it is impossible to call a *cofunction* without a ``cocall`` keyword,
- it automatically prevents the common mistake of forgetting to use
- ``yield from`` on generator-based coroutines. This proposal addresses
- this problem with a different approach, see `Debugging Features`_.
+4. Since it is impossible to call a *cofunction* without a ``cocall``
+ keyword, it automatically prevents the common mistake of forgetting
+ to use ``yield from`` on generator-based coroutines. This proposal
+ addresses this problem with a different approach, see `Debugging
+ Features`_.
-5. There are no equivalents of ``async for`` and ``async with`` in PEP 3152.
+5. There are no equivalents of ``async for`` and ``async with`` in PEP
+ 3152.
No implicit wrapping in Futures
-------------------------------
-There is a proposal to add similar mechanism to ECMAScript 7 [2]_. A key
-difference is that JavaScript "async functions" always return a Promise. While
-this approach has some advantages, it also implies that a new Promise object is
-created on each "async function" invocation.
+There is a proposal to add similar mechanism to ECMAScript 7 [2]_. A
+key difference is that JavaScript "async functions" always return a
+Promise. While this approach has some advantages, it also implies that
+a new Promise object is created on each "async function" invocation.
We could implement a similar functionality in Python, by wrapping all
-coroutines in a Future object, but this has the following disadvantages:
+coroutines in a Future object, but this has the following
+disadvantages:
-1. Performance. A new Future object would be instantiated on each coroutine
- call. Moreover, this makes implementation of ``await`` expressions slower
- (disabling optimizations of ``yield from``).
+1. Performance. A new Future object would be instantiated on each
+ coroutine call. Moreover, this makes implementation of ``await``
+ expressions slower (disabling optimizations of ``yield from``).
2. A new built-in ``Future`` object would need to be added.
-3. Coming up with a generic ``Future`` interface that is usable for any use
- case in any framework is a very hard to solve problem.
+3. Coming up with a generic ``Future`` interface that is usable for any
+ use case in any framework is a very hard to solve problem.
-4. It is not a feature that is used frequently, when most of the code is
- coroutines.
+4. It is not a feature that is used frequently, when most of the code
+ is coroutines.
Why "async" and "await" keywords
@@ -757,31 +785,32 @@
* and many other less popular languages.
-This is a huge benefit, as some users already have experience with async/await,
-and because it makes working with many languages in one project easier (Python
-with ECMAScript 7 for instance).
+This is a huge benefit, as some users already have experience with
+async/await, and because it makes working with many languages in one
+project easier (Python with ECMAScript 7 for instance).
Why "__aiter__" is a coroutine
------------------------------
-In principle, ``__aiter__`` could be a regular function. There are several
-good reasons to make it a coroutine:
+In principle, ``__aiter__`` could be a regular function. There are
+several good reasons to make it a coroutine:
-* as most of the ``__anext__``, ``__aenter__``, and ``__aexit__`` methods are
- coroutines, users would often make a mistake defining it as ``async``
- anyways;
+* as most of the ``__anext__``, ``__aenter__``, and ``__aexit__``
+ methods are coroutines, users would often make a mistake defining it
+ as ``async`` anyways;
-* there might be a need to run some asynchronous operations in ``__aiter__``,
- for instance to prepare DB queries or do some file operation.
+* there might be a need to run some asynchronous operations in
+ ``__aiter__``, for instance to prepare DB queries or do some file
+ operation.
Importance of "async" keyword
-----------------------------
-While it is possible to just implement ``await`` expression and treat all
-functions with at least one ``await`` as coroutines, this approach makes
-APIs design, code refactoring and its long time support harder.
+While it is possible to just implement ``await`` expression and treat
+all functions with at least one ``await`` as coroutines, this approach
+makes APIs design, code refactoring and its long time support harder.
Let's pretend that Python only has ``await`` keyword::
@@ -793,50 +822,54 @@
def important():
await useful()
-If ``useful()`` function is refactored and someone removes all ``await``
-expressions from it, it would become a regular python function, and all code
-that depends on it, including ``important()`` would be broken. To mitigate
-this issue a decorator similar to ``@asyncio.coroutine`` has to be introduced.
+If ``useful()`` function is refactored and someone removes all
+``await`` expressions from it, it would become a regular python
+function, and all code that depends on it, including ``important()``
+would be broken. To mitigate this issue a decorator similar to
+``@asyncio.coroutine`` has to be introduced.
Why "async def"
---------------
-For some people bare ``async name(): pass`` syntax might look more appealing
-than ``async def name(): pass``. It is certainly easier to type. But on the
-other hand, it breaks the symmetry between ``async def``, ``async with`` and
-``async for``, where ``async`` is a modifier, stating that the statement is
-asynchronous. It is also more consistent with the existing grammar.
+For some people bare ``async name(): pass`` syntax might look more
+appealing than ``async def name(): pass``. It is certainly easier to
+type. But on the other hand, it breaks the symmetry between ``async
+def``, ``async with`` and ``async for``, where ``async`` is a modifier,
+stating that the statement is asynchronous. It is also more consistent
+with the existing grammar.
Why not a __future__ import
---------------------------
-``__future__`` imports are inconvenient and easy to forget to add. Also, they
-are enabled for the whole source file. Consider that there is a big project
-with a popular module named "async.py". With future imports it is required to
-either import it using ``__import__()`` or ``importlib.import_module()`` calls,
-or to rename the module. The proposed approach makes it possible to continue
-using old code and modules without a hassle, while coming up with a migration
-plan for future python versions.
+``__future__`` imports are inconvenient and easy to forget to add.
+Also, they are enabled for the whole source file. Consider that there
+is a big project with a popular module named "async.py". With future
+imports it is required to either import it using ``__import__()`` or
+``importlib.import_module()`` calls, or to rename the module. The
+proposed approach makes it possible to continue using old code and
+modules without a hassle, while coming up with a migration plan for
+future python versions.
Why magic methods start with "a"
--------------------------------
-New asynchronous magic methods ``__aiter__``, ``__anext__``, ``__aenter__``,
-and ``__aexit__`` all start with the same prefix "a". An alternative proposal
-is to use "async" prefix, so that ``__aiter__`` becomes ``__async_iter__``.
-However, to align new magic methods with the existing ones, such as
-``__radd__`` and ``__iadd__`` it was decided to use a shorter version.
+New asynchronous magic methods ``__aiter__``, ``__anext__``,
+``__aenter__``, and ``__aexit__`` all start with the same prefix "a".
+An alternative proposal is to use "async" prefix, so that ``__aiter__``
+becomes ``__async_iter__``. However, to align new magic methods with
+the existing ones, such as ``__radd__`` and ``__iadd__`` it was decided
+to use a shorter version.
Why not reuse existing magic names
----------------------------------
-An alternative idea about new asynchronous iterators and context managers was
-to reuse existing magic methods, by adding an ``async`` keyword to their
-declarations::
+An alternative idea about new asynchronous iterators and context
+managers was to reuse existing magic methods, by adding an ``async``
+keyword to their declarations::
class CM:
async def __enter__(self): # instead of __aenter__
@@ -844,31 +877,31 @@
This approach has the following downsides:
-* it would not be possible to create an object that works in both ``with`` and
- ``async with`` statements;
+* it would not be possible to create an object that works in both
+ ``with`` and ``async with`` statements;
-* it would look confusing and would require some implicit magic behind the
- scenes in the interpreter;
+* it would look confusing and would require some implicit magic behind
+ the scenes in the interpreter;
-* one of the main points of this proposal is to make coroutines as simple
- and foolproof as possible.
+* one of the main points of this proposal is to make coroutines as
+ simple and foolproof as possible.
Comprehensions
--------------
-For the sake of restricting the broadness of this PEP there is no new syntax
-for asynchronous comprehensions. This should be considered in a separate PEP,
-if there is a strong demand for this feature.
+For the sake of restricting the broadness of this PEP there is no new
+syntax for asynchronous comprehensions. This should be considered in a
+separate PEP, if there is a strong demand for this feature.
Async lambdas
-------------
-Lambda coroutines are not part of this proposal. In this proposal they would
-look like ``async lambda(parameters): expression``. Unless there is a strong
-demand to have them as part of this proposal, it is recommended to consider
-them later in a separate PEP.
+Lambda coroutines are not part of this proposal. In this proposal they
+would look like ``async lambda(parameters): expression``. Unless there
+is a strong demand to have them as part of this proposal, it is
+recommended to consider them later in a separate PEP.
Performance
@@ -877,8 +910,8 @@
Overall Impact
--------------
-This proposal introduces no observable performance impact. Here is an output
-of python's official set of benchmarks [4]_:
+This proposal introduces no observable performance impact. Here is an
+output of python's official set of benchmarks [4]_:
::
@@ -906,16 +939,17 @@
Tokenizer modifications
-----------------------
-There is no observable slowdown of parsing python files with the modified
-tokenizer: parsing of one 12Mb file (``Lib/test/test_binop.py`` repeated 1000
-times) takes the same amount of time.
+There is no observable slowdown of parsing python files with the
+modified tokenizer: parsing of one 12Mb file
+(``Lib/test/test_binop.py`` repeated 1000 times) takes the same amount
+of time.
async/await
-----------
-The following micro-benchmark was used to determine performance difference
-between "async" functions and generators::
+The following micro-benchmark was used to determine performance
+difference between "async" functions and generators::
import sys
import time
@@ -942,8 +976,8 @@
print('{}({}) * {}: total {:.3f}s'.format(
gen.__name__, depth, repeat, t1-t0))
-The result is that there is no observable performance difference. Minimum
-timing of 3 runs
+The result is that there is no observable performance difference.
+Minimum timing of 3 runs
::
@@ -969,28 +1003,30 @@
3. New syntax for asynchronous context managers: ``async with``. And
associated protocol with ``__aenter__`` and ``__aexit__`` methods.
-4. New syntax for asynchronous iteration: ``async for``. And associated
- protocol with ``__aiter__``, ``__aexit__`` and new built-in exception
- ``StopAsyncIteration``.
+4. New syntax for asynchronous iteration: ``async for``. And
+ associated protocol with ``__aiter__``, ``__aexit__`` and new built-
+ in exception ``StopAsyncIteration``.
-5. New AST nodes: ``AsyncFunctionDef``, ``AsyncFor``, ``AsyncWith``, ``Await``.
+5. New AST nodes: ``AsyncFunctionDef``, ``AsyncFor``, ``AsyncWith``,
+ ``Await``.
6. New functions: ``sys.set_coroutine_wrapper(callback)``,
``sys.get_coroutine_wrapper()``, and ``types.coroutine(gen)``.
7. New ``CO_COROUTINE`` bit flag for code objects.
-While the list of changes and new things is not short, it is important to
-understand, that most users will not use these features directly. It is
-intended to be used in frameworks and libraries to provide users with
-convenient to use and unambiguous APIs with ``async def``, ``await``, ``async
-for`` and ``async with`` syntax.
+While the list of changes and new things is not short, it is important
+to understand, that most users will not use these features directly.
+It is intended to be used in frameworks and libraries to provide users
+with convenient to use and unambiguous APIs with ``async def``,
+``await``, ``async for`` and ``async with`` syntax.
Working example
---------------
-All concepts proposed in this PEP are implemented [3]_ and can be tested.
+All concepts proposed in this PEP are implemented [3]_ and can be
+tested.
::
@@ -998,7 +1034,8 @@
async def echo_server():
print('Serving on localhost:8000')
- await asyncio.start_server(handle_connection, 'localhost', 8000)
+ await asyncio.start_server(handle_connection,
+ 'localhost', 8000)
async def handle_connection(reader, writer):
print('New connection...')
@@ -1047,8 +1084,8 @@
Acknowledgments
===============
-I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew Svetlov,
-and Łukasz Langa for their initial feedback.
+I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew
+Svetlov, and Łukasz Langa for their initial feedback.
Copyright
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list