[Python-checkins] bpo-33649: More improvements (GH-9439)
Yury Selivanov
webhook-mailer at python.org
Thu Sep 20 12:44:10 EDT 2018
https://github.com/python/cpython/commit/e247b46cba4f4d32ea96a15dbc36d73265171106
commit: e247b46cba4f4d32ea96a15dbc36d73265171106
branch: master
author: Yury Selivanov <yury at magic.io>
committer: GitHub <noreply at github.com>
date: 2018-09-20T12:43:59-04:00
summary:
bpo-33649: More improvements (GH-9439)
files:
M Doc/library/asyncio-eventloop.rst
M Doc/library/asyncio-task.rst
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
index 92637327ae40..50d6ea4b1389 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -755,7 +755,7 @@ Watching file descriptors
invoke *callback* with the specified arguments once *fd* is available for
writing.
- Use :func:`functools.partial` :ref:`to pass keywords
+ Use :func:`functools.partial` :ref:`to pass keyword arguments
<asyncio-pass-keywords>` to *func*.
.. method:: loop.remove_writer(fd)
@@ -969,7 +969,7 @@ Unix signals
Raise :exc:`ValueError` if the signal number is invalid or uncatchable.
Raise :exc:`RuntimeError` if there is a problem setting up the handler.
- Use :func:`functools.partial` :ref:`to pass keywords
+ Use :func:`functools.partial` :ref:`to pass keyword arguments
<asyncio-pass-keywords>` to *func*.
.. method:: loop.remove_signal_handler(sig)
@@ -996,11 +996,52 @@ Executing code in thread or process pools
The *executor* argument should be an :class:`concurrent.futures.Executor`
instance. The default executor is used if *executor* is ``None``.
- Use :func:`functools.partial` :ref:`to pass keywords
- <asyncio-pass-keywords>` to *func*.
+ Example::
+
+ import asyncio
+ import concurrent.futures
+
+ def blocking_io():
+ # File operations (such as logging) can block the
+ # event loop: run them in a thread pool.
+ with open('/dev/urandom', 'rb') as f:
+ return f.read(100)
+
+ def cpu_bound():
+ # CPU-bound operations will block the event loop:
+ # in general it is preferable to run them in a
+ # process pool.
+ return sum(i * i for i in range(10 ** 7))
+
+ async def main():
+ loop = asyncio.get_running_loop()
+
+ ## Options:
+
+ # 1. Run in the default loop's executor:
+ result = await loop.run_in_executor(
+ None, blocking_io)
+ print('default thread pool', result)
+
+ # 2. Run in a custom thread pool:
+ with concurrent.futures.ThreadPoolExecutor() as pool:
+ result = await loop.run_in_executor(
+ pool, blocking_io)
+ print('custom thread pool', result)
+
+ # 3. Run in a custom process pool:
+ with concurrent.futures.ProcessPoolExecutor() as pool:
+ result = await loop.run_in_executor(
+ pool, cpu_bound)
+ print('custom process pool', result)
+
+ asyncio.run(main())
This method returns a :class:`asyncio.Future` object.
+ Use :func:`functools.partial` :ref:`to pass keyword arguments
+ <asyncio-pass-keywords>` to *func*.
+
.. versionchanged:: 3.5.3
:meth:`loop.run_in_executor` no longer configures the
``max_workers`` of the thread pool executor it creates, instead
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst
index 85292a61ecd6..b33a7df4e822 100644
--- a/Doc/library/asyncio-task.rst
+++ b/Doc/library/asyncio-task.rst
@@ -109,50 +109,89 @@ To actually run a coroutine asyncio provides three main mechanisms:
Awaitables
==========
-We say that an object is an *awaitable* object if it can be used
-in an :keyword:`await` expression.
+We say that an object is an **awaitable** object if it can be used
+in an :keyword:`await` expression. Many asyncio APIs are designed to
+accept awaitables.
+There are three main types of *awaitable* objects:
+**coroutines**, **Tasks**, and **Futures**.
-.. rubric:: Coroutines and Tasks
-Python coroutines are *awaitables*::
+.. rubric:: Coroutines
+
+Python coroutines are *awaitables* and therefore can be awaited from
+other coroutines::
+
+ import asyncio
async def nested():
return 42
async def main():
- # Will print "42":
- print(await nested())
+ # Nothing happens if we just call "nested()".
+ # (a coroutine object is created but not awaited)
+ nested()
+
+ # Let's do it differently now and await it:
+ print(await nested()) # will print "42".
+
+ asyncio.run(main())
+
+.. important::
+
+ In this documentation the term "coroutine" can be used for
+ two closely related concepts:
+
+ * a *coroutine function*: an :keyword:`async def` function;
+
+ * a *coroutine object*: an object returned by calling a
+ *coroutine function*.
+
+asyncio also supports legacy :ref:`generator-based
+<asyncio_generator_based_coro>` coroutines.
+
+
+.. rubric:: Tasks
*Tasks* are used to schedule coroutines *concurrently*.
-See the previous :ref:`section <coroutine>` for an introduction
-to coroutines and tasks.
-Note that in this documentation the term "coroutine" can be used for
-two closely related concepts:
+When a coroutine is wrapped into a *Task* with functions like
+:func:`asyncio.create_task` the coroutine is automatically
+scheduled to run soon::
+
+ import asyncio
+
+ async def nested():
+ return 42
-* a *coroutine function*: an :keyword:`async def` function;
+ async def main():
+ # Schedule nested() to run soon concurrently
+ # with "main()".
+ task = asyncio.create_task(nested())
-* a *coroutine object*: object returned by calling a
- *coroutine function*.
+ # "task" can now be used to cancel "nested()", or
+ # can simply be awaited to wait until it is complete:
+ await task
+
+ asyncio.run(main())
.. rubric:: Futures
-There is a dedicated section about the :ref:`asyncio Future object
-<asyncio-futures>`, but the concept is fundamental to asyncio so
-it needs a brief introduction in this section.
+A :class:`Future` is a special **low-level** awaitable object that
+represents an **eventual result** of an asynchronous operation.
+
+When a Future object is *awaited* it means that the coroutine will
+wait until the Future is resolved in some other place.
-A Future is a special **low-level** awaitable object that represents
-an **eventual result** of an asynchronous operation.
Future objects in asyncio are needed to allow callback-based code
to be used with async/await.
-Normally, **there is no need** to create Future objects at the
+Normally **there is no need** to create Future objects at the
application level code.
Future objects, sometimes exposed by libraries and some asyncio
-APIs, should be awaited::
+APIs, can be awaited::
async def main():
await function_that_returns_a_future_object()
@@ -163,6 +202,9 @@ APIs, should be awaited::
some_python_coroutine()
)
+A good example of a low-level function that returns a Future object
+is :meth:`loop.run_in_executor`.
+
Running an asyncio Program
==========================
@@ -192,8 +234,8 @@ Creating Tasks
.. function:: create_task(coro, \*, name=None)
- Wrap the *coro* :ref:`coroutine <coroutine>` into a Task and
- schedule its execution. Return the Task object.
+ Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
+ and schedule its execution. Return the Task object.
If *name* is not ``None``, it is set as the name of the task using
:meth:`Task.set_name`.
@@ -259,17 +301,17 @@ Sleeping
Running Tasks Concurrently
==========================
-.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False)
+.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False)
- Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+ Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
sequence *concurrently*.
- If any awaitable in *fs* is a coroutine, it is automatically
+ If any awaitable in *aws* is a coroutine, it is automatically
scheduled as a Task.
If all awaitables are completed successfully, the result is an
aggregate list of returned values. The order of result values
- corresponds to the order of awaitables in *fs*.
+ corresponds to the order of awaitables in *aws*.
If *return_exceptions* is ``True``, exceptions are treated the
same as successful results, and aggregated in the result list.
@@ -279,7 +321,7 @@ Running Tasks Concurrently
If ``gather`` is *cancelled*, all submitted awaitables
(that have not completed yet) are also *cancelled*.
- If any Task or Future from the *fs* sequence is *cancelled*, it is
+ If any Task or Future from the *aws* sequence is *cancelled*, it is
treated as if it raised :exc:`CancelledError` -- the ``gather()``
call is **not** cancelled in this case. This is to prevent the
cancellation of one submitted Task/Future to cause other
@@ -329,13 +371,13 @@ Running Tasks Concurrently
Shielding Tasks From Cancellation
=================================
-.. awaitablefunction:: shield(fut, \*, loop=None)
+.. awaitablefunction:: shield(aw, \*, loop=None)
Protect an :ref:`awaitable object <asyncio-awaitables>`
from being :meth:`cancelled <Task.cancel>`.
- *fut* can be a coroutine, a Task, or a Future-like object. If
- *fut* is a coroutine it is automatically scheduled as a Task.
+ *aw* can be a coroutine, a Task, or a Future-like object. If
+ *aw* is a coroutine it is automatically scheduled as a Task.
The statement::
@@ -367,12 +409,12 @@ Shielding Tasks From Cancellation
Timeouts
========
-.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None)
+.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None)
- Wait for the *fut* :ref:`awaitable <asyncio-awaitables>`
+ Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
to complete with a timeout.
- If *fut* is a coroutine it is automatically scheduled as a Task.
+ If *aw* is a coroutine it is automatically scheduled as a Task.
*timeout* can either be ``None`` or a float or int number of seconds
to wait for. If *timeout* is ``None``, block until the future
@@ -387,7 +429,7 @@ Timeouts
The function will wait until the future is actually cancelled,
so the total wait time may exceed the *timeout*.
- If the wait is cancelled, the future *fut* is also cancelled.
+ If the wait is cancelled, the future *aw* is also cancelled.
The *loop* argument is deprecated and scheduled for removal
in Python 4.0.
@@ -415,22 +457,22 @@ Timeouts
# timeout!
.. versionchanged:: 3.7
- When *fut* is cancelled due to a timeout, ``wait_for`` waits
- for *fut* to be cancelled. Previously, it raised
+ When *aw* is cancelled due to a timeout, ``wait_for`` waits
+ for *aw* to be cancelled. Previously, it raised
:exc:`asyncio.TimeoutError` immediately.
Waiting Primitives
==================
-.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\
+.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\
return_when=ALL_COMPLETED)
- Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+ Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
sequence concurrently and block until the condition specified
by *return_when*.
- If any awaitable in *fs* is a coroutine, it is automatically
+ If any awaitable in *aws* is a coroutine, it is automatically
scheduled as a Task.
Returns two sets of Tasks/Futures: ``(done, pending)``.
@@ -471,12 +513,12 @@ Waiting Primitives
Usage::
- done, pending = await asyncio.wait(fs)
+ done, pending = await asyncio.wait(aws)
-.. function:: as_completed(fs, \*, loop=None, timeout=None)
+.. function:: as_completed(aws, \*, loop=None, timeout=None)
- Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+ Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
set concurrently. Return an iterator of :class:`Future` objects.
Each Future object returned represents the earliest result
from the set of the remaining awaitables.
@@ -486,7 +528,7 @@ Waiting Primitives
Example::
- for f in as_completed(fs):
+ for f in as_completed(aws):
earliest_result = await f
# ...
@@ -679,6 +721,52 @@ Task Object
A Task is *done* when the wrapped coroutine either returned
a value, raised an exception, or the Task was cancelled.
+ .. method:: result()
+
+ Return the result of the Task.
+
+ If the Task is *done*, the result of the wrapped coroutine
+ is returned (or if the coroutine raised an exception, that
+ exception is re-raised.)
+
+ If the Task has been *cancelled*, this method raises
+ a :exc:`CancelledError` exception.
+
+ If the Task's result isn't yet available, this method raises
+ a :exc:`InvalidStateError` exception.
+
+ .. method:: exception()
+
+ Return the exception of the Task.
+
+ If the wrapped coroutine raised an exception that exception
+ is returned. If the wrapped coroutine returned normally
+ this method returns ``None``.
+
+ If the Task has been *cancelled*, this method raises a
+ :exc:`CancelledError` exception.
+
+ If the Task isn't *done* yet, this method raises an
+ :exc:`InvalidStateError` exception.
+
+ .. method:: add_done_callback(callback, *, context=None)
+
+ Add a callback to be run when the Task is *done*.
+
+ This method should only be used in low-level callback-based code.
+
+ See the documentation of :meth:`Future.add_done_callback`
+ for more details.
+
+ .. method:: remove_done_callback(callback)
+
+ Remove *callback* from the callbacks list.
+
+ This method should only be used in low-level callback-based code.
+
+ See the documentation of :meth:`Future.remove_done_callback`
+ for more details.
+
.. method:: get_stack(\*, limit=None)
Return the list of stack frames for this Task.
More information about the Python-checkins
mailing list