[Python-checkins] peps: Some more clarifications and edits. Describe datagram protocol.
guido.van.rossum
python-checkins at python.org
Tue Apr 30 06:43:17 CEST 2013
http://hg.python.org/peps/rev/6440a19fb6a4
changeset: 4867:6440a19fb6a4
user: Guido van Rossum <guido at python.org>
date: Mon Apr 29 21:24:46 2013 -0700
summary:
Some more clarifications and edits. Describe datagram protocol.
files:
pep-3156.txt | 210 +++++++++++++++++++++++---------------
1 files changed, 124 insertions(+), 86 deletions(-)
diff --git a/pep-3156.txt b/pep-3156.txt
--- a/pep-3156.txt
+++ b/pep-3156.txt
@@ -130,10 +130,6 @@
Details of the interfaces defined by the various standard types of
transports and protocols are given later.
-
-Specification
-=============
-
Dependencies
------------
@@ -143,6 +139,10 @@
packages, and no C code, except for the proactor-based event loop on
Windows.
+
+Event Loop Interface Specification
+==================================
+
Module Namespace
----------------
@@ -217,14 +217,6 @@
be retrieved by calling ``get_event_loop_policy()``. (TBD: Require
inheriting from ``AbstractEventLoopPolicy``?)
-Notes for the Event Loop Interface
-----------------------------------
-
-A note about times: as usual in Python, all timeouts, intervals and
-delays are measured in seconds, and may be ints or floats. The
-accuracy and precision of the clock are up to the implementation; the
-default implementation uses ``time.monotonic()``.
-
Event Loop Classes
------------------
@@ -298,6 +290,16 @@
- Signal callbacks: ``add_signal_handler()``,
``remove_signal_handler()``.
+Specifying Times
+----------------
+
+As usual in Python, all timeouts, intervals and delays are measured in
+seconds, and may be ints or floats. The accuracy and precision of the
+clock are up to the implementation; the default implementation uses
+``time.monotonic()``. Books could be written about the implications
+of this choice. Better read the docs for the stdandard library
+``time`` module.
+
Required Event Loop Methods
---------------------------
@@ -1033,7 +1035,9 @@
The optional second argument is the destination address. If
omitted, ``remote_addr`` must have been specified in the
``create_datagram_endpoint()`` call that created this transport. If
- present, and ``remote_addr`` was specified, they must match.
+ present, and ``remote_addr`` was specified, they must match. The
+ (data, addr) pair may be sent immediately or buffered. The return
+ value is None.
- ``abort()``. Immediately close the transport. Buffered data will
be discarded.
@@ -1056,20 +1060,26 @@
Protocols
---------
-XXX This is about where I left off.
-
-TBD Describe different kinds of protocols (bidrectional stream,
-unidirectional stream, datagram).
-
Protocols are always used in conjunction with transports. While a few
common protocols are provided (e.g. decent though not necessarily
excellent HTTP client and server implementations), most protocols will
be implemented by user code or third-party libraries.
-A protocol must implement the following methods, which will be called
-by the transport. Consider these callbacks that are always called by
-the event loop in the right context. (See the "Context" section
-above.)
+
+Like for transports, we distinguish between stream protocols, datagram
+protocols, and perhaps other custom protocols. The most common type
+of protocol is a bidirectional stream protocol. (There are no
+unidirectional protocols.)
+
+(TBD: should protocol callbacks be allowed to be coroutines?)
+
+Stream Protocols
+''''''''''''''''
+
+A (bidirectional) stream protocol must implement the following
+methods, which will be called by the transport. Think of these as
+callbacks that are always called by the event loop in the right
+context. (See the "Context" section way above.)
- ``connection_made(transport)``. Indicates that the transport is
ready and connected to the entity at the other end. The protocol
@@ -1108,6 +1118,36 @@
TBD: Discuss whether user code needs to do anything to make sure that
protocol and transport aren't garbage-collected prematurely.
+Datagram Protocols
+''''''''''''''''''
+
+Datagram protocols have ``connection_made()`` and
+``connection_lost()`` methods with the same signatures as stream
+protocols. (As explained in the section about datagram transports, we
+prefer the slightly odd nomenclature over defining different method
+names to indicating the opening and closing of the socket.)
+
+In addition, they have the following methods:
+
+- ``datagram_received(data, addr)``. Indicates that a datagram
+ ``data`` (a bytes objects) was received from remote address ``addr``
+ (an IPv4 2-tuple or an IPv6 4-tuple).
+
+- ``connection_refused(exc)``. Indicates that a send or receive
+ operation raised a ``ConnectionRefused`` exception. This typically
+ indicates that a negative acknowledgment was received for a
+ previously sent datagram (not for the datagram that was being sent,
+ if the exception was raised by a send operation). Immediately after
+ this the socket will be closed and ``connection_lost()`` will be
+ called with the same exception argument.
+
+Here is a chart indicating the order and multiplicity of calls:
+
+ 1. ``connection_made()`` -- exactly once
+ 2. ``datagram_received()`` -- zero or more times
+ 3. ``connection_refused()`` -- at most once
+ 4. ``connection_lost()`` -- exactly once
+
Callback Style
--------------
@@ -1128,13 +1168,6 @@
loop.call_soon(functools.partial(foo, "abc", repeat=42))
-Choosing an Event Loop Implementation
--------------------------------------
-
-TBD. (This is about the choice to use e.g. select vs. poll vs. epoll,
-and how to override the choice. Probably belongs in the event loop
-policy.)
-
Coroutines and the Scheduler
============================
@@ -1159,25 +1192,27 @@
different (though related) concepts:
- The function that defines a coroutine (a function definition
- decorated with ``tulip.coroutine``). If disambiguation is needed,
- we call this a *coroutine function*.
+ decorated with ``tulip.coroutine``). If disambiguation is needed
+ we will call this a *coroutine function*.
- The object obtained by calling a coroutine function. This object
represents a computation or an I/O operation (usually a combination)
- that will complete eventually. For disambiguation we call it a
- *coroutine object*.
+ that will complete eventually. If disambiguation is needed we will
+ call it a *coroutine object*.
Things a coroutine can do:
- ``result = yield from future`` -- suspends the coroutine until the
- future is done, then returns the future's result, or raises its
- exception, which will be propagated.
+ future is done, then returns the future's result, or raises an
+ exception, which will be propagated. (If the future is cancelled,
+ it will raise a ``CancelledError`` exception.) Note that tasks are
+ futures, and everything said about futures also applies to tasks.
- ``result = yield from coroutine`` -- wait for another coroutine to
produce a result (or raise an exception, which will be propagated).
The ``coroutine`` expression must be a *call* to another coroutine.
-- ``return result`` -- produce a result to the coroutine that is
+- ``return expression`` -- produce a result to the coroutine that is
waiting for this one using ``yield from``.
- ``raise exception`` -- raise an exception in the coroutine that is
@@ -1191,7 +1226,7 @@
(assuming the other coroutine is already running!), or convert it to a
Task (see below).
-Coroutines can only run when the event loop is running.
+Coroutines (and tasks) can only run when the event loop is running.
Waiting for Multiple Coroutines
-------------------------------
@@ -1207,13 +1242,14 @@
tuple of two sets of Futures, ``(done, pending)``, where ``done`` is
the set of original Futures (or wrapped coroutines) that are done
(or cancelled), and ``pending`` is the rest, i.e. those that are
- still not done (nor cancelled). Optional arguments ``timeout`` and
- ``return_when`` have the same meaning and defaults as for
- ``concurrent.futures.wait()``: ``timeout``, if not ``None``,
- specifies a timeout for the overall operation; ``return_when``,
- specifies when to stop. The constants ``FIRST_COMPLETED``,
- ``FIRST_EXCEPTION``, ``ALL_COMPLETED`` are defined with the same
- values and the same meanings as in PEP 3148:
+ still not done (nor cancelled). Note that with the defaults for
+ ``timeout`` and ``return_when``, ``done`` will always be an empty
+ list. Optional arguments ``timeout`` and ``return_when`` have the
+ same meaning and defaults as for ``concurrent.futures.wait()``:
+ ``timeout``, if not ``None``, specifies a timeout for the overall
+ operation; ``return_when``, specifies when to stop. The constants
+ ``FIRST_COMPLETED``, ``FIRST_EXCEPTION``, ``ALL_COMPLETED`` are
+ defined with the same values and the same meanings as in PEP 3148:
- ``ALL_COMPLETED`` (default): Wait until all Futures are done or
completed (or until the timeout occurs).
@@ -1239,30 +1275,49 @@
result = yield from f # May raise an exception.
# Use result.
+ Note: if you do not wait for the futures as they are produced by the
+ iterator, your ``for`` loop may not make progress (since you are not
+ allowing other tasks to run).
+
+Sleeping
+--------
+
+The coroutine ``sleep(delay)`` returns after a given time delay.
+
+(TBD: Should the optional second argument, ``result``, be part of the
+spec?)
+
Tasks
-----
A Task is an object that manages an independently running coroutine.
-The Task interface is the same as the Future interface. The task
-becomes done when its coroutine returns or raises an exception; if it
-returns a result, that becomes the task's result, if it raises an
-exception, that becomes the task's exception.
+The Task interface is the same as the Future interface, and in fact
+``Task`` is a subclass of ``Future``. The task becomes done when its
+coroutine returns or raises an exception; if it returns a result, that
+becomes the task's result, if it raises an exception, that becomes the
+task's exception.
Cancelling a task that's not done yet prevents its coroutine from
-completing; in this case an exception is thrown into the coroutine
-that it may catch to further handle cancellation, but it doesn't have
-to (this is done using the standard ``close()`` method on generators,
-described in PEP 342).
+completing. In this case a ``CancelledError`` exception is thrown
+into the coroutine that it may catch to further handle cancellation.
+If the exception is not caught, the generator will be properly
+finalized anyway, as described in PEP 342.
Tasks are also useful for interoperating between coroutines and
callback-based frameworks like Twisted. After converting a coroutine
into a Task, callbacks can be added to the Task.
-You may ask, why not convert all coroutines to Tasks? The
-``@tulip.coroutine`` decorator could do this. This would slow things
-down considerably in the case where one coroutine calls another (and
-so on), as switching to a "bare" coroutine has much less overhead than
-switching to a Task.
+There are two ways to convert a coroutine into a task: explicitly, by
+calling the coroutine function and then passing the resulting
+coroutine object to the ``tulip.Task()`` constructor; or implicitly,
+by decorating the coroutine with ``@tulip.task`` (instead of
+``@tulip.coroutine``).
+
+You may ask, why not automatically convert all coroutines to Tasks?
+The ``@tulip.coroutine`` decorator could do this. However, this would
+slow things down considerably in the case where one coroutine calls
+another (and so on), as switching to a "bare" coroutine has much less
+overhead than switching to a Task.
The Scheduler
-------------
@@ -1274,38 +1329,21 @@
public interface of the event loop, so it will work with third-party
event loop implementations, too.
-Sleeping
---------
-
-TBD: ``yield sleep(seconds)``. Can use ``sleep(0)`` to suspend to
-poll for I/O.
-
Coroutines and Protocols
------------------------
-The best way to use coroutines to implement protocols is probably to
-use a streaming buffer that gets filled by ``data_received()`` and can
-be read asynchronously using methods like ``read(n)`` and
-``readline()`` that return a Future. When the connection is closed,
-``read()`` should return a Future whose result is ``b''``, or raise an
-exception if ``connection_closed()`` is called with an exception.
+The best way to use coroutines to implement an Internet protocol such
+as FTP is probably to use a streaming buffer that gets filled by
+``data_received()`` and can be read asynchronously using methods like
+``read(n)`` and ``readline()`` that are coroutines or return a Future.
+When the connection is closed, ``read()`` should eventually produce
+``b''``, or raise an exception if ``connection_closed()`` is called
+with an exception.
-To write, the ``write()`` method (and friends) on the transport can be
-used -- these do not return Futures. A standard protocol
-implementation should be provided that sets this up and kicks off the
-coroutine when ``connection_made()`` is called.
-
-TBD: Be more specific.
-
-Cancellation
-------------
-
-TBD. When a Task is cancelled its coroutine may see an exception at
-any point where it is yielding to the scheduler (i.e., potentially at
-any ``yield from`` operation). We need to spell out which exception
-is raised.
-
-Also TBD: timeouts.
+To write a response, the ``write()`` method (and friends) on the
+transport can be used -- these do not return Futures. A standard
+protocol implementation should be provided that sets this up and kicks
+off the coroutine when ``connection_made()`` is called.
Open Issues
@@ -1336,7 +1374,7 @@
these would all require using Tulip internals.
- Locks and queues? The Tulip implementation contains implementations
- of most types of locks and queues modeled after the stdlib
+ of most types of locks and queues modeled after the standard library
``threading`` and ``queue`` modules. Should we incorporate these in
the PEP?
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list