[Python-checkins] peps: More formatting and fill-column fixes.
georg.brandl
python-checkins at python.org
Sun Oct 30 11:45:16 CET 2011
http://hg.python.org/peps/rev/01720a11ec36
changeset: 3977:01720a11ec36
user: Georg Brandl <georg at python.org>
date: Sun Oct 30 12:45:18 2011 +0100
summary:
More formatting and fill-column fixes.
files:
pep-0404.txt | 2 +-
pep-3153.txt | 178 +++++++++++++++++++-------------------
pep-3154.txt | 107 ++++++++++++-----------
pep-3155.txt | 45 +++++----
4 files changed, 171 insertions(+), 161 deletions(-)
diff --git a/pep-0404.txt b/pep-0404.txt
--- a/pep-0404.txt
+++ b/pep-0404.txt
@@ -44,7 +44,7 @@
to perform a delicate boot-strapping dance at every startup.
(Virtualenv copies the binary because symlinking it does not provide
isolation, as Python dereferences a symlinked executable before
-searching for `sys.prefix`.)
+searching for ``sys.prefix``.)
The ``PYTHONHOME`` environment variable, Python's only existing
built-in solution for virtual environments, requires
diff --git a/pep-3153.txt b/pep-3153.txt
--- a/pep-3153.txt
+++ b/pep-3153.txt
@@ -25,37 +25,37 @@
People who want to write asynchronous code in Python right now have a
few options:
- - ``asyncore`` and ``asynchat``
- - something bespoke, most likely based on the ``select`` module
- - using a third party library, such as Twisted_ or gevent_
+- ``asyncore`` and ``asynchat``
+- something bespoke, most likely based on the ``select`` module
+- using a third party library, such as Twisted_ or gevent_
Unfortunately, each of these options has its downsides, which this PEP
tries to address.
-Despite having been part of the Python standard library for a long time,
-the asyncore module suffers from fundamental flaws following from
-an inflexible API that does not stand up to the expectations of
-a modern asynchronous networking module.
+Despite having been part of the Python standard library for a long
+time, the asyncore module suffers from fundamental flaws following
+from an inflexible API that does not stand up to the expectations of a
+modern asynchronous networking module.
-Moreover, its approach is too simplistic to provide developers with all
-the tools they need in order to fully exploit the potential of asynchronous
-networking.
+Moreover, its approach is too simplistic to provide developers with
+all the tools they need in order to fully exploit the potential of
+asynchronous networking.
The most popular solution right now used in production involves the
-use of third party libraries. These often provide satisfactory
+use of third party libraries. These often provide satisfactory
solutions, but there is a lack of compatibility between these
libraries, which tends to make codebases very tightly coupled to the
library they use.
This current lack of portability between different asynchronous IO
libraries causes a lot of duplicated effort for third party library
-developers. A sufficiently powerful abstraction could mean that
+developers. A sufficiently powerful abstraction could mean that
asynchronous code gets written once, but used everywhere.
An eventual added goal would be for standard library implementations
of wire and network protocols to evolve towards being real protocol
implementations, as opposed to standalone libraries that do everything
-including calling ``recv()`` blockingly. This means they could be
+including calling ``recv()`` blockingly. This means they could be
easily reused for both synchronous and asynchronous code.
.. _Twisted: http://www.twistedmatrix.com/
@@ -71,103 +71,104 @@
bytes to different kinds of connections. Transports in this PEP are
always ordered, reliable, bidirectional, stream-oriented two-endpoint
connections. This might be a TCP socket, an SSL connection, a pipe
-(named or otherwise), a serial port... It may abstract a file descriptor
-on POSIX platforms or a Handle on Windows or some other data structure
-appropriate to a particular platform. It encapsulates all of the
-particular implementation details of using that platform data structure
-and presents a uniform interface for application developers.
+(named or otherwise), a serial port... It may abstract a file
+descriptor on POSIX platforms or a Handle on Windows or some other
+data structure appropriate to a particular platform. It encapsulates
+all of the particular implementation details of using that platform
+data structure and presents a uniform interface for application
+developers.
-Transports talk to two things: the other side of the connection on
-one hand, and a protocol on the other. It's a bridge between the
-specific underlying transfer mechanism and the protocol. Its job can
-be described as allowing the protocol to just send and receive bytes,
-taking care of all of the magic that needs to happen to those bytes
-to be eventually sent across the wire.
+Transports talk to two things: the other side of the connection on one
+hand, and a protocol on the other. It's a bridge between the specific
+underlying transfer mechanism and the protocol. Its job can be
+described as allowing the protocol to just send and receive bytes,
+taking care of all of the magic that needs to happen to those bytes to
+be eventually sent across the wire.
The primary feature of a transport is sending bytes to a protocol and
-receiving bytes from the underlying protocol. Writing to the transport
-is done using the ``write`` and ``write_sequence`` methods. The latter
-method is a performance optimization, to allow software to take
-advantage of specific capabilities in some transport
-mechanisms. Specifically, this allows transports to use writev_
-instead of write_ or send_, also known as scatter/gather IO.
+receiving bytes from the underlying protocol. Writing to the
+transport is done using the ``write`` and ``write_sequence`` methods.
+The latter method is a performance optimization, to allow software to
+take advantage of specific capabilities in some transport mechanisms.
+Specifically, this allows transports to use writev_ instead of write_
+or send_, also known as scatter/gather IO.
-A transport can be paused and resumed. This will cause it to buffer
+A transport can be paused and resumed. This will cause it to buffer
data coming from protocols and stop sending received data to the
protocol.
-A transport can also be closed, half-closed and aborted. A closed
+A transport can also be closed, half-closed and aborted. A closed
transport will finish writing all of the data queued in it to the
-underlying mechanism, and will then stop reading or writing
-data. Aborting a transport stops it, closing the connection without
-sending any data that is still queued.
+underlying mechanism, and will then stop reading or writing data.
+Aborting a transport stops it, closing the connection without sending
+any data that is still queued.
-Further writes will result in exceptions being thrown. A half-closed
+Further writes will result in exceptions being thrown. A half-closed
transport may not be written to anymore, but will still accept
incoming data.
Protocols
---------
-Protocols are probably more familiar to new users. The terminology is
+Protocols are probably more familiar to new users. The terminology is
consistent with what you would expect from something called a
protocol: the protocols most people think of first, like HTTP, IRC,
SMTP... are all examples of something that would be implemented in a
protocol.
The shortest useful definition of a protocol is a (usually two-way)
-bridge between the transport and the rest of the application logic. A
+bridge between the transport and the rest of the application logic. A
protocol will receive bytes from a transport and translates that
information into some behavior, typically resulting in some method
-calls on an object. Similarly, application logic calls some methods on
-the protocol, which the protocol translates into bytes and
+calls on an object. Similarly, application logic calls some methods
+on the protocol, which the protocol translates into bytes and
communicates to the transport.
One of the simplest protocols is a line-based protocol, where data is
-delimited by ``\r\n``. The protocol will receive bytes from the
-transport and buffer them until there is at least one complete
-line. Once that's done, it will pass this line along to some
-object. Ideally that would be accomplished using a callable or even a
+delimited by ``\r\n``. The protocol will receive bytes from the
+transport and buffer them until there is at least one complete line.
+Once that's done, it will pass this line along to some object.
+Ideally that would be accomplished using a callable or even a
completely separate object composed by the protocol, but it could also
be implemented by subclassing (as is the case with Twisted's
-``LineReceiver``). For the other direction, the protocol could have a
+``LineReceiver``). For the other direction, the protocol could have a
``write_line`` method, which adds the required ``\r\n`` and passes the
new bytes buffer on to the transport.
This PEP suggests a generalized ``LineReceiver`` called
``ChunkProtocol``, where a "chunk" is a message in a stream, delimited
-by the specified delimiter. Instances take a delimiter and a callable
+by the specified delimiter. Instances take a delimiter and a callable
that will be called with a chunk of data once it's received (as
-opposed to Twisted's subclassing behavior). ``ChunkProtocol`` also has
-a ``write_chunk`` method analogous to the ``write_line`` method
+opposed to Twisted's subclassing behavior). ``ChunkProtocol`` also
+has a ``write_chunk`` method analogous to the ``write_line`` method
described above.
Why separate protocols and transports?
--------------------------------------
This separation between protocol and transport often confuses people
-who first come across it. In fact, the standard library itself does
+who first come across it. In fact, the standard library itself does
not make this distinction in many cases, particularly not in the API
it provides to users.
-It is nonetheless a very useful distinction. In the worst case, it
-simplifies the implementation by clear separation of
-concerns. However, it often serves the far more useful purpose of
-being able to reuse protocols across different transports.
+It is nonetheless a very useful distinction. In the worst case, it
+simplifies the implementation by clear separation of concerns.
+However, it often serves the far more useful purpose of being able to
+reuse protocols across different transports.
-Consider a simple RPC protocol. The same bytes may be transferred
-across many different transports, for example pipes or sockets. To
-help with this, we separate the protocol out from the transport. The
+Consider a simple RPC protocol. The same bytes may be transferred
+across many different transports, for example pipes or sockets. To
+help with this, we separate the protocol out from the transport. The
protocol just reads and writes bytes, and doesn't really care what
mechanism is used to eventually transfer those bytes.
This also allows for protocols to be stacked or nested easily,
-allowing for even more code reuse. A common example of this is
+allowing for even more code reuse. A common example of this is
JSON-RPC: according to the specification, it can be used across both
-sockets and HTTP[#jsonrpc]_ . In practice, it tends to be primarily
-encapsulated in HTTP. The protocol-transport abstraction allows us to
+sockets and HTTP[#jsonrpc]_ . In practice, it tends to be primarily
+encapsulated in HTTP. The protocol-transport abstraction allows us to
build a stack of protocols and transports that allow you to use HTTP
-as if it were a transport. For JSON-RPC, that might get you a stack
+as if it were a transport. For JSON-RPC, that might get you a stack
somewhat like this:
1. TCP socket transport
@@ -182,16 +183,16 @@
Consumers
---------
-Consumers consume bytes produced by producers. Together with
+Consumers consume bytes produced by producers. Together with
producers, they make flow control possible.
-Consumers primarily play a passive role in flow control. They get
-called whenever a producer has some data available. They then process
+Consumers primarily play a passive role in flow control. They get
+called whenever a producer has some data available. They then process
that data, and typically yield control back to the producer.
-Consumers typically implement buffers of some sort. They make flow
+Consumers typically implement buffers of some sort. They make flow
control possible by telling their producer about the current status of
-those buffers. A consumer can instruct a producer to stop producing
+those buffers. A consumer can instruct a producer to stop producing
entirely, stop producing temporarily, or resume producing if it has
been told to pause previously.
@@ -204,23 +205,23 @@
Where consumers consume bytes, producers produce them.
Producers are modeled after the IPushProducer_ interface found in
-Twisted. Although there is an IPullProducer_ as well, it is on the
+Twisted. Although there is an IPullProducer_ as well, it is on the
whole far less interesting and therefore probably out of the scope of
this PEP.
Although producers can be told to stop producing entirely, the two
-most interesting methods they have are ``pause`` and ``resume``. These
-are usually called by the consumer, to signify whether it is ready to
-process ("consume") more data or not. Consumers and producers
-cooperate to make flow control possible.
+most interesting methods they have are ``pause`` and ``resume``.
+These are usually called by the consumer, to signify whether it is
+ready to process ("consume") more data or not. Consumers and
+producers cooperate to make flow control possible.
In addition to the Twisted IPushProducer_ interface, producers have a
``half_register`` method which is called with the consumer when the
-consumer tries to register that producer. In most cases, this will
+consumer tries to register that producer. In most cases, this will
just be a case of setting ``self.consumer = consumer``, but some
producers may require more complex preconditions or behavior when a
-consumer is registered. End-users are not supposed to call this method
-directly.
+consumer is registered. End-users are not supposed to call this
+method directly.
===========================
Considered API alternatives
@@ -229,41 +230,42 @@
Generators as producers
~~~~~~~~~~~~~~~~~~~~~~~
-Generators have been suggested as way to implement producers. However,
-there appear to be a few problems with this.
+Generators have been suggested as way to implement producers.
+However, there appear to be a few problems with this.
-First of all, there is a conceptual problem. A generator, in a sense,
-is "passive". It needs to be told, through a method call, to take
-action. A producer is "active": it initiates those method calls. A
-real producer has a symmetric relationship with it's consumer. In the
+First of all, there is a conceptual problem. A generator, in a sense,
+is "passive". It needs to be told, through a method call, to take
+action. A producer is "active": it initiates those method calls. A
+real producer has a symmetric relationship with it's consumer. In the
case of a generator-turned-producer, only the consumer would have a
reference, and the producer is blissfully unaware of the consumer's
existence.
This conceptual problem translates into a few technical issues as
-well. After a successful ``write`` method call on its consumer, a
-(push) producer is free to take action once more. In the case of a
+well. After a successful ``write`` method call on its consumer, a
+(push) producer is free to take action once more. In the case of a
generator, it would need to be told, either by asking for the next
object through the iteration protocol (a process which could block
indefinitely), or perhaps by throwing some kind of signal exception
into it.
This signaling setup may provide a technically feasible solution, but
-it is still unsatisfactory. For one, this introduces unwarranted
+it is still unsatisfactory. For one, this introduces unwarranted
complexity in the consumer, which now not only needs to understand how
to receive and process data, but also how to ask for new data and deal
with the case of no new data being available.
-This latter edge case is particularly problematic. It needs to be
-taken care of, since the entire operation is not allowed to
-block. However, generators can not raise an exception on iteration
-without terminating, thereby losing the state of the generator. As a
-result, signaling a lack of available data would have to be done using
-a sentinel value, instead of being done using th exception mechanism.
+This latter edge case is particularly problematic. It needs to be
+taken care of, since the entire operation is not allowed to block.
+However, generators can not raise an exception on iteration without
+terminating, thereby losing the state of the generator. As a result,
+signaling a lack of available data would have to be done using a
+sentinel value, instead of being done using th exception mechanism.
Last but not least, nobody produced actually working code
demonstrating how they could be used.
+
References
==========
diff --git a/pep-3154.txt b/pep-3154.txt
--- a/pep-3154.txt
+++ b/pep-3154.txt
@@ -16,28 +16,30 @@
========
Data serialized using the pickle module must be portable across Python
-versions. It should also support the latest language features as well as
-implementation-specific features. For this reason, the pickle module knows
-about several protocols (currently numbered from 0 to 3), each of which
-appeared in a different Python version. Using a low-numbered protocol
-version allows to exchange data with old Python versions, while using a
-high-numbered protocol allows access to newer features and sometimes more
-efficient resource use (both CPU time required for (de)serializing, and
-disk size / network bandwidth required for data transfer).
+versions. It should also support the latest language features as well
+as implementation-specific features. For this reason, the pickle
+module knows about several protocols (currently numbered from 0 to 3),
+each of which appeared in a different Python version. Using a
+low-numbered protocol version allows to exchange data with old Python
+versions, while using a high-numbered protocol allows access to newer
+features and sometimes more efficient resource use (both CPU time
+required for (de)serializing, and disk size / network bandwidth
+required for data transfer).
Rationale
=========
-The latest current protocol, coincidentally named protocol 3, appeared with
-Python 3.0 and supports the new incompatible features in the language
-(mainly, unicode strings by default and the new bytes object). The
-opportunity was not taken at the time to improve the protocol in other ways.
+The latest current protocol, coincidentally named protocol 3, appeared
+with Python 3.0 and supports the new incompatible features in the
+language (mainly, unicode strings by default and the new bytes
+object). The opportunity was not taken at the time to improve the
+protocol in other ways.
-This PEP is an attempt to foster a number of small incremental improvements
-in a future new protocol version. The PEP process is used in order to gather
-as many improvements as possible, because the introduction of a new protocol
-version should be a rare occurrence.
+This PEP is an attempt to foster a number of small incremental
+improvements in a future new protocol version. The PEP process is
+used in order to gather as many improvements as possible, because the
+introduction of a new protocol version should be a rare occurrence.
Improvements in discussion
@@ -46,67 +48,69 @@
64-bit compatibility for large objects
--------------------------------------
-Current protocol versions export object sizes for various built-in types
-(str, bytes) as 32-bit ints. This forbids serialization of large data [1]_.
-New opcodes are required to support very large bytes and str objects.
+Current protocol versions export object sizes for various built-in
+types (str, bytes) as 32-bit ints. This forbids serialization of
+large data [1]_. New opcodes are required to support very large bytes
+and str objects.
Native opcodes for sets and frozensets
--------------------------------------
-Many common built-in types (such as str, bytes, dict, list, tuple) have
-dedicated opcodes to improve resource consumption when serializing and
-deserializing them; however, sets and frozensets don't. Adding such opcodes
-would be an obvious improvement. Also, dedicated set support could help
-remove the current impossibility of pickling self-referential sets
-[2]_.
+Many common built-in types (such as str, bytes, dict, list, tuple)
+have dedicated opcodes to improve resource consumption when
+serializing and deserializing them; however, sets and frozensets
+don't. Adding such opcodes would be an obvious improvement. Also,
+dedicated set support could help remove the current impossibility of
+pickling self-referential sets [2]_.
Calling __new__ with keyword arguments
--------------------------------------
-Currently, classes whose __new__ mandates the use of keyword-only arguments
-can not be pickled (or, rather, unpickled) [3]_. Both a new special method
-(``__getnewargs_ex__`` ?) and a new opcode (NEWOBJEX ?) are needed.
+Currently, classes whose __new__ mandates the use of keyword-only
+arguments can not be pickled (or, rather, unpickled) [3]_. Both a new
+special method (``__getnewargs_ex__`` ?) and a new opcode (NEWOBJEX ?)
+are needed.
Serializing more callable objects
---------------------------------
-Currently, only module-global functions are serializable. Multiprocessing
-has custom support for pickling other callables such as bound methods [4]_.
-This support could be folded in the protocol, and made more efficient
-through a new GETATTR opcode.
+Currently, only module-global functions are serializable.
+Multiprocessing has custom support for pickling other callables such
+as bound methods [4]_. This support could be folded in the protocol,
+and made more efficient through a new GETATTR opcode.
Serializing "pseudo-global" objects
-----------------------------------
-Objects which are not module-global, but should be treated in a similar
-fashion -- such as unbound methods [5]_ or nested classes -- cannot currently
-be pickled (or, rather, unpickled) because the pickle protocol does not
-correctly specify how to retrieve them. One solution would be through the
-adjunction of a ``__namespace__`` (or ``__qualname__``) to all class and
-function objects, specifying the full "path" by which they can be retrieved.
-For globals, this would generally be ``"{}.{}".format(obj.__module__, obj.__name__)``.
-Then a new opcode can resolve that path and push the object on the stack,
+Objects which are not module-global, but should be treated in a
+similar fashion -- such as unbound methods [5]_ or nested classes --
+cannot currently be pickled (or, rather, unpickled) because the pickle
+protocol does not correctly specify how to retrieve them. One
+solution would be through the adjunction of a ``__namespace__`` (or
+``__qualname__``) to all class and function objects, specifying the
+full "path" by which they can be retrieved. For globals, this would
+generally be ``"{}.{}".format(obj.__module__, obj.__name__)``. Then a
+new opcode can resolve that path and push the object on the stack,
similarly to the GLOBAL opcode.
Binary encoding for all opcodes
-------------------------------
-The GLOBAL opcode, which is still used in protocol 3, uses the so-called
-"text" mode of the pickle protocol, which involves looking for newlines
-in the pickle stream. Looking for newlines is difficult to optimize on
-a non-seekable stream, and therefore a new version of GLOBAL (BINGLOBAL?)
-could use a binary encoding instead.
+The GLOBAL opcode, which is still used in protocol 3, uses the
+so-called "text" mode of the pickle protocol, which involves looking
+for newlines in the pickle stream. Looking for newlines is difficult
+to optimize on a non-seekable stream, and therefore a new version of
+GLOBAL (BINGLOBAL?) could use a binary encoding instead.
-It seems that all other opcodes emitted when using protocol 3 already use
-binary encoding.
+It seems that all other opcodes emitted when using protocol 3 already
+use binary encoding.
Better string encoding
----------------------
-Short str objects currently have their length coded as a 4-bytes integer,
-which is wasteful. A specific opcode with a 1-byte length would make
-many pickles smaller.
-
+Short str objects currently have their length coded as a 4-bytes
+integer, which is wasteful. A specific opcode with a 1-byte length
+would make many pickles smaller.
Acknowledgments
@@ -133,6 +137,7 @@
.. [5] "pickle should support methods":
http://bugs.python.org/issue9276
+
Copyright
=========
diff --git a/pep-3155.txt b/pep-3155.txt
--- a/pep-3155.txt
+++ b/pep-3155.txt
@@ -15,12 +15,13 @@
Rationale
=========
-Python's introspection facilities have long had poor support for nested
-classes. Given a class object, it is impossible to know whether it was
-defined inside another class or at module top-level; and, if the former,
-it is also impossible to know in which class it was defined. While
-use of nested classes is often considered poor style, the only reason
-for them to have second class introspection support is a lousy pun.
+Python's introspection facilities have long had poor support for
+nested classes. Given a class object, it is impossible to know
+whether it was defined inside another class or at module top-level;
+and, if the former, it is also impossible to know in which class it
+was defined. While use of nested classes is often considered poor
+style, the only reason for them to have second class introspection
+support is a lousy pun.
Python 3 adds insult to injury by dropping what was formerly known as
unbound methods. In Python 2, given the following definition::
@@ -49,23 +50,24 @@
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__']
-This limits again the introspection capabilities available to the user.
-It can produce actual issues when porting software to Python 3, for example
-Twisted Core where the issue of introspecting method objects came up
-several times. It also limits pickling support [1]_.
+This limits again the introspection capabilities available to the
+user. It can produce actual issues when porting software to Python 3,
+for example Twisted Core where the issue of introspecting method
+objects came up several times. It also limits pickling support [1]_.
Proposal
========
-This PEP proposes the addition of a ``__qname__`` attribute to functions
-and classes. For top-level functions and classes, the ``__qname__``
-attribute is equal to the ``__name__`` attribute. For nested classed,
-methods, and nested functions, the ``__qname__`` attribute contains a
-dotted path leading to the object from the module top-level.
+This PEP proposes the addition of a ``__qname__`` attribute to
+functions and classes. For top-level functions and classes, the
+``__qname__`` attribute is equal to the ``__name__`` attribute. For
+nested classed, methods, and nested functions, the ``__qname__``
+attribute contains a dotted path leading to the object from the module
+top-level.
-The repr() and str() of functions and classes is modified to use ``__qname__``
-rather than ``__name__``.
+The repr() and str() of functions and classes is modified to use
+``__qname__`` rather than ``__name__``.
Example with nested classes
---------------------------
@@ -100,10 +102,10 @@
Limitations
===========
-With nested functions (and classes defined inside functions), the dotted
-path will not be walkable programmatically as a function's namespace is not
-available from the outside. It will still be more helpful to the human
-reader than the bare ``__name__``.
+With nested functions (and classes defined inside functions), the
+dotted path will not be walkable programmatically as a function's
+namespace is not available from the outside. It will still be more
+helpful to the human reader than the bare ``__name__``.
As the ``__name__`` attribute, the ``__qname__`` attribute is computed
statically and it will not automatically follow rebinding.
@@ -115,6 +117,7 @@
.. [1] "pickle should support methods":
http://bugs.python.org/issue9276
+
Copyright
=========
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list