http://hg.python.org/peps/rev/53d5bdd95cf5 changeset: 4017:53d5bdd95cf5 user: Nick Coghlan <ncoghlan@gmail.com> date: Fri Jan 27 20:50:50 2012 +1000 summary: Add PEP 408: __preview__ namespace files: pep-0408.txt | 291 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 291 insertions(+), 0 deletions(-) diff --git a/pep-0408.txt b/pep-0408.txt new file mode 100644 --- /dev/null +++ b/pep-0408.txt @@ -0,0 +1,291 @@ +PEP: 408 +Title: Standard library __preview__ package +Version: $Revision$ +Last-Modified: $Date$ +Author: Nick Coghlan <ncoghlan@gmail.com>, + Eli Bendersky <eliben@gmail.com> +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 2012-01-07 +Python-Version: 3.3 +Post-History: 2012-01-27 + + +Abstract +======== + +The process of including a new module into the Python standard library is +hindered by the API lock-in and promise of backward compatibility implied by +a module being formally part of Python. This PEP proposes a transitional +state for modules - inclusion in a special ``__preview__`` package for the +duration of a minor release (roughly 18 months) prior to full acceptance into +the standard library. On one hand, this state provides the module with the +benefits of being formally part of the Python distribution. On the other hand, +the core development team explicitly states that no promises are made with +regards to the module's eventual full inclusion into the standard library, +or to the stability of its API, which may change for the next release. + + +Proposal - the __preview__ package +================================== + +Whenever the Python core development team decides that a new module should be +included into the standard library, but isn't entirely sure about whether the +module's API is optimal, the module can be placed in a special package named +``__preview__`` for a single minor release. + +In the next minor release, the module may either be "graduated" into the +standard library (and occupy its natural place within its namespace, leaving the +``__preview__`` package), or be rejected and removed entirely from the Python +source tree. If the module ends up graduating into the standard library after +spending a minor release in ``__preview__``, its API may be changed according +to accumulated feedback. The core development team explicitly makes no +guarantees about API stability and backward compatibility of modules in +``__preview__``. + +Entry into the ``__preview__`` package marks the start of a transition of the +module into the standard library. It means that the core development team +assumes responsibility of the module, similarly to any other module in the +standard library. + + +Which modules should go through ``__preview__`` +----------------------------------------------- + +We expect most modules proposed for addition into the Python standard library +to go through a minor release in ``__preview__``. There may, however, be some +exceptions, such as modules that use a pre-defined API (for example ``lzma``, +which generally follows the API of the existing ``bz2`` module), or modules +with an API that has wide acceptance in the Python development community. + +In any case, modules that are proposed to be added to the standard library, +whether via ``__preview__`` or directly, must fulfill the acceptance conditions +set by PEP 2. + +It is important to stress that the aim of of this proposal is not to make the +process of adding new modules to the standard library more difficult. On the +contrary, it tries to provide a means to add *more* useful libraries. Modules +which are obvious candidates for entry can be added as before. Modules which +due to uncertainties about the API could be stalled for a long time now have +a means to still be distributed with Python, via an incubation period in the +``__preview__`` package. + + +Criteria for "graduation" +------------------------- + +In principle, most modules in the ``__preview__`` package should eventually +graduate to the stable standard library. Some reasons for not graduating are: + +* The module may prove to be unstable or fragile, without sufficient developer + support to maintain it. +* A much better alternative module may be found during the preview release + +Essentially, the decision will be made by the core developers on a per-case +basis. The point to emphasize here is that a module's appearance in the +``__preview__`` package in some release does not guarantee it will continue +being part of Python in the next release. + + +Example +------- + +Suppose the ``example`` module is a candidate for inclusion in the standard +library, but some Python developers aren't convinced that it presents the best +API for the problem it intends to solve. The module can then be added to the +``__preview__`` package in release ``3.X``, importable via:: + + from __preview__ import example + +Assuming the module is then promoted to the the standard library proper in +release ``3.X+1``, it will be moved to a permanent location in the library:: + + import example + +And importing it from ``__preview__`` will no longer work. + + +Rationale +========= + +Benefits for the core development team +-------------------------------------- + +Currently, the core developers are really reluctant to add new interfaces to +the standard library. This is because as soon as they're published in a +release, API design mistakes get locked in due to backward compatibility +concerns. + +By gating all major API additions through some kind of a preview mechanism +for a full release, we get one full release cycle of community feedback +before we lock in the APIs with our standard backward compatibility guarantee. + +We can also start integrating preview modules with the rest of the standard +library early, so long as we make it clear to packagers that the preview +modules should not be considered optional. The only difference between preview +APIs and the rest of the standard library is that preview APIs are explicitly +exempted from the usual backward compatibility guarantees. + +Essentially, the ``__preview__`` package is intended to lower the risk of +locking in minor API design mistakes for extended periods of time. Currently, +this concern can block new additions, even when the core development team +consensus is that a particular addition is a good idea in principle. + + +Benefits for end users +---------------------- + +For future end users, the broadest benefit lies in a better "out-of-the-box" +experience - rather than being told "oh, the standard library tools for task X +are horrible, download this 3rd party library instead", those superior tools +are more likely to be just be an import away. + +For environments where developers are required to conduct due diligence on +their upstream dependencies (severely harming the cost-effectiveness of, or +even ruling out entirely, much of the material on PyPI), the key benefit lies +in ensuring that anything in the ``__preview__`` package is clearly under +python-dev's aegis from at least the following perspectives: + +* Licensing: Redistributed by the PSF under a Contributor Licensing Agreement. +* Documentation: The documentation of the module is published and organized via + the standard Python documentation tools (i.e. ReST source, output generated + with Sphinx and published on http://docs.python.org). +* Testing: The module test suites are run on the python.org buildbot fleet + and results published via http://www.python.org/dev/buildbot. +* Issue management: Bugs and feature requests are handled on + http://bugs.python.org +* Source control: The master repository for the software is published + on http://hg.python.org. + + +Candidates for inclusion into __preview__ +========================================= + +For Python 3.3, there are a number of clear current candidates: + +* ``regex`` (http://pypi.python.org/pypi/regex) +* ``daemon`` (PEP 3143) +* ``ipaddr`` (PEP 3144) + +Other possible future use cases include: + +* Improved HTTP modules (e.g. ``requests``) +* HTML 5 parsing support (e.g. ``html5lib``) +* Improved URL/URI/IRI parsing +* A standard image API (PEP 368) +* Encapsulation of the import state (PEP 368) +* Standard event loop API (PEP 3153) +* A binary version of WSGI for Python 3 (e.g. PEP 444) +* Generic function support (e.g. ``simplegeneric``) + + +Relationship with PEP 407 +========================= + +PEP 407 proposes a change to the core Python release cycle to permit interim +releases every 6 months (perhaps limited to standard library updates). If +such a change to the release cycle is made, the following policy for the +``__preview__`` namespace is suggested: + +* For long term support releases, the ``__preview__`` namespace would always + be empty. +* New modules would be accepted into the ``__preview__`` namespace only in + interim releases that immediately follow a long term support release. +* All modules added will either be migrated to their final location in the + standard library or dropped entirely prior to the next long term support + release. + + +Rejected alternatives and variations +==================================== + + +Using ``__future__`` +-------------------- + +Python already has a "forward-looking" namespace in the form of the +``__future__`` module, so it's reasonable to ask why that can't be re-used for +this new purpose. + +There are two reasons why doing so not appropriate: + +1. The ``__future__`` module is actually linked to a separate compiler +directives feature that can actually change the way the Python interpreter +compiles a module. We don't want that for the preview package - we just want +an ordinary Python package. + +2. The ``__future__`` module comes with an express promise that names will be +maintained in perpetuity, long after the associated features have become the +compiler's default behaviour. Again, this is precisely the opposite of what is +intended for the preview package - it is almost certain that all names added to +the preview will be removed at some point, most likely due to their being moved +to a permanent home in the standard library, but also potentially due to their +being reverted to third party package status (if community feedback suggests the +proposed addition is irredeemably broken). + + +Versioning the package +---------------------- + +One proposed alternative [1]_ was to add explicit versioning to the +``__preview__`` package, i.e. ``__preview34__``. We think that it's better to +simply define that a module being in ``__preview__`` in Python 3.X will either +graduate to the normal standard library namespace in Python 3.X+1 or will +disappear from the Python source tree altogether. Versioning the ``_preview__`` +package complicates the process and does not align well with the main intent of +this proposal. + + +Using a package name without leading and trailing underscores +------------------------------------------------------------- + +It was proposed [1]_ to use a package name like ``preview`` or ``exp``, instead +of ``__preview__``. This was rejected in the discussion due to the special +meaning a "dunder" (double-underscore) package name (a name *with* leading and +trailing underscores) conveys in Python. Besides, a non-dunder name indicates +stability, which is not the intention of the ``__preview__`` package. + + +Preserving pickle compatibility +------------------------------- + +A pickled class instance based on a module in ``__preview__`` in release 3.X +won't be unpickle-able in release 3.X+1, where the module won't be in +``__preview__``. Special code may be added to make this work, but this goes +against the intent of this proposal, since it implies backward compatibility. +Therefore, this PEP does not propose to preserve pickle compatibility. + + +Credits +======= + +Dj Gilcrease initially proposed the idea of having a ``__preview__`` package +in Python [2]_. Although his original proposal uses the name +``__experimental__``, we feel that ``__preview__`` conveys the meaning of this +package in a better way. + + +References +========== + +.. [#] Discussed in this thread: + http://mail.python.org/pipermail/python-ideas/2012-January/013246.html + +.. [#] http://mail.python.org/pipermail/python-ideas/2011-August/011278.html + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: http://hg.python.org/peps