[Python-checkins] peps: Add PEP 408: __preview__ namespace

[nick.coghlan]

http://hg.python.org/peps/rev/53d5bdd95cf5
changeset:   4017:53d5bdd95cf5
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Fri Jan 27 20:50:50 2012 +1000
summary:
  Add PEP 408: __preview__ namespace

files:
  pep-0408.txt |  291 +++++++++++++++++++++++++++++++++++++++
  1 files changed, 291 insertions(+), 0 deletions(-)


diff --git a/pep-0408.txt b/pep-0408.txt
new file mode 100644
--- /dev/null
+++ b/pep-0408.txt
@@ -0,0 +1,291 @@
+PEP: 408
+Title: Standard library __preview__ package
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan at gmail.com>,
+        Eli Bendersky <eliben at gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 2012-01-07
+Python-Version: 3.3
+Post-History: 2012-01-27
+
+
+Abstract
+========
+
+The process of including a new module into the Python standard library is
+hindered by the API lock-in and promise of backward compatibility implied by
+a module being formally part of Python.  This PEP proposes a transitional
+state for modules - inclusion in a special ``__preview__`` package for the
+duration of a minor release (roughly 18 months) prior to full acceptance into
+the standard library.  On one hand, this state provides the module with the
+benefits of being formally part of the Python distribution.  On the other hand,
+the core development team explicitly states that no promises are made with
+regards to the module's eventual full inclusion into the standard library,
+or to the stability of its API, which may change for the next release.
+
+
+Proposal - the __preview__ package
+==================================
+
+Whenever the Python core development team decides that a new module should be
+included into the standard library, but isn't entirely sure about whether the
+module's API is optimal, the module can be placed in a special package named
+``__preview__`` for a single minor release.
+
+In the next minor release, the module may either be "graduated" into the
+standard library (and occupy its natural place within its namespace, leaving the
+``__preview__`` package), or be rejected and removed entirely from the Python
+source tree.  If the module ends up graduating into the standard library after
+spending a minor release in ``__preview__``, its API may be changed according
+to accumulated feedback.  The core development team explicitly makes no
+guarantees about API stability and backward compatibility of modules in
+``__preview__``.
+
+Entry into the ``__preview__`` package marks the start of a transition of the
+module into the standard library.  It means that the core development team
+assumes responsibility of the module, similarly to any other module in the
+standard library.
+
+
+Which modules should go through ``__preview__``
+-----------------------------------------------
+
+We expect most modules proposed for addition into the Python standard library
+to go through a minor release in ``__preview__``. There may, however, be some
+exceptions, such as modules that use a pre-defined API (for example ``lzma``,
+which generally follows the API of the existing ``bz2`` module), or modules
+with an API that has wide acceptance in the Python development community.
+
+In any case, modules that are proposed to be added to the standard library,
+whether via ``__preview__`` or directly, must fulfill the acceptance conditions
+set by PEP 2.
+
+It is important to stress that the aim of of this proposal is not to make the
+process of adding new modules to the standard library more difficult.  On the
+contrary, it tries to provide a means to add *more* useful libraries.  Modules
+which are obvious candidates for entry can be added as before.  Modules which
+due to uncertainties about the API could be stalled for a long time now have
+a means to still be distributed with Python, via an incubation period in the
+``__preview__`` package.
+
+
+Criteria for "graduation"
+-------------------------
+
+In principle, most modules in the ``__preview__`` package should eventually
+graduate to the stable standard library.  Some reasons for not graduating are:
+
+* The module may prove to be unstable or fragile, without sufficient developer
+  support to maintain it.
+* A much better alternative module may be found during the preview release
+
+Essentially, the decision will be made by the core developers on a per-case
+basis.  The point to emphasize here is that a module's appearance in the
+``__preview__`` package in some release does not guarantee it will continue
+being part of Python in the next release.
+
+
+Example
+-------
+
+Suppose the ``example`` module is a candidate for inclusion in the standard
+library, but some Python developers aren't convinced that it presents the best
+API for the problem it intends to solve.  The module can then be added to the
+``__preview__`` package in release ``3.X``, importable via::
+
+    from __preview__ import example
+
+Assuming the module is then promoted to the the standard library proper in
+release ``3.X+1``, it will be moved to a permanent location in the library::
+
+    import example
+
+And importing it from ``__preview__`` will no longer work.
+
+
+Rationale
+=========
+
+Benefits for the core development team
+--------------------------------------
+
+Currently, the core developers are really reluctant to add new interfaces to
+the standard library.  This is because as soon as they're published in a
+release, API design mistakes get locked in due to backward compatibility
+concerns.
+
+By gating all major API additions through some kind of a preview mechanism
+for a full release, we get one full release cycle of community feedback
+before we lock in the APIs with our standard backward compatibility guarantee.
+
+We can also start integrating preview modules with the rest of the standard
+library early, so long as we make it clear to packagers that the preview
+modules should not be considered optional.  The only difference between preview
+APIs and the rest of the standard library is that preview APIs are explicitly
+exempted from the usual backward compatibility guarantees.
+
+Essentially, the ``__preview__`` package is intended to lower the risk of
+locking in minor API design mistakes for extended periods of time.  Currently,
+this concern can block new additions, even when the core development team
+consensus is that a particular addition is a good idea in principle.
+
+
+Benefits for end users
+----------------------
+
+For future end users, the broadest benefit lies in a better "out-of-the-box"
+experience - rather than being told "oh, the standard library tools for task X
+are horrible, download this 3rd party library instead", those superior tools
+are more likely to be just be an import away.
+
+For environments where developers are required to conduct due diligence on
+their upstream dependencies (severely harming the cost-effectiveness of, or
+even ruling out entirely, much of the material on PyPI), the key benefit lies
+in ensuring that anything in the ``__preview__`` package is clearly under
+python-dev's aegis from at least the following perspectives:
+
+* Licensing:  Redistributed by the PSF under a Contributor Licensing Agreement.
+* Documentation: The documentation of the module is published and organized via
+  the standard Python documentation tools (i.e. ReST source, output generated
+  with Sphinx and published on http://docs.python.org).
+* Testing: The module test suites are run on the python.org buildbot fleet
+  and results published via http://www.python.org/dev/buildbot.
+* Issue management: Bugs and feature requests are handled on
+  http://bugs.python.org
+* Source control: The master repository for the software is published
+  on http://hg.python.org.
+
+
+Candidates for inclusion into __preview__
+=========================================
+
+For Python 3.3, there are a number of clear current candidates:
+
+* ``regex`` (http://pypi.python.org/pypi/regex)
+* ``daemon`` (PEP 3143)
+* ``ipaddr`` (PEP 3144)
+
+Other possible future use cases include:
+
+* Improved HTTP modules (e.g. ``requests``)
+* HTML 5 parsing support (e.g. ``html5lib``)
+* Improved URL/URI/IRI parsing
+* A standard image API (PEP 368)
+* Encapsulation of the import state (PEP 368)
+* Standard event loop API (PEP 3153)
+* A binary version of WSGI for Python 3 (e.g. PEP 444)
+* Generic function support (e.g. ``simplegeneric``)
+
+
+Relationship with PEP 407
+=========================
+
+PEP 407 proposes a change to the core Python release cycle to permit interim
+releases every 6 months (perhaps limited to standard library updates). If
+such a change to the release cycle is made, the following policy for the
+``__preview__`` namespace is suggested:
+
+* For long term support releases, the ``__preview__`` namespace would always
+  be empty.
+* New modules would be accepted into the ``__preview__`` namespace only in
+  interim releases that immediately follow a long term support release.
+* All modules added will either be migrated to their final location in the
+  standard library or dropped entirely prior to the next long term support
+  release.
+
+
+Rejected alternatives and variations
+====================================
+
+
+Using ``__future__``
+--------------------
+
+Python already has a "forward-looking" namespace in the form of the
+``__future__`` module, so it's reasonable to ask why that can't be re-used for
+this new purpose.
+
+There are two reasons why doing so not appropriate:
+
+1. The ``__future__`` module is actually linked to a separate compiler
+directives feature that can actually change the way the Python interpreter
+compiles a module.  We don't want that for the preview package - we just want
+an ordinary Python package.
+
+2. The ``__future__`` module comes with an express promise that names will be
+maintained in perpetuity, long after the associated features have become the
+compiler's default behaviour.  Again, this is precisely the opposite of what is
+intended for the preview package - it is almost certain that all names added to
+the preview will be removed at some point, most likely due to their being moved
+to a permanent home in the standard library, but also potentially due to their
+being reverted to third party package status (if community feedback suggests the
+proposed addition is irredeemably broken).
+
+
+Versioning the package
+----------------------
+
+One proposed alternative [1]_ was to add explicit versioning to the
+``__preview__`` package, i.e. ``__preview34__``.  We think that it's better to
+simply define that a module being in ``__preview__`` in Python 3.X will either
+graduate to the normal standard library namespace in Python 3.X+1 or will
+disappear from the Python source tree altogether.  Versioning the ``_preview__``
+package complicates the process and does not align well with the main intent of
+this proposal.
+
+
+Using a package name without leading and trailing underscores
+-------------------------------------------------------------
+
+It was proposed [1]_ to use a package name like ``preview`` or ``exp``, instead
+of ``__preview__``.  This was rejected in the discussion due to the special
+meaning a "dunder" (double-underscore) package name (a name *with* leading and
+trailing underscores) conveys in Python.  Besides, a non-dunder name indicates
+stability, which is not the intention of the ``__preview__`` package.
+
+
+Preserving pickle compatibility
+-------------------------------
+
+A pickled class instance based on a module in ``__preview__`` in release 3.X
+won't be unpickle-able in release 3.X+1, where the module won't be in
+``__preview__``.  Special code may be added to make this work, but this goes
+against the intent of this proposal, since it implies backward compatibility.
+Therefore, this PEP does not propose to preserve pickle compatibility.
+
+
+Credits
+=======
+
+Dj Gilcrease initially proposed the idea of having a ``__preview__`` package
+in Python [2]_.  Although his original proposal uses the name
+``__experimental__``, we feel that ``__preview__`` conveys the meaning of this
+package in a better way.
+
+
+References
+==========
+
+.. [#] Discussed in this thread:
+       http://mail.python.org/pipermail/python-ideas/2012-January/013246.html
+
+.. [#] http://mail.python.org/pipermail/python-ideas/2011-August/011278.html
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End:

-- 
Repository URL: http://hg.python.org/peps