[Python-checkins] peps: Introduce PEP 0497 from Ed Schofield

chris.angelico python-checkins at python.org
Wed Aug 5 18:02:45 CEST 2015


https://hg.python.org/peps/rev/bc2cb060f261
changeset:   5921:bc2cb060f261
user:        Chris Angelico <rosuav at gmail.com>
date:        Thu Aug 06 02:02:39 2015 +1000
summary:
  Introduce PEP 0497 from Ed Schofield

files:
  pep-0497.txt |  313 +++++++++++++++++++++++++++++++++++++++
  1 files changed, 313 insertions(+), 0 deletions(-)


diff --git a/pep-0497.txt b/pep-0497.txt
new file mode 100644
--- /dev/null
+++ b/pep-0497.txt
@@ -0,0 +1,313 @@
+PEP: 497
+Title: A standard mechanism for backward compatibility
+Version: $Revision$
+Last-Modified: $Date$
+Author:	Ed Schofield <ed at pythoncharmers.com>
+Status:	Draft
+Type: Process
+Created: 04-Aug-2015
+Content-Type: text/x-rst
+
+
+Scope
+=====
+
+This PEP is complementary to PEPs 5, 236, and 387, and shares similar
+goals.
+
+This PEP explains the need for an additional compatibility mechanism
+in support of PEP 5, "Guidelines for Language Evolution". PEP 236,
+"Back to the __future__", introduced a mechanism for forward
+compatibility in support of PEP 5 but noted that a new mechanism for
+backward compatibility was outside the scope of that PEP. A related
+PEP (in progress) introduces such a mechanism for backward
+compatibility.
+
+PEP 5, "Guidelines for Language Evolution", notes that "This PEP [PEP 5]
+does not replace or preclude other compatibility strategies such as
+dynamic loading of backwards-compatible parsers."
+
+
+Context
+=======
+
+From PEP 236: "From time to time, Python makes an incompatible change
+to the advertised semantics of core language constructs, or changes
+their accidental (implementation-dependent) behavior in some way.
+While this is never done capriciously, and is always done with the aim
+of improving the language over the long term, over the short term it's
+contentious and disrupting. PEP 5, Guidelines for Language Evolution,
+suggests ways to ease the pain, and this PEP [236] introduces some
+machinery in support of that."
+
+Also from PEP 236: "The purpose of future_statement is to make life
+easier for people who keep current with the latest release in a timely
+fashion. We don't hate you if you don't, but your problems are much
+harder to solve, and somebody with those problems will need to write a
+PEP addressing them. future_statement is aimed at a different
+audience."
+
+
+The current situation
+=====================
+
+When an incompatible change to core language syntax or semantics is
+being made, Python currently provides the future_statement mechanism
+for providing forward compatibility until the release that enforces
+the new syntax or semantics, but provides no corresponding standard
+mechanism for providing backward compatibility after this release.
+
+
+Problem
+=======
+
+A consequence of this asymmetry is that, with respect to a breaking
+change, the older (pre-breaking) version of the Python interpreter is
+more capable than the newer (breaking) version; the older interpreter
+can use both code designed prior to the change and newer code, whereas
+the newer interpreter is only capable of using code that has been
+upgraded to support the changed feature.
+
+As an example, consider the changes to the division operator
+introduced in PEP 238 in 2001, soon after PEP 236 introduced the
+future_statement mechanism. PEP 238 outlines a suite of useful
+forward-compatibility mechanisms for "true division" in the Python 2.x
+series but omits to include any backward-compatibility mechanisms for
+after "true division" was first enforced in Python 3.0. Python versions
+since 3.0 do not provide a backward compatibility mechanism such as
+``from __past__ import division`` for code that expects the old
+"classic division" semantics, whereas Python versions prior to 3.0 do
+support both "classic division" code and also forward compatibility
+with code expecting "true division". A further consequence of this is
+that the "most compatible" interpreter with respect to the variety of
+division-related Python code in the wild is Python 2.7, the version
+before the breaking change was first enforced.
+
+
+Backward compatibility as enabler for "downhill upgrades"
+=========================================================
+
+In contrast to this situation, newer versions of application software
+such as office suites tend to be more capable than earlier versions
+with respect to their support for loading different versions of their
+data file formats. The pattern is usually that the newer application
+versions can transparently load data from either their newer or their
+older data formats, and that the newer version defaults to saving data
+in the newer format. Newer application software versions tend to be
+backward-compatible by default. Forward compatibility is relatively
+rare.
+
+This policy puts the user of the newer application software at an
+advantage over the user of the older software, which is usually
+incapable of loading data in the newer format. Sometimes it is
+possible for a user of a newer software application version to export
+data in an older version by choosing this option explicitly. In these
+cases, the forward-compatibility this enables may or may not be
+perfect; some features may be missing or the results may be otherwise
+suboptimal. Upgrading is therefore easy, whereas downgrading is
+harder.
+
+The emergent behaviour over many users from such a policy of new
+attractive features plus backward compatibility features is that a
+natural pressure builds up on each individual user to upgrade his or
+her own application version, and, the more other users an individual
+exchanges data files with, the more acute this pressure becomes.
+
+
+Proposal - part 1
+=================
+
+This PEP makes two specific, related proposals. The first is that:
+
+    PEP 5 be augmented with a 6th step in the section "Steps for
+    Introducing Backwards-Incompatible Features" to indicate that, when an
+    incompatible change to core language syntax or semantics is being
+    made, Python-dev's policy is to prefer and expect that, wherever
+    possible, a mechanism for backward compatibility be considered and
+    provided for future Python versions after the breaking change is
+    adopted by default, in addition to any mechanisms proposed for forward
+    compatibility such as new future_statements. Furthermore, PEP 387,
+    "Backwards Compatibility Policy" (if accepted) would be
+    augmented with the same 6th step.
+
+
+Example
+~~~~~~~
+
+As an example of how this PEP is to be applied, if the latest revision
+of the "true division" PEP (238) were proposed today, it would be
+considered incomplete. PEP 238 notes the "severe backwards
+compatibility issues" raised by the proposal and describes several
+measures for forward compatibility in the Abstract and API Changes
+sections. It also mentions some backward compatibility ideas raised on
+c.l.py, including "Use ``from __past__ import division`` to use
+classic division semantics in a module", but it does not put forward
+any backward compatibility plan as part of the proposal.
+
+If this PEP is accepted, it would be expected that a proposal such as
+PEP 238, because of its large-scale compatibility implications, would
+also be accompanied by a backward compatibility plan that enables
+users of future Python versions after the breaking change has come
+into effect to re-enable the classic division behaviour easily in
+their code.
+
+
+Proposal - part 2
+=================
+
+The second proposal is that:
+
+    Python provide a standard backward compatibility mechanism in
+    parallel to the ``__future__`` module mechanism for forward
+    compatibility.
+
+For reference, this document will refer to this as a "``__past__``"
+mechanism hereon, although it need not have all the characteristics
+of the ``__future__`` module and ``future_statement`` mechanism.
+
+The specific form and implementation of the ``__past__`` mechanism is
+the subject of a separate PEP (in progress).  However, this PEP
+recommends that this ``__past__`` mechanism be designed to meet
+similar criteria to those outlined in PEP 296 for ``__future__``.
+Specifically:
+
+a. It should enable individual modules to specify obsolete behaviours
+to re-enable from older Python versions on a module-by-module basis.
+
+b. It should be flexible enough for both Python 3.6+ and point
+releases of earlier versions to reintroduce backward compatibility
+with older Python syntax or semantics for user modules that invoke the
+``__past__`` mechanism.
+
+c. It should be possible to run older code augmented to invoke
+``__past__`` behaviours on older Python versions such as 2.x that have
+no knowledge of the specific ``__past__`` features invoked, or even
+that the ``__past__`` mechanism for backward-compatibility exists.
+
+
+Counter-examples
+~~~~~~~~~~~~~~~~
+
+Some implementations of ``__past__`` mechanisms that would violate
+these criteria are:
+
+a. Import hooks. These would normally fail to work on a
+module-by-module basis; instead they apply recursively to all new
+modules imported from within a module.
+
+b. A new piece of syntax or new semantics for Python 3.6 that is
+incompatible with prior versions.
+
+c. A function added in Python 3.6 to a module in the Python standard
+library that exists under the same name in prior Python versions.
+
+
+Benefits
+========
+
+The benefit to Python-dev of adopting this proposal is that future
+backward-incompatible changes can be less disruptive if these changes
+each have a corresponding ``__past__`` feature that has been
+implemented and can be invoked easily by users of future Python
+versions. This can help the language to evolve more quickly and more
+effectively to correct for design mistakes.
+
+The benefit to conservative users is obvious: they can add support for
+the latest shiny compatibility-breaking Python version to their code
+merely by adding a ``__past__`` incantation (perhaps a single line) to
+each module, and that this can be automated. They can then upgrade
+their interpreter to the latest version and gain access to the latest
+shiny Python features.
+
+The benefit to the community is that, if ten thousand users rely on
+package XYZ, and package XYZ can trivially add support for the latest
+Python version, those ten thousand users can also upgrade to the
+latest Python version quickly, without being held back waiting for
+package XYZ to do this.
+
+
+Questions and answers
+=====================
+
+Q1: Does this PEP require that Python keep two possible sets of semantics
+for each backward-incompatible feature forever?
+
+A1: Definitely not. Legacy features can still be phased out when
+appropriate -- that is, when the majority of the user-base has
+migrated to the newer Python version. This PEP merely proposes to
+shift the emphasis of the development effort directed at compatibility
+from 100% forwards to at least 50% backwards. Backwards compatibility
+is the more powerful of the two concepts for allowing a user-base to
+adopt the latest Python interpreter version.
+
+Notice that it has been a long time since most users have cared about
+backwards compatibility for non-nested scopes, because most users have
+moved comfortably past Python 2.1.
+
+Q2: But Python-dev is already overwhelmed and doesn't have the
+bandwidth to implement / maintain the additional complexity!
+
+A2: Python-dev can ask the community of developers to step up and
+maintain backward compatibility in Python for legacy language features
+they care about. When the community stops caring about a particular
+obsolete behaviour, Python-dev can stop caring too. 
+
+The ``__past__`` mechanism could possibly be designed to be extensible
+by the community, e.g.  as a standard but "blessed" PyPI package, to
+reduce the load on the core developers.
+
+Q3: Won't backward compatibility features lead to lots of cruft and
+bloat and baggage in Python?
+
+A3: Not necessarily. First, proposals for new compatibility-breaking
+features in Python could be evaluated partly on the simplicity and
+maintainability of the implementation of their associated ``__past__``
+feature up-front.
+
+Second, some old features are simple to provide backward compatibility
+for. Consider the "classic division" behaviour before Python 3.0. The
+``python-future`` project contains a compatible implementation of
+classic division in the function ``future.utils.old_div``:
+
+::
+
+    def old_div(a, b):
+        """
+        Equivalent to ``a / b`` on Python 2 without ``from __future__ import
+        division``.
+        """
+        if isinstance(a, numbers.Integral) and isinstance(b, numbers.Integral):
+            return a // b
+        else:
+            return a / b
+
+
+Bundling such a function with Python 3.x versions, together with
+a simple mechanism to invoke it for every appearance of ``a
+/ b`` after an appropriate ``__past__`` invocation, need not be
+onerous.
+
+
+Q4: What about performance? Won't the performance of newer Python
+versions suffer under the weight of legacy features?
+
+A4: This can be evaluated on a case-by-case basis. The major potential
+concern is that the performance with the new default behaviour does
+not suffer unduly because of the presence of the legacy option. The
+performance under the influence of the ``__past__`` invocation is of
+secondary importance.
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8

-- 
Repository URL: https://hg.python.org/peps


More information about the Python-checkins mailing list