[python-committers] Deprecation Policy PEP

Ezio Melotti ezio.melotti at gmail.com
Fri Jan 29 12:11:24 EST 2016 

Hi,
in the past I suggested to document our deprecation policy on a PEP.
Since the issue came up again on a few issues on the tracker and on
#python-dev, I adapted the content of my previous email and wrote a
PEP.

Attached you can find the full text, including references to the
previous discussions on python-dev and related issues.

Best Regards,
Ezio Melotti

---------

PEP: XXX
Title: Deprecation Policy
Version: $Revision$
Last-Modified: $Date$
Author: Ezio Melotti <ezio.melotti at gmail.com>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 29-Jan-2016
Post-History:


Abstract
========

The goal of this PEP is to document when and how to deprecate
APIs in the Python language and in the standard library.


Rationale
=========

Python doesn't currently have a documented policy regarding
deprecations.  This leads to repeated discussions and
inconsistencies in how we handle and document deprecations
[#]_ [#]_ [#]_ [#]_ [#]_ [#]_ [#]_.


What and when to deprecate
==========================

* An API can be deprecated if a better alternative exists, if the
  implementation is incorrect or obsolete, or if the name or module
  has been changed.  If possible, an alternative should be provided.
* If the API is public, it must go through the deprecation
  process.  If the API is private or provisional, it might.
* The number of releases before an API is removed is decided
  on a case-by-case basis depending on widely used the API is,
  in which Python versions the alternative is available, which
  Python versions are currently supported by different operating
  systems, distros, and projects, and how easy it is to replace
  the API.
* In general it's better to be conservative, and if the API is
  deprecated in ``3.X``, it shouldn't be removed before ``3.X+2``.
  This should also take into account which Python versions are
  currently .


Porting from 2.x to 3.x
-----------------------

Some APIs that are available in Python 2.7 might get deprecated
in Python 3, and people that upgrade directly from 2.7 to 3.X might
not see any warnings.

In order to make porting code to 3.X easier:

* nothing that is available and not deprecated in 2.7 should be
  removed from Python 3 as long as 2.7 is officially supported;
* deprecation warnings can be backported to 2.7 and enabled
  using the ``-3`` flag;


Deprecation Warnings
====================

Python offers two kinds of deprecation warnings:

* ``PendingDeprecationWarning``
* ``DeprecationWarning``

Initially, only ``PendingDeprecationWarning``\ s were silenced by
default.  Starting from Python 2.7, ``DeprecationWarning``\ s
are also silenced by default [#]_ [#]_.

Since this distinction is no longer true:

* ``PendingDeprecationWarning`` should not be used
* ``DeprecationWarning`` will be used for all deprecations

``PendingDeprecationWarning`` won't be removed, and 3rd-party
projects are allowed to use it as they see fit.


Deprecation Progression
=======================

In the past, the following deprecation progression has been used:

1. in release ``3.X`` a ``PendingDeprecationWarning`` is added
2. in release ``3.X+1`` it becomes a ``DeprecationWarning``
3. in release ``3.X+2`` the API is removed

The warning were occasionally left for more than one release,
either intentionally or because no one remembered to update them.

Since ``PendingDeprecationWarning`` should no longer be used, this
can be simplified to:

1. in release ``3.X`` a ``DeprecationWarning`` is added
2. in release ``3.X+N`` the API is removed

``N`` can be ``>=1``, should be decided beforehand, and should be
documented explicitly.


Deprecation Process
===================

These are the steps required to deprecate and remove an API:

1. propose to deprecate an API on a tracker issue or on python-dev
   and decide in which version it will be removed.

2. attach to the issue a patch to deprecate the API that:

  * adds a ``DeprecationWarning`` to the code
  * adds the deprecation to the documentation
  * adds a test to verify that the warning is raised
  * possibly updates code/examples that uses the deprecated API

3. after review, apply the patch to the current in-development
   branch and close the issue.

4. attach to a separate issue a patch to remove the API that:

  * removes the API and the warning
  * removes the tests for the API and for the deprecation
  * removes the API documentation

5. once the designated branch is available, apply the patch and
   close the issue.


Deprecation Messages
====================

When a deprecated API is used, a ``DeprecationWarning`` should
be raised.  The message should mention that the API is
deprecated, and if possible it should indicate when it will be
removed and what should be used instead.

These messages are intended for developers, and are therefore
hidden by default to prevent end-users to see them.

These messages should be enabled by default in places where only
developers will see them, such as tests [#]_ and in the interactive
interpreter [#]_.


Documenting the deprecations
============================

* All deprecations should be documented in the docs, using the
  ``deprecated-removed`` directive.
* If an alternative is available, it should be mentioned, possibly
  including examples.
* Each "what's new" document should include either a section or a
  link to a separate document [#]_ listing all the new deprecations,
  the APIs that have been removed and possibly the planned
  removals for the upcoming releases.  This could be generated
  automatically with Sphinx.
* Simply removing the documentation for deprecated APIs is
  not acceptable, since people that see the API used in the
  code won't know if they can still use the API or not.


Deprecation warnings rendering
------------------------------

On one hand deprecation warnings should be clearly visible, on the
other hand we want to avoid riddling the docs with red boxes [#]_.

In order to find a balance between the two:

* Individual functions/methods/attributes should have a label next
  to the name to indicate the deprecation, and a more verbose
  description underneath.  The label will be red, the description
  doesn't have to [#]_.
* Modules and classes can use a red warning box.  If the
  documentation spans more than a single screen of text, additional
  warning labels can be added to the individual functions/methods.
* Links to deprecated objects should be marked differently.

This will require some tweak to the ``deprecated-removed``
directive, in order to produce different CSS classes for
modules/classes, functions/methods/attributes, and links.


Updating deprecated code
========================

As mentioned above, the error messages and documentation should
provide an alternative whenever possible.  When the alternative
is not straightforward, more complex examples or a new section
can added to the documentation to explain how to convert the code.

Another option is to write scripts that can automatically update
Python code to use the new alternative.  This requires:

1. writing a 3to3 tool similar to (and possibly based on) 2to3;
2. documenting clearly its API and providing several examples;
3. providing fixers for deprecated APIs that can be used to
   automatically update deprecated code;

This can be done as a GSoC project, and if done correctly, it will
provide an easy way for people to upgrade their codebases.


See also
========

* :pep:`387` -- Backwards Compatibility Policy
* :pep:`4` -- Deprecation of Standard Modules


References
==========

.. [#] Deprecation Policy
   (https://mail.python.org/pipermail/python-dev/2011-October/114199.html)

.. [#] Deprecation Policy
   (https://mail.python.org/pipermail/python-dev/2014-January/132069.html)

.. [#] deprecated in 3.2/3.3, should be removed in 3.5 or ???
   (https://bugs.python.org/issue13248)

.. [#] DeprecationWarning for PEP 479 (generator_stop)
   (http://bugs.python.org/issue26136)

.. [#] Deprecate threading.Thread.isDaemon etc
   (http://bugs.python.org/issue24203)

.. [#] Remove the Deprecated API in trace module
   (http://bugs.python.org/issue26069)

.. [#] Resurrect inspect.getargspec() in 3.6
   (http://bugs.python.org/issue25486)

.. [#] What's New in Python 2.7 -- Changes to the Handling of
   Deprecation Warnings (https://docs.python.org/3/whatsnew/2.7.html)

.. [#] Silence DeprecationWarning by default
   (https://bugs.python.org/issue7319)

.. [#] Enable warnings by default in unittest
   (https://bugs.python.org/issue10535)

.. [#] DeprecationWarnings should be visible by default in the
   interactive REPL (https://bugs.python.org/issue24294)

.. [#] Django has a "Django Deprecation Timeline" page
   (https://docs.djangoproject.com/en/dev/internals/deprecation/)

.. [#] Consistent documentation practices for security concerns
   and considerations (https://bugs.python.org/issue13515)

.. [#] Put "deprecated" warnings first
   (http://bugs.python.org/issue25467)


Copyright
=========

This document has been placed in the public domain.

More information about the python-committers mailing list