Hi, a couple of years ago I suggested to define and document our deprecation policy in this thread: https://mail.python.org/pipermail/python-dev/2011-October/114199.html I didn't receive many replies and eventually nothing was done. Lately the same issue came up on #python-dev and Larry and Nick suggested me to bring this up again. Nick also suggested to document our deprecation policy in PEP 5 (Guidelines for Language Evolution: http://www.python.org/dev/peps/pep-0005/ ). I'm including below the full text of the original email. Best Regards, Ezio Melotti ------------------------------- Hi, our current deprecation policy is not so well defined (see e.g. [0]), and it seems to me that it's something like: 1) deprecate something and add a DeprecationWarning; 2) forget about it after a while; 3) wait a few versions until someone notices it; 4) actually remove it; I suggest to follow the following process: 1) deprecate something and add a DeprecationWarning; 2) decide how long the deprecation should last; 3) use the deprecated-remove[1] directive to document it; 4) add a test that fails after the update so that we remember to remove it[2]; Other related issues: PendingDeprecationWarnings: * AFAIK the difference between PDW and DW is that PDW are silenced by default; * now DW are silence by default too, so there are no differences; * I therefore suggest we stop using it, but we can leave it around[3] (other projects might be using it for something different); Deprecation Progression: Before, we more or less used to deprecated in release X and remove in X+1, or add a PDW in X, DW in X+1, and remove it in X+2. I suggest we drop this scheme and just use DW until X+N, where N is
=1 and depends on what is being removed. We can decide to leave the DW for 2-3 versions before removing something widely used, or just deprecate in X and remove in X+1 for things that are less used.
Porting from 2.x to 3.x: Some people will update directly from 2.7 to 3.2 or even later versions (3.3, 3.4, ...), without going through earlier 3.x versions. If something is deprecated on 3.2 but not in 2.7 and then is removed in 3.3, people updating from 2.7 to 3.3 won't see any warning, and this will make the porting even more difficult. I suggest that: * nothing that is available and not deprecated in 2.7, will be removed until 3.x (x needs to be defined); * possibly we start backporting warnings to 2.7 so that they are visible while running with -3; Documenting the deprecations: In order to advertise the deprecations, they should be documented: * in their doc, using the deprecated-removed directive (and possibly not the 'deprecated' one); * in the what's new, possibly listing everything that is currently deprecated, and when it will be removed; Django seems to do something similar[4]. (Another thing I would like is a different rending for deprecated functions. Some part of the docs have a deprecation warning on the top of the section and the single functions look normal if you miss that. Also while linking to a deprecated function it would be nice to have it rendered with a different color or something similar.) Testing the deprecations: Tests that fail when a new release is made and the version number is bumped should be added to make sure we don't forget to remove it. The test should have a related issue with a patch to remove the deprecated function and the test. Setting the priority of the issue to release blocker or deferred blocker can be done in addition/instead, but that works well only when N == 1 (the priority could be updated for every release though). The tests could be marked with an expected failure to give some time after the release to remove them. All the deprecation-related tests might be added to the same file, or left in the test file of their module. Where to add this: Once we agree about the process we should write it down somewhere. Possible candidates are: * PEP387: Backwards Compatibility Policy[5] (it has a few lines about this); * a new PEP; * the devguide; I think having it in a PEP would be good, the devguide can then link to it. Best Regards, Ezio Melotti [0]: http://bugs.python.org/issue13248 [1]: deprecated-removed doesn't seem to be documented in the documenting doc, but it was added here: http://hg.python.org/cpython/rev/03296316a892 [2]: see e.g. http://hg.python.org/cpython/file/default/Lib/unittest/test/test_case.py#l11... [3]: we could also introduce a MetaDeprecationWarning and make PendingDeprecationWarning inherit from it so that it can be used to pending-deprecate itself. Once PendingDeprecationWarning is gone, the MetaDeprecationWarning will become useless and can then be used to meta-deprecate itself. [4]: https://docs.djangoproject.com/en/dev/internals/deprecation/ [5]: http://www.python.org/dev/peps/pep-0387/
On Sat, Jan 25, 2014 at 8:29 AM, Ezio Melotti <ezio.melotti@gmail.com>wrote:
Hi, a couple of years ago I suggested to define and document our deprecation policy in this thread: https://mail.python.org/pipermail/python-dev/2011-October/114199.html I didn't receive many replies and eventually nothing was done. Lately the same issue came up on #python-dev and Larry and Nick suggested me to bring this up again. Nick also suggested to document our deprecation policy in PEP 5 (Guidelines for Language Evolution: http://www.python.org/dev/peps/pep-0005/ ).
I'm including below the full text of the original email.
Best Regards, Ezio Melotti
-------------------------------
Hi, our current deprecation policy is not so well defined (see e.g. [0]), and it seems to me that it's something like: 1) deprecate something and add a DeprecationWarning; 2) forget about it after a while; 3) wait a few versions until someone notices it; 4) actually remove it;
I suggest to follow the following process: 1) deprecate something and add a DeprecationWarning; 2) decide how long the deprecation should last; 3) use the deprecated-remove[1] directive to document it; 4) add a test that fails after the update so that we remember to remove it[2];
Other related issues:
PendingDeprecationWarnings: * AFAIK the difference between PDW and DW is that PDW are silenced by default; * now DW are silence by default too, so there are no differences; * I therefore suggest we stop using it, but we can leave it around[3] (other projects might be using it for something different);
Deprecation Progression: Before, we more or less used to deprecated in release X and remove in X+1, or add a PDW in X, DW in X+1, and remove it in X+2. I suggest we drop this scheme and just use DW until X+N, where N is
=1 and depends on what is being removed. We can decide to leave the DW for 2-3 versions before removing something widely used, or just deprecate in X and remove in X+1 for things that are less used.
The way I used PendingDeprecationWarning is that it is used until two releases before removal. If we want to add a keyword-only parameter to DeprecationWarning which specifies when the feature will be removed then PendingDeprecationWarning is really not needed; 'removal' maybe? Could then come up with a template format that's consistent when 'removal' is specified: '{} is deprecated and slated for removal in {} {}'.format(str(self.args), self.project, self.removal) # self.project defaults to 'Python'. Depending on how fancy we get we could have it so that when you warn on a DeprecationWarning where 'removal' equals sys.version (or whatever) then it always turns into an error case and thus raised as an exception. -Brett
Porting from 2.x to 3.x: Some people will update directly from 2.7 to 3.2 or even later versions (3.3, 3.4, ...), without going through earlier 3.x versions. If something is deprecated on 3.2 but not in 2.7 and then is removed in 3.3, people updating from 2.7 to 3.3 won't see any warning, and this will make the porting even more difficult. I suggest that: * nothing that is available and not deprecated in 2.7, will be removed until 3.x (x needs to be defined); * possibly we start backporting warnings to 2.7 so that they are visible while running with -3;
Documenting the deprecations: In order to advertise the deprecations, they should be documented: * in their doc, using the deprecated-removed directive (and possibly not the 'deprecated' one); * in the what's new, possibly listing everything that is currently deprecated, and when it will be removed; Django seems to do something similar[4]. (Another thing I would like is a different rending for deprecated functions. Some part of the docs have a deprecation warning on the top of the section and the single functions look normal if you miss that. Also while linking to a deprecated function it would be nice to have it rendered with a different color or something similar.)
Testing the deprecations: Tests that fail when a new release is made and the version number is bumped should be added to make sure we don't forget to remove it. The test should have a related issue with a patch to remove the deprecated function and the test. Setting the priority of the issue to release blocker or deferred blocker can be done in addition/instead, but that works well only when N == 1 (the priority could be updated for every release though). The tests could be marked with an expected failure to give some time after the release to remove them. All the deprecation-related tests might be added to the same file, or left in the test file of their module.
Where to add this: Once we agree about the process we should write it down somewhere. Possible candidates are: * PEP387: Backwards Compatibility Policy[5] (it has a few lines about this); * a new PEP; * the devguide; I think having it in a PEP would be good, the devguide can then link to it.
Best Regards, Ezio Melotti
[0]: http://bugs.python.org/issue13248 [1]: deprecated-removed doesn't seem to be documented in the documenting doc, but it was added here: http://hg.python.org/cpython/rev/03296316a892 [2]: see e.g. http://hg.python.org/cpython/file/default/Lib/unittest/test/test_case.py#l11... [3]: we could also introduce a MetaDeprecationWarning and make PendingDeprecationWarning inherit from it so that it can be used to pending-deprecate itself. Once PendingDeprecationWarning is gone, the MetaDeprecationWarning will become useless and can then be used to meta-deprecate itself. [4]: https://docs.djangoproject.com/en/dev/internals/deprecation/ [5]: http://www.python.org/dev/peps/pep-0387/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
On Jan 25, 2014, at 5:29 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
Nick also suggested to document our deprecation policy in PEP 5 (Guidelines for Language Evolution: http://www.python.org/dev/peps/pep-0005/ ).
Here's a few thoughts on deprecations: * If we care at all about people moving to Python 3, then we'll stop doing anything that makes the process more difficult. For someone moving from Python 2.7, it really doesn't matter if something that existed in 2.7 got deprecated in 3.1 and removed in 3.3; from their point-of-view, it just one more thing that won't work. * The notion of PendingDeprecationWarnings didn't work out very well. Conceptually, it was a nice idea, but in practice no one was benefitting from it. The warnings slowed down working, but not yet deprecated code. And no one was actually seeing the pending deprecations. * When a module becomes obsolete (say optparse vs argparse), there isn't really anything wrong with just leaving it in and making the docs indicate that something better is available. AFAICT, there isn't much value in actually removing the older tool. * A good use for deprecations is for features that were flat-out misdesigned and prone to error. For those, there is nothing wrong with deprecating them right away. Once deprecated though, there doesn't need to be a rush to actually remove it -- that just makes it harder for people with currently working code to upgrade to newer versions of Python. * When I became a core developer well over a decade ago, I was a little deprecation happy (old stuff must go, keep everything nice and clean, etc). What I learned though is that deprecations are very hard on users and that the purported benefits usually aren't really important. my-two-cents, Raymond
On 26 January 2014 12:30, Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
On Jan 25, 2014, at 5:29 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
Nick also suggested to document our deprecation policy in PEP 5 (Guidelines for Language Evolution: http://www.python.org/dev/peps/pep-0005/ ).
Here's a few thoughts on deprecations:
* If we care at all about people moving to Python 3, then we'll stop doing anything that makes the process more difficult. For someone moving from Python 2.7, it really doesn't matter if something that existed in 2.7 got deprecated in 3.1 and removed in 3.3; from their point-of-view, it just one more thing that won't work.
* The notion of PendingDeprecationWarnings didn't work out very well. Conceptually, it was a nice idea, but in practice no one was benefitting from it. The warnings slowed down working, but not yet deprecated code. And no one was actually seeing the pending deprecations.
* When a module becomes obsolete (say optparse vs argparse), there isn't really anything wrong with just leaving it in and making the docs indicate that something better is available. AFAICT, there isn't much value in actually removing the older tool.
* A good use for deprecations is for features that were flat-out misdesigned and prone to error. For those, there is nothing wrong with deprecating them right away. Once deprecated though, there doesn't need to be a rush to actually remove it -- that just makes it harder for people with currently working code to upgrade to newer versions of Python.
* When I became a core developer well over a decade ago, I was a little deprecation happy (old stuff must go, keep everything nice and clean, etc). What I learned though is that deprecations are very hard on users and that the purported benefits usually aren't really important.
+1 to what Raymond send - unless something is actively causing us maintenance hassles or is a genuine bug magnet (cf. contextlib.nested), we should be very cautious about forcing users to change their code, especially in the context of raising additional barriers to migration from Python 2 to Python 3. Ezio's suggestions make sense as a policy for how to handle the situation when we *do* decide something needs programmatic deprecation, but I think there needs to be an explicit caveat that programmatic deprecation should be a last resort. Wherever possible, we should also provide a PyPI module that helps address the issue in a cross-version compatible way (whether through a dedicated upstream and/or backport module like contextlib2 or by adding features to existing cross-version compatibility modules like six and future) As an example that hopefully helps illustrate the two different cases, the shift to enabling proper SSL/TLS verification by default would qualify as a worthy use of programmatic deprecation (since the current insecure-by-default behaviour is a genuine bug magnet that leads to security flaws), but I'm not yet convinced that we actually gain any significant benefit from deprecating the legacy import plugin APIs. While PEP 451 nominally approved deprecating the latter, on the basis that deprecating them will make importlib easier to maintain, they received a stay of execution in 3.4 because the extension module import system currently still needs them, and there are other valid use cases that PEP 451 doesn't currently cover, At this point, I have come to believe that retaining the existing strictly-more-powerful-but-also-harder-to-use-correctly plugin API is a better way to handle those more complex use cases rather than inventing something new on top of PEP 451, as the latter approach would make things *harder* to maintain (due to increased complexity and needing to manage the deprecation process), and provide a worse experience for users implementing custom import hooks that need to support multiple versions (due to the introduction of new cross-version compatibility issues). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, 25 Jan 2014 18:30:24 -0800 Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
On Jan 25, 2014, at 5:29 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
Nick also suggested to document our deprecation policy in PEP 5 (Guidelines for Language Evolution: http://www.python.org/dev/peps/pep-0005/ ).
Here's a few thoughts on deprecations:
* If we care at all about people moving to Python 3, then we'll stop doing anything that makes the process more difficult. For someone moving from Python 2.7, it really doesn't matter if something that existed in 2.7 got deprecated in 3.1 and removed in 3.3; from their point-of-view, it just one more thing that won't work.
+1.
* The notion of PendingDeprecationWarnings didn't work out very well. Conceptually, it was a nice idea, but in practice no one was benefitting from it. The warnings slowed down working, but not yet deprecated code. And no one was actually seeing the pending deprecations.
+1 too, especially since DeprecationWarnings are now silent by default, there's no reason to start with a PendingDeprecationWarning IMO. Regards Antoine.
participants (6)
-
Antoine Pitrou -
Brett Cannon -
Ethan Furman -
Ezio Melotti -
Nick Coghlan -
Raymond Hettinger