
Hi, As a follow-up to the "[Python-Dev] (Looking for) A Retrospective on the Move to Python 3" thread, I will like to clarify how a feature should be removed from Python. We have multiple tools: * Emit a PendingDeprecationWarning warning at runtime for *slow* deprecation (remove a feature in at least 3 cycles) * Emit a DeprecationWarning warning at runtime for fast deprecation (remove a feature in 2 cycles) * Document that a feature is deprecated in the module documentation * "What's New in Python X.Y?" documents: especially Deprecated, Removed and Porting to Python X.Y sections. * Communicate on python-dev, Twitter, Python Insider blog, etc. * Collaborate with major Python projects to help them to migrate the alternative IMHO a feature should not be removed if there is no alternative, or if the alternative is too new. The usual process is: * Document the deprecation and emit DeprecationWarning in version N * Document the removal and remove the feature in version N+1 Slow deprecation: * Document the deprecation and emit PendingDeprecationWarning in version N * Emit a DeprecationWarning in version N+1 * Document the removal and remove the feature in version N+2 The hidden ghost is the old rule: "keep old APIs around to make porting from Python 2 easier" Is this rule still applicable? Does it mean that the Python 3 release following Python 2 end-of-life (2020) will be our next feared "Python 4"? Are we going to remove all deprecated features at once, to maximize incompatibilities and make users unhappy? Should we always document in which version a feature will be removed? Some features are deprecated since many versions, and the deprecated features are still there. In most cases, it's because of the Python 2 rule. Sometimes, we forget to remove features which has been scheduled for removal in a specific version. Maybe we should create a tool to list features scheduled for removal, and open a discussion to check each removal? Ten years ago, I wanted to remove most modules and many functions from the standard library. Now my experience showed me that *each* removal is very painful, hurt more projects than expected, and takes longer than 3 years to be fully effective (no longer used in most common 3rd party modules). The usual issue is to write a single code base working on all Python versions. For example, the new alternative is only available on recent Python versions, which requires to have at least two code paths to support all versions. Sometimes, there are 3 code paths... For a recent example, see "remove platform.linux_distribution()": https://bugs.python.org/issue28167 Removing a feature from the C API is more complex, since there is no portable way to emit a deprecation warning at compilation. There is Py_DEPRECATED() which seems to only be available on GCC (3.1 and newer). Maybe we should stop to remove features, except if there is really a good reason to do that? Victor

On Wed, 2 May 2018 11:11:16 +0200 Victor Stinner <vstinner@redhat.com> wrote:
It's at least possible with gcc, clang and MSVC: https://stackoverflow.com/questions/295120/c-mark-as-deprecated/295229#29522... You can even add a deprecation message. Regards Antoine.

2018-05-02 11:17 GMT+02:00 Antoine Pitrou <solipsis@pitrou.net>:
Aha, I opened an issue: https://bugs.python.org/issue33407 "Implement Py_DEPRECATED() macro for Visual Studio" Victor

On Wed, 2 May 2018 at 02:12 Victor Stinner <vstinner@redhat.com> wrote:
Python 2 is still supported, so I assume so.
I don't see why removing features that already raise a DeprecationWarning would require bumping the major version number. Personally, I assumed either Python 3.9 or 3.10 would be the version where we were okay clearing out the stuff that had been raising DeprecationWarning for years.
Should we always document in which version a feature will be removed?
We should at least open an issue to track when the removal is scheduled. But if we know ahead of time then I so no reason not to document it.
Right, which is why we should open an issue immediately. I had a "remove pyvenv" issue open and assigned to myself for years while I waited for Python 3.8 development to open. Staring at that issue for so long made sure I didn't forget. ;)
Maybe we should create a tool to list features scheduled for removal, and open a discussion to check each removal?
I don't know if a tool is necessary. We could have a meta issue or text file somewhere to track what's to be removed in a certain version.
I thought that already was the policy. ;) I think the real question is what is people's definition of a "good reason". For instance, I'm all for removing unsupported code so we don't have to maintain it, even if it's just from code modernization and such (i.e. pruning down the stdlib). But I also know others disagree with me and are fine just having modules sit there as long as the issue rate is low enough to not notice. It's just one of those things we don't have an official policy on (yet?). -Brett

2018-05-02 14:24 GMT-03:00 Brett Cannon <brett@python.org>:
Maybe a specific PEP that list all removals that happened, those that will happen next version, and next ones after that? IOW, a single point where one can see which features/details will fly away and when. Probably it's a good idea to see everything at once and evaluate if "we're removing too many things at once", and have a better deprecation/removal cadence. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

On Thu, 3 May 2018 at 12:01 Facundo Batista <facundobatista@gmail.com> wrote:
I don't know if we need a history section as the What's New docs cover that. But having a PEP or some other file/doc that acts like a future What's New but for removals seems like a reasonable idea. -Brett

2018-05-02 19:24 GMT+02:00 Brett Cannon <brett@python.org>:
Sorry, when I wrote "Python 4" I mean "the new Python release which introduces a lot of backward incompatible changes and will annoy everyone". It can be Python 3.9 or 3.10, or whatever version (including 4.3 if you want) :-) My point is that deprecating a feature is one thing, removing it is something else. We should slow down feature removal, or more generally reduce the number of backward incompatible changes per release. Maybe keep a deprecating warning for 10 years is just fine. Extract of the Zen of Python: "Although practicality beats purity." ;-) Victor

On Fri, May 04, 2018 at 12:56:57PM +0200, Victor Stinner wrote:
I call that "Python 4000", in analogy with Python 3000 which became Python 3, and to further emphasise how far away it is, I've started calling it "Python 5000". As I understand it, Guido has said that we won't be doing a repeat of the 2 to 3 break-a-lot-of-stuff-at-once transition unless there is some unforeseen necessity.
Have there been many features removed since 3.1? I know there were some features removed in 3.0, like callable(), which were later put back in, but I can't think of anythin removed since then. If there were, the pace of it is pretty slow.
Maybe keep a deprecating warning for 10 years is just fine.
Extract of the Zen of Python: "Although practicality beats purity." ;-)
Aside from things that need to be removed for security reasons, this seems good to me. -- Steve

I would like to take a step back, I think that for removal of feature you need to make sure that the "old way" is not common enough. I think that in many context, users of Python see DeprecationWarnings as a stick. A deprecation warning means you'll have to do some work. A pep or a document that list all the removal may be useful, but I'm unsure if in reading what's knew you'll catch all the removal of functionality you use and convince yourself (or your management) to thouroughly go through your code base and fix these things. Maybe to help transition you need to soften the stick by being more explicit as since when things are deprecated and what can be done. Personally I find deprecations when I run test-suites and use libraries interactively, not by reading ahead of time documents. Python has made huge progress in the deprecation messages. And the usage of "X Is is deprecated, use Y" has become common, this is already tremendously more helpful than just "X is deprecated". But when I hit a DeprecationWarning message there is one crucial piece of information missing most of the time: Since which version number it's deprecated (and sometime since when the replacement is available could be good if overlap between functionality there was). Yes it is available in the docs (or in the docstring), but if I have to switch context to search in the documentation I might not bother. In particular when the version is in the deprecation message, as I know the python requirement of my project, I can immediately know if it's a search and replace away, or if I need to have conditional branches. That is at least to me the biggest change that would push me to update the deprecated features in my own project, and make me do drive by contributions on Open-Source project I use where I spotted one of those. There are other improvement possible to deprecation messages, but I think this one would have the largest impact for many developers who do not closely follow python development, and for whom DeprecationWarnings are just an extra hassle. Also if there is a standard, or conventional way _in warnings_ to say since when something is deprecated, then it's relatively easy to search for it when version N+p is released and remove what you believe is old enough. Thanks, -- Matthias On Fri, 4 May 2018 at 09:03, Serhiy Storchaka <storchaka@gmail.com> wrote:

04.05.18 20:57, Matthias Bussonnier пише:
I think the information about since which version number it will be removed is more useful. Different cases need different deprecation periods. The more common the case, the longer deprecation period should be. Some recently added warnings contain this information. Ideally any deprecated feature should have a replacement, and this replacement should be available in at least one version before adding the deprecation warning. X.Y: added a replacement X.Y+1: added a deprecation warning. Many users need to support only two recent versions and can move to using the replacement now. X.Y+3 (or X.Y+2): removed the deprecated feature. Versions older than X.Y should grew out of use at that moment.

On Fri, 4 May 2018 at 11:49, Serhiy Storchaka <storchaka@gmail.com> wrote:
Maybe to push people forward, but from experience it is hard to predict future, so saying when it _will_ be remove is hard. When you _want_ to remove it probably. Victor's first mail in this thread is a good example. The functionality was marked to be removed from 3.7, but I think it is likely too late now. You can always update, but I hate giving differing information between software version. I'm curious about your use case for the version of removal, I usually don't care when it's going to be removed, I prefer since when the functionality is deprecated. if pyversion < deprecated_version: old_stuff else: new_stuff And I soon as my project drop deprecated_version, I remove the conditional. I do not try/except on purpose to be able to grep for when to remove the code. Could you share you use case ? Of be ore detailed ? We can also be more generic and say that if DeprecationWarning messages could contain timeline informations it would likely encourage the migration. One related question is how much are DeprecationWarning messages stables between versions ? Would any update to many of these be accepted of refused because users might be filtering them ? -- Matthias

On Fri, 4 May 2018 at 12:09 Matthias Bussonnier < bussonniermatthias@gmail.com> wrote:
So there is actually an opportunity here to programmatically help prevent missing a removal. If we attach a version number for when a DeprecationWarning is expected to go away due to removal we can then have the warning itself raise a warning that while the DeprecationWarning was supposed to be gone by e.g. 3.7, it's still being raised. The drawback to this is it will make the first cut-over to a newer version of Python a bit more painful, but as long as warnings don't raise exceptions it should just be a noisy.
The other way to look at that is: if pyversion < replacement: old_stuff else: new_stuff In which case you don't care about when the deprecation happened, just when it's replacement was introduced (which Serhiy suggested being a version before the deprecation is added).
Same as any other exception; typically left alone unless there's a better way to phrase things (IOW don't parse the message ;) . So at this point there are basically two threads going on here. One is an official deprecation policy. E.g. X.Y something happens, X.Y+1 something else happens, etc. The other one is how to make what gets deprecated more discoverable. E.g. an informational PEP that tracks what's planned, warnings having more metadata, etc. Both of these things seem independent, although one of them will require a PEP while the other just takes some work (I leave it as an exercise for the reader to figure out which one is which ;) . -Brett

04.05.18 22:08, Matthias Bussonnier пише:
Right. But the data of removing is usually specified when the code for removing already is written, or even merged in the next branch. This is true for most warnings introduced in 3.7 that warns about the removal in 3.8. This is true for Py3k warnings backported to 2.7. In short-term deprecations you have the only one release in which the warning is emitted.

2018-05-04 20:48 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
I think the information about since which version number it will be removed is more useful.
About deprecation, there is the funny story of bytes filenames on Windows. I deprecated this feature in Windows 3.3 since it was broken. I really wanted hard to remove support for bytes filenames on Windows. One day, Steve Dower showed up with the PEP 529 "Change Windows filesystem encoding to UTF-8" and he just *removed the deprecation warning* from Python 3.6, since bytes filenames are now working as intended :-) Victor

To be fair, if they hadn’t already been deprecated we would have had to deprecate the old behaviour for a couple of releases before changing it. It just so happened that we did the deprecation first without knowing what the fix would be :) Top-posted from my Windows phone From: Victor Stinner Sent: Friday, May 4, 2018 15:24 To: Serhiy Storchaka Cc: python-dev Subject: Re: [Python-Dev] Process to remove a Python feature 2018-05-04 20:48 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
I think the information about since which version number it will be removed is more useful.
About deprecation, there is the funny story of bytes filenames on Windows. I deprecated this feature in Windows 3.3 since it was broken. I really wanted hard to remove support for bytes filenames on Windows. One day, Steve Dower showed up with the PEP 529 "Change Windows filesystem encoding to UTF-8" and he just *removed the deprecation warning* from Python 3.6, since bytes filenames are now working as intended :-) Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

On 05/04/2018 11:48 AM, Serhiy Storchaka wrote:
04.05.18 20:57, Matthias Bussonnier пише:
If we are going to provide extra metadata, we may as well supply it all. Some folks will need the start date, some the end date, and probably some both dates.
X.Y+1: added a deprecation warning. Many users need to support only two recent versions and can move to using the replacement now.
I'm curious how you arrived at this conclusion? I know I've only worked at two different Python-using companies, but neither aggressively tracks the latest Python minor version, and as a library author I support more than the two most recent versions. -- ~Ethan~

08.05.18 19:10, Ethan Furman пише:
Maybe I was too optimistic. ;-) Libraries need to support more Python versions of course. But two versions is a minimum, and I thing that for many applications (if they are targeted to the specific OS version or shipped with an own Python) this is enough. Even if their number is not large, they will get a benefit from introducing a replacement before adding a warning. If you support versions X.Y-1 and X.Y, you just use the old feature. If you support versions X.Y and X.Y+1, you replace it with the new feature. If you support versions X.Y-1, X.Y and X.Y+1 you need either to ignore varnings, or to add a runtime check for switching between using the old and the new feature. This complicates the code. But at least the code is clear in the first two cases.

On Wed, May 9, 2018 at 2:53 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
Here in this house, we have: * 3.8, with or without various patches as are being proposed upstream * 3.7, a slightly old alpha build, as a secondary on the laptop * 3.6 on the latest Ubuntu * 3.5 on the Raspberry Pi * 3.4 as shipped by Debian, as the laptop's primary Python 3 * And I'm not even counting the various different 2.7s. My brother and I built a TCP-managed doorbell involving the rPi. At an absolute minimum, it has to support 3.4, 3.5, and 3.6; and supporting 3.7 is important, given how close it is to release. Supporting 3.8 is no harder than supporting 3.7, but anything that did actual version number checks would need to be aware of it. So that's potentially five different versions. (Fortunately, I have all five installed on one computer, so testing isn't hard.) Supporting just two versions seems a bit too hopeful. Supporting three would be a minimum for an in-house app; and if your users run different Linux distros with different release cadences, four wouldn't be unlikely, even among current releases. As a general rule, I prefer to avoid writing "before" and "after" code. For a lot of deprecations, that's easy - when "await" started becoming a keyword, I renamed a function to "wait" [1], and it was 100% compatible with all versions. If there absolutely has to be code doing two different things, I'd rather catch ImportError than do actual version checks. But if it has to check versions, it'll need to be aware of quite a few. ChrisA [1] https://github.com/Rosuav/LetMeKnow/commit/2ecbbdcc3588139932525140ceb8c2cb6...

2018-05-04 23:59 GMT+02:00 Terry Reedy <tjreedy@udel.edu>:
I suggest you to watch Daniele Esposti's talk "Evolution or stagnation programming languages". He explains that Javascript is more successful than Python to introduce *language* evolutions thanks to transpiling (things like babel and polyfill): https://www.pycon.it/conference/talks/evolution-or-stagnation-programming-la... Victor

On 5/4/2018 6:18 PM, Victor Stinner wrote:
I ran through the slides and found the babelsite. What I found: Babel translates new code back to a sufficiently powerful and presumably ubiquitous older version. It does so on a selectable feature basis rather than a language version basis. (In other words, define your own 'new' version.) Polyfill supplies the backported new objects needed to make the back translations run with new semantics. This would be equivalent of defining, for instance, 2.7 as a base version and having 3xbytesto27, 35asyncto27, etc for every new 3.x feature. Some people wanted this, but, of course, 2.7 is *not* installed everywhere. If Microsoft were to treat Python like it once did Basic, and install it on all Windows machines, it would start with recent 3.x. Neither the slides nor site said anything about bug fixes and about the need to have multiple versions of every function touched. Because of the unique features of how Javascript is distributed and used, I don't see how the Babel example would apply very well to Python. -- Terry Jan Reedy

On 5 May 2018 at 07:59, Terry Reedy <tjreedy@udel.edu> wrote:
That capability actually exists in the python-future project (in the form of their "pasteurize" script, which takes idiomatic Python 3 code and turns it into Python 2.7 compatible code). One major issue though is that without a JIT compiler to speed things back up, the polyfills needed to target old versions in the general case (rather than when restricting yourself to a "backport friendly Python subset") are often either going to be unacceptably slow, or else unacceptably irritating to maintain, so folks have invested their time into figuring out how to ship their own runtimes in order to avoid being restricted by redistributors with slow runtime update cycles, and in being able to rely on PyPI packages rather than standard library packages. (That's very different from the incentives for browser vendors, who have users that expect them to be able to make arbitrary websites work, and blame the browser rather than the site when particular sites they want to use don't work properly) The other thing worth keeping in mind is that one of the core assumptions of JavaScript tooling is that debugging tools will be able to reach out to the internet and download additional files (most notably source maps) to help make sense of the currently running code. Without that ability to trace from compiled code back to the preferred source form for editing, making sense of an otherwise straightforward traceback can become a complex debugging exercise. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Le 02/05/2018 à 11:11, Victor Stinner a écrit :
* Communicate on python-dev, Twitter, Python Insider blog, etc. * Collaborate with major Python projects to help them to migrate the alternative
I wonder if it would be interesting to have a package available by pypi.org which would provide the removed features. In your example, the developers would have to update their source code: # giving 'obsolete' as name for this package platform.linux_distribution() -> obsolete.platform.linux_distribution() The code in 'obsolete' package could come from the removed code from cpython or a wrapper around a third-party package ('distro' package in the example). Plus: - quick temporary fix for users -> the removal is less painful - the name of the import is a hint that something has to be fixed -> useful for new comers on the user source code Cons: - it pushes the question to how many times the previous behavior should be maintained from python language to 'obsolete' package. So it's not completely solved. - it adds a step to removal procedure. - I guess there are some features not movable into a package. -- Stéphane

On Tue, May 08, 2018 at 03:20:50PM +0200, Stéphane Blondon wrote:
Its not solved at all. Who is responsible for moving code to this third-party package? Who is responsible for maintaining that third-party package? If it is the core developers, then it isn't really removed at all. If nobody is maintaining it, then its going to suffer bitrot and become unusable. Python is open source, so anyone can fork a module or library from the std lib and move it to PyPI, they don't need the core dev's blessing, they just need to follow the licence requirements. -- Steve

On Wed, 2 May 2018 11:11:16 +0200 Victor Stinner <vstinner@redhat.com> wrote:
It's at least possible with gcc, clang and MSVC: https://stackoverflow.com/questions/295120/c-mark-as-deprecated/295229#29522... You can even add a deprecation message. Regards Antoine.

2018-05-02 11:17 GMT+02:00 Antoine Pitrou <solipsis@pitrou.net>:
Aha, I opened an issue: https://bugs.python.org/issue33407 "Implement Py_DEPRECATED() macro for Visual Studio" Victor

On Wed, 2 May 2018 at 02:12 Victor Stinner <vstinner@redhat.com> wrote:
Python 2 is still supported, so I assume so.
I don't see why removing features that already raise a DeprecationWarning would require bumping the major version number. Personally, I assumed either Python 3.9 or 3.10 would be the version where we were okay clearing out the stuff that had been raising DeprecationWarning for years.
Should we always document in which version a feature will be removed?
We should at least open an issue to track when the removal is scheduled. But if we know ahead of time then I so no reason not to document it.
Right, which is why we should open an issue immediately. I had a "remove pyvenv" issue open and assigned to myself for years while I waited for Python 3.8 development to open. Staring at that issue for so long made sure I didn't forget. ;)
Maybe we should create a tool to list features scheduled for removal, and open a discussion to check each removal?
I don't know if a tool is necessary. We could have a meta issue or text file somewhere to track what's to be removed in a certain version.
I thought that already was the policy. ;) I think the real question is what is people's definition of a "good reason". For instance, I'm all for removing unsupported code so we don't have to maintain it, even if it's just from code modernization and such (i.e. pruning down the stdlib). But I also know others disagree with me and are fine just having modules sit there as long as the issue rate is low enough to not notice. It's just one of those things we don't have an official policy on (yet?). -Brett

2018-05-02 14:24 GMT-03:00 Brett Cannon <brett@python.org>:
Maybe a specific PEP that list all removals that happened, those that will happen next version, and next ones after that? IOW, a single point where one can see which features/details will fly away and when. Probably it's a good idea to see everything at once and evaluate if "we're removing too many things at once", and have a better deprecation/removal cadence. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

On Thu, 3 May 2018 at 12:01 Facundo Batista <facundobatista@gmail.com> wrote:
I don't know if we need a history section as the What's New docs cover that. But having a PEP or some other file/doc that acts like a future What's New but for removals seems like a reasonable idea. -Brett

2018-05-02 19:24 GMT+02:00 Brett Cannon <brett@python.org>:
Sorry, when I wrote "Python 4" I mean "the new Python release which introduces a lot of backward incompatible changes and will annoy everyone". It can be Python 3.9 or 3.10, or whatever version (including 4.3 if you want) :-) My point is that deprecating a feature is one thing, removing it is something else. We should slow down feature removal, or more generally reduce the number of backward incompatible changes per release. Maybe keep a deprecating warning for 10 years is just fine. Extract of the Zen of Python: "Although practicality beats purity." ;-) Victor

On Fri, May 04, 2018 at 12:56:57PM +0200, Victor Stinner wrote:
I call that "Python 4000", in analogy with Python 3000 which became Python 3, and to further emphasise how far away it is, I've started calling it "Python 5000". As I understand it, Guido has said that we won't be doing a repeat of the 2 to 3 break-a-lot-of-stuff-at-once transition unless there is some unforeseen necessity.
Have there been many features removed since 3.1? I know there were some features removed in 3.0, like callable(), which were later put back in, but I can't think of anythin removed since then. If there were, the pace of it is pretty slow.
Maybe keep a deprecating warning for 10 years is just fine.
Extract of the Zen of Python: "Although practicality beats purity." ;-)
Aside from things that need to be removed for security reasons, this seems good to me. -- Steve

I would like to take a step back, I think that for removal of feature you need to make sure that the "old way" is not common enough. I think that in many context, users of Python see DeprecationWarnings as a stick. A deprecation warning means you'll have to do some work. A pep or a document that list all the removal may be useful, but I'm unsure if in reading what's knew you'll catch all the removal of functionality you use and convince yourself (or your management) to thouroughly go through your code base and fix these things. Maybe to help transition you need to soften the stick by being more explicit as since when things are deprecated and what can be done. Personally I find deprecations when I run test-suites and use libraries interactively, not by reading ahead of time documents. Python has made huge progress in the deprecation messages. And the usage of "X Is is deprecated, use Y" has become common, this is already tremendously more helpful than just "X is deprecated". But when I hit a DeprecationWarning message there is one crucial piece of information missing most of the time: Since which version number it's deprecated (and sometime since when the replacement is available could be good if overlap between functionality there was). Yes it is available in the docs (or in the docstring), but if I have to switch context to search in the documentation I might not bother. In particular when the version is in the deprecation message, as I know the python requirement of my project, I can immediately know if it's a search and replace away, or if I need to have conditional branches. That is at least to me the biggest change that would push me to update the deprecated features in my own project, and make me do drive by contributions on Open-Source project I use where I spotted one of those. There are other improvement possible to deprecation messages, but I think this one would have the largest impact for many developers who do not closely follow python development, and for whom DeprecationWarnings are just an extra hassle. Also if there is a standard, or conventional way _in warnings_ to say since when something is deprecated, then it's relatively easy to search for it when version N+p is released and remove what you believe is old enough. Thanks, -- Matthias On Fri, 4 May 2018 at 09:03, Serhiy Storchaka <storchaka@gmail.com> wrote:

04.05.18 20:57, Matthias Bussonnier пише:
I think the information about since which version number it will be removed is more useful. Different cases need different deprecation periods. The more common the case, the longer deprecation period should be. Some recently added warnings contain this information. Ideally any deprecated feature should have a replacement, and this replacement should be available in at least one version before adding the deprecation warning. X.Y: added a replacement X.Y+1: added a deprecation warning. Many users need to support only two recent versions and can move to using the replacement now. X.Y+3 (or X.Y+2): removed the deprecated feature. Versions older than X.Y should grew out of use at that moment.

On Fri, 4 May 2018 at 11:49, Serhiy Storchaka <storchaka@gmail.com> wrote:
Maybe to push people forward, but from experience it is hard to predict future, so saying when it _will_ be remove is hard. When you _want_ to remove it probably. Victor's first mail in this thread is a good example. The functionality was marked to be removed from 3.7, but I think it is likely too late now. You can always update, but I hate giving differing information between software version. I'm curious about your use case for the version of removal, I usually don't care when it's going to be removed, I prefer since when the functionality is deprecated. if pyversion < deprecated_version: old_stuff else: new_stuff And I soon as my project drop deprecated_version, I remove the conditional. I do not try/except on purpose to be able to grep for when to remove the code. Could you share you use case ? Of be ore detailed ? We can also be more generic and say that if DeprecationWarning messages could contain timeline informations it would likely encourage the migration. One related question is how much are DeprecationWarning messages stables between versions ? Would any update to many of these be accepted of refused because users might be filtering them ? -- Matthias

On Fri, 4 May 2018 at 12:09 Matthias Bussonnier < bussonniermatthias@gmail.com> wrote:
So there is actually an opportunity here to programmatically help prevent missing a removal. If we attach a version number for when a DeprecationWarning is expected to go away due to removal we can then have the warning itself raise a warning that while the DeprecationWarning was supposed to be gone by e.g. 3.7, it's still being raised. The drawback to this is it will make the first cut-over to a newer version of Python a bit more painful, but as long as warnings don't raise exceptions it should just be a noisy.
The other way to look at that is: if pyversion < replacement: old_stuff else: new_stuff In which case you don't care about when the deprecation happened, just when it's replacement was introduced (which Serhiy suggested being a version before the deprecation is added).
Same as any other exception; typically left alone unless there's a better way to phrase things (IOW don't parse the message ;) . So at this point there are basically two threads going on here. One is an official deprecation policy. E.g. X.Y something happens, X.Y+1 something else happens, etc. The other one is how to make what gets deprecated more discoverable. E.g. an informational PEP that tracks what's planned, warnings having more metadata, etc. Both of these things seem independent, although one of them will require a PEP while the other just takes some work (I leave it as an exercise for the reader to figure out which one is which ;) . -Brett

04.05.18 22:08, Matthias Bussonnier пише:
Right. But the data of removing is usually specified when the code for removing already is written, or even merged in the next branch. This is true for most warnings introduced in 3.7 that warns about the removal in 3.8. This is true for Py3k warnings backported to 2.7. In short-term deprecations you have the only one release in which the warning is emitted.

2018-05-04 20:48 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
I think the information about since which version number it will be removed is more useful.
About deprecation, there is the funny story of bytes filenames on Windows. I deprecated this feature in Windows 3.3 since it was broken. I really wanted hard to remove support for bytes filenames on Windows. One day, Steve Dower showed up with the PEP 529 "Change Windows filesystem encoding to UTF-8" and he just *removed the deprecation warning* from Python 3.6, since bytes filenames are now working as intended :-) Victor

To be fair, if they hadn’t already been deprecated we would have had to deprecate the old behaviour for a couple of releases before changing it. It just so happened that we did the deprecation first without knowing what the fix would be :) Top-posted from my Windows phone From: Victor Stinner Sent: Friday, May 4, 2018 15:24 To: Serhiy Storchaka Cc: python-dev Subject: Re: [Python-Dev] Process to remove a Python feature 2018-05-04 20:48 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
I think the information about since which version number it will be removed is more useful.
About deprecation, there is the funny story of bytes filenames on Windows. I deprecated this feature in Windows 3.3 since it was broken. I really wanted hard to remove support for bytes filenames on Windows. One day, Steve Dower showed up with the PEP 529 "Change Windows filesystem encoding to UTF-8" and he just *removed the deprecation warning* from Python 3.6, since bytes filenames are now working as intended :-) Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

On 05/04/2018 11:48 AM, Serhiy Storchaka wrote:
04.05.18 20:57, Matthias Bussonnier пише:
If we are going to provide extra metadata, we may as well supply it all. Some folks will need the start date, some the end date, and probably some both dates.
X.Y+1: added a deprecation warning. Many users need to support only two recent versions and can move to using the replacement now.
I'm curious how you arrived at this conclusion? I know I've only worked at two different Python-using companies, but neither aggressively tracks the latest Python minor version, and as a library author I support more than the two most recent versions. -- ~Ethan~

08.05.18 19:10, Ethan Furman пише:
Maybe I was too optimistic. ;-) Libraries need to support more Python versions of course. But two versions is a minimum, and I thing that for many applications (if they are targeted to the specific OS version or shipped with an own Python) this is enough. Even if their number is not large, they will get a benefit from introducing a replacement before adding a warning. If you support versions X.Y-1 and X.Y, you just use the old feature. If you support versions X.Y and X.Y+1, you replace it with the new feature. If you support versions X.Y-1, X.Y and X.Y+1 you need either to ignore varnings, or to add a runtime check for switching between using the old and the new feature. This complicates the code. But at least the code is clear in the first two cases.

On Wed, May 9, 2018 at 2:53 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
Here in this house, we have: * 3.8, with or without various patches as are being proposed upstream * 3.7, a slightly old alpha build, as a secondary on the laptop * 3.6 on the latest Ubuntu * 3.5 on the Raspberry Pi * 3.4 as shipped by Debian, as the laptop's primary Python 3 * And I'm not even counting the various different 2.7s. My brother and I built a TCP-managed doorbell involving the rPi. At an absolute minimum, it has to support 3.4, 3.5, and 3.6; and supporting 3.7 is important, given how close it is to release. Supporting 3.8 is no harder than supporting 3.7, but anything that did actual version number checks would need to be aware of it. So that's potentially five different versions. (Fortunately, I have all five installed on one computer, so testing isn't hard.) Supporting just two versions seems a bit too hopeful. Supporting three would be a minimum for an in-house app; and if your users run different Linux distros with different release cadences, four wouldn't be unlikely, even among current releases. As a general rule, I prefer to avoid writing "before" and "after" code. For a lot of deprecations, that's easy - when "await" started becoming a keyword, I renamed a function to "wait" [1], and it was 100% compatible with all versions. If there absolutely has to be code doing two different things, I'd rather catch ImportError than do actual version checks. But if it has to check versions, it'll need to be aware of quite a few. ChrisA [1] https://github.com/Rosuav/LetMeKnow/commit/2ecbbdcc3588139932525140ceb8c2cb6...

2018-05-04 23:59 GMT+02:00 Terry Reedy <tjreedy@udel.edu>:
I suggest you to watch Daniele Esposti's talk "Evolution or stagnation programming languages". He explains that Javascript is more successful than Python to introduce *language* evolutions thanks to transpiling (things like babel and polyfill): https://www.pycon.it/conference/talks/evolution-or-stagnation-programming-la... Victor

On 5/4/2018 6:18 PM, Victor Stinner wrote:
I ran through the slides and found the babelsite. What I found: Babel translates new code back to a sufficiently powerful and presumably ubiquitous older version. It does so on a selectable feature basis rather than a language version basis. (In other words, define your own 'new' version.) Polyfill supplies the backported new objects needed to make the back translations run with new semantics. This would be equivalent of defining, for instance, 2.7 as a base version and having 3xbytesto27, 35asyncto27, etc for every new 3.x feature. Some people wanted this, but, of course, 2.7 is *not* installed everywhere. If Microsoft were to treat Python like it once did Basic, and install it on all Windows machines, it would start with recent 3.x. Neither the slides nor site said anything about bug fixes and about the need to have multiple versions of every function touched. Because of the unique features of how Javascript is distributed and used, I don't see how the Babel example would apply very well to Python. -- Terry Jan Reedy

On 5 May 2018 at 07:59, Terry Reedy <tjreedy@udel.edu> wrote:
That capability actually exists in the python-future project (in the form of their "pasteurize" script, which takes idiomatic Python 3 code and turns it into Python 2.7 compatible code). One major issue though is that without a JIT compiler to speed things back up, the polyfills needed to target old versions in the general case (rather than when restricting yourself to a "backport friendly Python subset") are often either going to be unacceptably slow, or else unacceptably irritating to maintain, so folks have invested their time into figuring out how to ship their own runtimes in order to avoid being restricted by redistributors with slow runtime update cycles, and in being able to rely on PyPI packages rather than standard library packages. (That's very different from the incentives for browser vendors, who have users that expect them to be able to make arbitrary websites work, and blame the browser rather than the site when particular sites they want to use don't work properly) The other thing worth keeping in mind is that one of the core assumptions of JavaScript tooling is that debugging tools will be able to reach out to the internet and download additional files (most notably source maps) to help make sense of the currently running code. Without that ability to trace from compiled code back to the preferred source form for editing, making sense of an otherwise straightforward traceback can become a complex debugging exercise. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Le 02/05/2018 à 11:11, Victor Stinner a écrit :
* Communicate on python-dev, Twitter, Python Insider blog, etc. * Collaborate with major Python projects to help them to migrate the alternative
I wonder if it would be interesting to have a package available by pypi.org which would provide the removed features. In your example, the developers would have to update their source code: # giving 'obsolete' as name for this package platform.linux_distribution() -> obsolete.platform.linux_distribution() The code in 'obsolete' package could come from the removed code from cpython or a wrapper around a third-party package ('distro' package in the example). Plus: - quick temporary fix for users -> the removal is less painful - the name of the import is a hint that something has to be fixed -> useful for new comers on the user source code Cons: - it pushes the question to how many times the previous behavior should be maintained from python language to 'obsolete' package. So it's not completely solved. - it adds a step to removal procedure. - I guess there are some features not movable into a package. -- Stéphane

On Tue, May 08, 2018 at 03:20:50PM +0200, Stéphane Blondon wrote:
Its not solved at all. Who is responsible for moving code to this third-party package? Who is responsible for maintaining that third-party package? If it is the core developers, then it isn't really removed at all. If nobody is maintaining it, then its going to suffer bitrot and become unusable. Python is open source, so anyone can fork a module or library from the std lib and move it to PyPI, they don't need the core dev's blessing, they just need to follow the licence requirements. -- Steve
participants (16)
-
Antoine Pitrou
-
Brett Cannon
-
Chris Angelico
-
Ethan Furman
-
Facundo Batista
-
Guido van Rossum
-
Joni Orponen
-
Matthias Bussonnier
-
Nathaniel Smith
-
Nick Coghlan
-
Serhiy Storchaka
-
Steve Dower
-
Steven D'Aprano
-
Stéphane Blondon
-
Terry Reedy
-
Victor Stinner