Mailman 3 Should we require all deprecations to have a removal version that we follow through on? - Python-Dev

Should we require all deprecations to have a removal version that we follow through on?

Brett Cannon

28 Nov 2019 28 Nov '19

12:08 a.m.

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal? Now I'm not suggesting it **must** be in three feature releases. A deprecation could be set to Python 4 if people truly feel keeping the code is as easy it gets in terms of maintenance and there's enough usage to warrant such a potential indefinite deprecation/maintenance while keeping the code and docs alive (basically "there are better ways to do this, but this works fine, too, if you were already using it"). But what I am trying to avoid is the semi-regular discussion we seem to have of what is or is not reasonable to deprecate and leave versus deprecate and eventually remove because we didn't specify what the eventual plan was for a deprecation. Also feel like it would help users to know specifically what the plan is with a deprecation rather than wondering if something will get removed in the next feature release or not because we left off a specific removal version. If people are amenable to this idea I will update PEP 387 (which I plan to start a discussion about post-SC election) and I would also plan to add a warnings._deprecate() which I roughly outlined once at https://discuss.python.org/t/pendingdeprecationwarning-is-really-useful/1038....

Show replies by date

Batuhan Taskaya

28 Nov 28 Nov

12:25 a.m.

...

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal?

IMHO that really helps to reduce amount of discussions for choosing when to remove. Like should we remove it in 3.9 or we should wait to 3.10 etc. Also the consumers of that depracted parts will know the exact version they can't support if they'll continue using it. So big +1 from me On Wed, Nov 27, 2019, 9:43 PM Brett Cannon wrote:

...

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal?

Now I'm not suggesting it **must** be in three feature releases. A deprecation could be set to Python 4 if people truly feel keeping the code is as easy it gets in terms of maintenance and there's enough usage to warrant such a potential indefinite deprecation/maintenance while keeping the code and docs alive (basically "there are better ways to do this, but this works fine, too, if you were already using it"). But what I am trying to avoid is the semi-regular discussion we seem to have of what is or is not reasonable to deprecate and leave versus deprecate and eventually remove because we didn't specify what the eventual plan was for a deprecation. Also feel like it would help users to know specifically what the plan is with a deprecation rather than wondering if something will get removed in the next feature release or not because we left off a specific removal version.

If people are amenable to this idea I will update PEP 387 (which I plan to start a discussion about post-SC election) and I would also plan to add a warnings._deprecate() which I roughly outlined once at https://discuss.python.org/t/pendingdeprecationwarning-is-really-useful/1038... . _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BRA7G4UI... Code of Conduct: http://python.org/psf/codeofconduct/

Ethan Furman

12:37 a.m.

On 11/27/2019 10:38 AM, Brett Cannon wrote:

...

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal?

I would advocate for a plan in which removals happen every so many releases -- in other words, if a removal release is being prepped and a feature has been deprecated for at least two cycles, toss it. With a yearly cadence I would suggest every 5 years. I.E. deprecated removed ---------- ------- 3.6 3.10 3.7 3.10 3.8 3.10 3.9 3.15 3.10 3.15 This way folks know that upgrading from 3.8 to 3.9 should be painless, but going to 3.10 will require more careful search for deprecations. It also serves as a reminder for us to double-check for needed removals on */5 releases. -- ~Ethan~

Brett Cannon

11:28 p.m.

Ethan Furman wrote:

...

On 11/27/2019 10:38 AM, Brett Cannon wrote:

...
What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal? I would advocate for a plan in which removals happen every so many releases -- in other words, if a removal release is being prepped and a feature has been deprecated for at least two cycles, toss it. With a yearly cadence I would suggest every 5 years. I.E. deprecated removed

3.6 3.10 3.7 3.10 3.8 3.10 3.9 3.15 3.10 3.15 This way folks know that upgrading from 3.8 to 3.9 should be painless,

But there is other things that might break your code between releases, such as bug fixes, language changes that become the default, etc. Are deprecations the biggest pain point in transitioning to a new Python version for people, or is it just part of a greater culmination of changes?

...

but going to 3.10 will require more careful search for deprecations. It also serves as a reminder for us to double-check for needed removals on */5 releases.

For some reason this makes the "clean up" release feel like an anti-LTS because it's like, "3.9 works, 3.10 works, 3.11 works, 3.12 works, 3.13 works, 3.14 works, boom!"

Eric Fahlgren

11:49 p.m.

On Thu, Nov 28, 2019 at 10:02 AM Brett Cannon wrote:

...

But there is other things that might break your code between releases, such as bug fixes, language changes that become the default, etc. Are deprecations the biggest pain point in transitioning to a new Python version for people, or is it just part of a greater culmination of changes?

I just started the 3.7 -> 3.8 migration on our codebase yesterday, so this is fresh in my mind (about 500k LOC with 72 external packages plus four home-built extension modules, three of which use SWIG wrapping). My biggest pain point is making the external extension modules work due to API changes; deprecations are only a minor issue, although that's probably because we turn all warnings on during development and clean things up as soon as they appear (if there are any win32.pywintypes devs listening, fix that use of the old imp module at line 2). Things I fixed quickly include some int vs float warnings in our GUI code, replacing the call to create a new CodeType object with a code.replace, and repair some SyntaxWarnings where "is" had crept into places where "==" should have been used. So, all good improvements to the code and I've spent far longer thus far on other aspects of the porting. I would be on the side of "sooner is better", with three releases of deprecated, we-really-mean-it and gone.

Nick Coghlan

29 Nov 29 Nov

4:31 a.m.

On Thu., 28 Nov. 2019, 4:43 am Brett Cannon, wrote:

...

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal?

Now I'm not suggesting it **must** be in three feature releases. A deprecation could be set to Python 4 if people truly feel keeping the code is as easy it gets in terms of maintenance and there's enough usage to warrant such a potential indefinite deprecation/maintenance while keeping the code and docs alive (basically "there are better ways to do this, but this works fine, too, if you were already using it").

As long as it's optional, a new deprecation helper that automatically escalated to FutureWarning in X.Y.0a0, and an outright error in X.Y.0b0 seems like a good idea to me. (Escalating to errors as soon as the version number changes would be a problem from a development process point of view) If we don't add a helper API, then this sounds like a restatement of the status quo to me, rather than a change in policy. Cheers, Nick.

...

Brett Cannon

30 Nov 30 Nov

12:41 a.m.

Nick Coghlan wrote:

...

On Thu., 28 Nov. 2019, 4:43 am Brett Cannon, brett@python.org wrote:

...
What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal? Now I'm not suggesting it must be in three feature releases. A deprecation could be set to Python 4 if people truly feel keeping the code is as easy it gets in terms of maintenance and there's enough usage to warrant such a potential indefinite deprecation/maintenance while keeping the code and docs alive (basically "there are better ways to do this, but this works fine, too, if you were already using it"). As long as it's optional, a new deprecation helper that automatically escalated to FutureWarning in X.Y.0a0, and an outright error in X.Y.0b0 seems like a good idea to me.

The helper wouldn't be required, but I would say the message of when the removal will occur would be. And good idea about flipping in the beta stage.

...

(Escalating to errors as soon as the version number changes would be a problem from a development process point of view) If we don't add a helper API, then this sounds like a restatement of the status quo to me, rather than a change in policy.

Well, it's an unspoken policy that I would like to write down. :) I would also then advocate to go into the code and docs and update it all to provide removal version targets.

Victor Stinner

6 Dec 6 Dec

9:22 p.m.

Hi, Le mer. 27 nov. 2019 à 19:40, Brett Cannon a écrit :

...

What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal?

Isn't it already the current unwritten deprecation process? The common case is to schedule the removal. Previously, it was common to wait 2 releases before removing anything: pending deprecatation in 3.6, deprecation in 3.7, removal in 3.8. It means that developers are warned during 2 releases: 3 years (I'm optimistic and consider that developers pay attention to PendingDeprecationWarning). So deprecation during 3 releases with the new release cadence would fit the previous practice: 3 years ;-) Are you suggesting to start with a PendingDeprecationWarning? I recall a debate about the value of this warning which is quiet by default, whereas DeprecationWarning is now displayed in the __main__ module. I'm not sure about enforcing the removal. Some APIs were deprecated, but it was unsure if we really wanted to remove them. Or because we had no idea how long it would take for developers to upgrade to the new functions. For example, the C API for the old Unicode API using "Py_UNICODE*" type is deprecated since Python 3.3 but there is no concrete plan to remove it (just a vague "Python 4.0 removal" plan). I deprecated the support for bytes filenames on Windows in Python 3.3, but Steve Dower implemented his PEP 529 (use UTF-8 for bytes on Windows) in Python 3.6 and so the deprecation warnings have been simply removed! I would suggest to schedule removal, but not require it too strongly ;-)

...

Now I'm not suggesting it **must** be in three feature releases. A deprecation could be set to Python 4 (...)

I know that it's an unpopular opinion, but I would strongly prefer Python 4 to behave exactly as any another Python 3.x release in term of deprecation. If I exaggerate, I would prefer that Python 4.0 would have *less* incompatible changes than the last Python 3.x release. I hope that upgrading from Python 3 to Python 4 will be as smooth as possible. Changing sys.version_info.major and "python4" program name are already going to cause enough troubles. Recently, I looked at all deprecation changes scheduled for removal in Python 4.0. I removed these features in Python 3.9, especially for features deprecated for more than 2 releases. See: https://docs.python.org/dev/whatsnew/3.9.html#removed For example, I removed the "U" mode of open() which was deprecated since Python 3.4 and scheduled for removal in Python 4.0. My plan is to revert these changes if too many users complain (see my rejected PEP 606 and 608 :-D). -- I like to remove deprecated features at the beginning of a dev cycle to get users feedback earlier. If they are removed late in the dev cycle, there is a higher risk that a problematic incompatible change goes through a 3.x.0 final release and so cannot be reverted anymore. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

Brett Cannon

11:36 p.m.

Victor Stinner wrote:

...

Hi, Le mer. 27 nov. 2019 à 19:40, Brett Cannon brett@python.org a écrit :

...
What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And that we also follow through with that removal? Isn't it already the current unwritten deprecation process?

If it is we are not all doing a great job of following it. ;)

...

The common case is to schedule the removal. Previously, it was common to wait 2 releases before removing anything: pending deprecation in 3.6, deprecation in 3.7, removal in 3.8.

And this is why unwritten rule are bad: for me the rule was deprecation, then removal which was 1.5 years and will now be 2 years as we will be shifting to at least a two release deprecation cycle.

...

It means that developers are warned during 2 releases: 3 years (I'm optimistic and consider that developers pay attention to PendingDeprecationWarning). So deprecation during 3 releases with the new release cadence would fit the previous practice: 3 years ;-)

...

Are you suggesting to start with a PendingDeprecationWarning?

No.

...

I recall a debate about the value of this warning which is quiet by default, whereas DeprecationWarning is now displayed in the __main__ module. I'm not sure about enforcing the removal. Some APIs were deprecated, but it was unsure if we really wanted to remove them. Or because we had no idea how long it would take for developers to upgrade to the new functions.

Deprecations can always be extended, but leaving it perpetually open-ended seems like a bad idea to me.

...

For example, the C API for the old Unicode API using "Py_UNICODE*" type is deprecated since Python 3.3 but there is no concrete plan to remove it (just a vague "Python 4.0 removal" plan). I deprecated the support for bytes filenames on Windows in Python 3.3, but Steve Dower implemented his PEP 529 (use UTF-8 for bytes on Windows) in Python 3.6 and so the deprecation warnings have been simply removed! I would suggest to schedule removal, but not require it too strongly ;-)

...
Now I'm not suggesting it must be in three feature releases. A deprecation could be set to Python 4 (...) I know that it's an unpopular opinion, but I would strongly prefer Python 4 to behave exactly as any another Python 3.x release in term of deprecation.

It will, it just might have a lot more code removed than a typical release.

...

If I exaggerate, I would prefer that Python 4.0 would have less incompatible changes than the last Python 3.x release.

Have "less incompatible changes" compared to what? By definition a shift in major version means there will be a difference.

...

I hope that upgrading from Python 3 to Python 4 will be as smooth as possible.

I think we all do. And if people follow standard procedures to check for deprecations in the last release before Py4K then it will be like any other release for upgrading (at least in that regard). -Brett

...

Changing sys.version_info.major and "python4" program name are already going to cause enough troubles. Recently, I looked at all deprecation changes scheduled for removal in Python 4.0. I removed these features in Python 3.9, especially for features deprecated for more than 2 releases. See: https://docs.python.org/dev/whatsnew/3.9.html#removed For example, I removed the "U" mode of open() which was deprecated since Python 3.4 and scheduled for removal in Python 4.0. My plan is to revert these changes if too many users complain (see my rejected PEP 606 and 608 :-D). -- I like to remove deprecated features at the beginning of a dev cycle to get users feedback earlier. If they are removed late in the dev cycle, there is a higher risk that a problematic incompatible change goes through a 3.x.0 final release and so cannot be reverted anymore. Victor Night gathers, and now my watch begins. It shall not end until my death.

Kyle Stanley

7 Dec 7 Dec

12:46 a.m.

Victor Stinner wrote:

...

Isn't it already the current unwritten deprecation process?

Personally, I don't think we should rely on an unwritten process for something as important and potentially breaking as a deprecation process. Regardless of the outcome of this discussion, I think we should try to fully write out the process in the devguide; while still providing some amount of flexibility. Victor Stinner wrote:

...

I'm not sure about enforcing the removal. Some APIs were deprecated, but it was unsure if we really wanted to remove them. Or because we had no idea how long it would take for developers to upgrade to the new functions. ... I would suggest to schedule removal, but not require it too strongly ;-)

Would it be reasonable to require an minimum amount of versions to be specified (such as n versions ahead), but provide flexibility in terms to delaying the removal, as needed? IMO, it would be more convenient for users to have a "minimum removal" version in mind, rather than an unscheduled deprecation that could have a version+2 (deprecation and removal in 2 versions) assigned at any point in time. This can be difficult sometimes when we're not sure what n versions from now will actually be called (such as with 3.10 and 4.0), but I don't think it would be an issue to state something like this: "func() has been deprecated, and will be removed in no sooner than 4 versions after 3.9" instead of: "func() has been deprecated, and will be removed in 3.13" for long term deprecations. By "long term", I mean more than two versions. Victor Stinner wrote:

...

I know that it's an unpopular opinion, but I would strongly prefer Python 4 to behave exactly as any another Python 3.x release in term of deprecation. If I exaggerate, I would prefer that Python 4.0 would have *less* incompatible changes than the last Python 3.x release.

My understanding was that we were specifically waiting on considering a Python 4.0 until there were major enough improvements/changes (such as the "GILectomy", for example) to justify some degree of incompatibility. If it would behave exactly the same as a standard 3.x release in terms to incompatible changes (or have less), what would be the purpose of making a major version change in the first place? I don't see an issue with indefinitely continuing the 3.x line in the meantime. Victor Stinner wrote:

...

I like to remove deprecated features at the beginning of a dev cycle to get users feedback earlier. If they are removed late in the dev cycle, there is a higher risk that a problematic incompatible change goes through a 3.x.0 final release and so cannot be reverted anymore.

I agree, the earlier a deprecation occurs in a given release cycle, the easier it is for users to provide feedback and then potentially prepare for the change. Perhaps we could establish some form of guideline, for example: "When possible, the removal of previously deprecated features should occur as early as possible in a version's dev cycle, preferably during the alpha phases. This provides users more time to provide feedback and plan for the potential removal". On Fri, Dec 6, 2019 at 10:58 AM Victor Stinner wrote:

...

Hi,

...
What do people think of the idea of requiring all deprecations specifying a version that the feature will be removed in (which under our annual release cadence would be at least the third release from the start of the deprecation, hence the deprecation being public for 2 years)? And

Le mer. 27 nov. 2019 à 19:40, Brett Cannon a écrit : that we also follow through with that removal?

Isn't it already the current unwritten deprecation process? The common case is to schedule the removal. Previously, it was common to wait 2 releases before removing anything: pending deprecatation in 3.6, deprecation in 3.7, removal in 3.8. It means that developers are warned during 2 releases: 3 years (I'm optimistic and consider that developers pay attention to PendingDeprecationWarning). So deprecation during 3 releases with the new release cadence would fit the previous practice: 3 years ;-)

Are you suggesting to start with a PendingDeprecationWarning? I recall a debate about the value of this warning which is quiet by default, whereas DeprecationWarning is now displayed in the __main__ module.

I'm not sure about enforcing the removal. Some APIs were deprecated, but it was unsure if we really wanted to remove them. Or because we had no idea how long it would take for developers to upgrade to the new functions.

For example, the C API for the old Unicode API using "Py_UNICODE*" type is deprecated since Python 3.3 but there is no concrete plan to remove it (just a vague "Python 4.0 removal" plan).

I deprecated the support for bytes filenames on Windows in Python 3.3, but Steve Dower implemented his PEP 529 (use UTF-8 for bytes on Windows) in Python 3.6 and so the deprecation warnings have been simply removed!

I would suggest to schedule removal, but not require it too strongly ;-)

...
Now I'm not suggesting it **must** be in three feature releases. A deprecation could be set to Python 4 (...)

I know that it's an unpopular opinion, but I would strongly prefer Python 4 to behave exactly as any another Python 3.x release in term of deprecation. If I exaggerate, I would prefer that Python 4.0 would have *less* incompatible changes than the last Python 3.x release. I hope that upgrading from Python 3 to Python 4 will be as smooth as possible. Changing sys.version_info.major and "python4" program name are already going to cause enough troubles.

Recently, I looked at all deprecation changes scheduled for removal in Python 4.0. I removed these features in Python 3.9, especially for features deprecated for more than 2 releases. See: https://docs.python.org/dev/whatsnew/3.9.html#removed

For example, I removed the "U" mode of open() which was deprecated since Python 3.4 and scheduled for removal in Python 4.0.

My plan is to revert these changes if too many users complain (see my rejected PEP 606 and 608 :-D).

--

I like to remove deprecated features at the beginning of a dev cycle to get users feedback earlier. If they are removed late in the dev cycle, there is a higher risk that a problematic incompatible change goes through a 3.x.0 final release and so cannot be reverted anymore.

Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DLHMA5BZ... Code of Conduct: http://python.org/psf/codeofconduct/

Serhiy Storchaka

1:29 p.m.

06.12.19 21:16, Kyle Stanley пише:

...

Would it be reasonable to require an minimum amount of versions to be specified (such as n versions ahead), but provide flexibility in terms to delaying the removal, as needed? IMO, it would be more convenient for users to have a "minimum removal" version in mind, rather than an unscheduled deprecation that could have a version+2 (deprecation and removal in 2 versions) assigned at any point in time.

We have a special deprecated-removed directive in docs for this.

1599

Age (days ago)

1609

Last active (days ago)

List overview

Download

10 comments

8 participants

participants (8)

Batuhan Taskaya
Brett Cannon
Eric Fahlgren
Ethan Furman
Kyle Stanley
Nick Coghlan
Serhiy Storchaka
Victor Stinner

Should we require all deprecations to have a removal version that we follow through on?

tags

participants (8)