Please explain how to migrate when a function is removed, thanks ;-)
Hi, We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to build. I'm not here to talk about DeprecationWarning, but how we communicate on incompatible changes made on purpose. For example, What's New in Python 3.8 announces: "In asyncio, the explicit passing of a loop argument has been deprecated and will be removed in version 3.10". As expected, the parameter was removed in Python 3.10 (bpo-42392), but I cannot see anything in What's New in Python 3.10. The problem is that I don't know how to fix projects broken by this change. I'm not complaining about this specific change, but more generally. I strongly suggest to well document incompatible changes. The bare minimum is to mention them in "Porting to Python 3.10" section: https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10 A link to the bpo sometimes helps to understand how to port code. But I would really appreciate if authors of incompatible changes would explain how to add Python 3.10 support to existing projects, without losing support for older Python versions. Not just "this function is now removed, good luck!" :-) I didn't touch asyncio for at least 1 year, so I don't know what happens if I remove a loop argument. Does an application remain compatible with Python 3.6 without passing loop? I know that I made multiple incompatible changes myself and I'm sure that the documentation should probably be enhanced, but I also expect others to help on that! ;-) Please think about people who have to port 4000 Python projects to Python 3.10! --- The best would be to ship a tool which adds Python 3.10 support to existing projects without losing support with Python 3.6. Maybe something like https://github.com/asottile/pyupgrade could be used for that? pyupgrade looks more specific to the Python syntax, than the usage of the stdlib. I wrote such tool to add Python 3.10 support to C extensions without losing support with Python 2.7. It relies on a header file (pythoncapi_compat.h) which provides new C API functions on old Python versions. https://github.com/pythoncapi/pythoncapi_compat For example, it replaces "obj->ob_refcnt = refcnt;" with "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python 3.8 and older (function added to Python 3.9) Victor -- Night gathers, and now my watch begins. It shall not end until my death.
A PR was proposed to document the removal of the loop parameter: https://github.com/python/cpython/pull/24256 Victor On Tue, Jan 19, 2021 at 1:34 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to build.
I'm not here to talk about DeprecationWarning, but how we communicate on incompatible changes made on purpose. For example, What's New in Python 3.8 announces: "In asyncio, the explicit passing of a loop argument has been deprecated and will be removed in version 3.10". As expected, the parameter was removed in Python 3.10 (bpo-42392), but I cannot see anything in What's New in Python 3.10. The problem is that I don't know how to fix projects broken by this change. I'm not complaining about this specific change, but more generally.
I strongly suggest to well document incompatible changes. The bare minimum is to mention them in "Porting to Python 3.10" section: https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10
A link to the bpo sometimes helps to understand how to port code. But I would really appreciate if authors of incompatible changes would explain how to add Python 3.10 support to existing projects, without losing support for older Python versions. Not just "this function is now removed, good luck!" :-)
I didn't touch asyncio for at least 1 year, so I don't know what happens if I remove a loop argument. Does an application remain compatible with Python 3.6 without passing loop?
I know that I made multiple incompatible changes myself and I'm sure that the documentation should probably be enhanced, but I also expect others to help on that! ;-)
Please think about people who have to port 4000 Python projects to Python 3.10!
---
The best would be to ship a tool which adds Python 3.10 support to existing projects without losing support with Python 3.6. Maybe something like https://github.com/asottile/pyupgrade could be used for that? pyupgrade looks more specific to the Python syntax, than the usage of the stdlib.
I wrote such tool to add Python 3.10 support to C extensions without losing support with Python 2.7. It relies on a header file (pythoncapi_compat.h) which provides new C API functions on old Python versions. https://github.com/pythoncapi/pythoncapi_compat
For example, it replaces "obj->ob_refcnt = refcnt;" with "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python 3.8 and older (function added to Python 3.9)
Victor -- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.
Thanks for bringing attention to this, Victor, and to Ken Jin (GH: Fidget-Spinner) for the PR. I've just completed reviewing and merging the PR, so hopefully anyone affected will now have a more clear idea of how to migrate their asyncio code to 3.10. Having the porting method explicitly documented certainly helps to smooth the version transition process and reduce headaches. :-) Going forward, I'll try to make more of an active effort to ensure any potentially incompatible changes I'm involved with include a clear method of porting documented in their respective whatsnew. It can be easy to forget at times that a seemingly minor fix which is intuitively clear to the authors of a change may not be as clear to those not involved with it, regardless of how difficult the fix actually is. On Tue, Jan 19, 2021 at 12:03 PM Victor Stinner <vstinner@python.org> wrote:
A PR was proposed to document the removal of the loop parameter: https://github.com/python/cpython/pull/24256
Victor
On Tue, Jan 19, 2021 at 1:34 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to build.
I'm not here to talk about DeprecationWarning, but how we communicate on incompatible changes made on purpose. For example, What's New in Python 3.8 announces: "In asyncio, the explicit passing of a loop argument has been deprecated and will be removed in version 3.10". As expected, the parameter was removed in Python 3.10 (bpo-42392), but I cannot see anything in What's New in Python 3.10. The problem is that I don't know how to fix projects broken by this change. I'm not complaining about this specific change, but more generally.
I strongly suggest to well document incompatible changes. The bare minimum is to mention them in "Porting to Python 3.10" section: https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10
A link to the bpo sometimes helps to understand how to port code. But I would really appreciate if authors of incompatible changes would explain how to add Python 3.10 support to existing projects, without losing support for older Python versions. Not just "this function is now removed, good luck!" :-)
I didn't touch asyncio for at least 1 year, so I don't know what happens if I remove a loop argument. Does an application remain compatible with Python 3.6 without passing loop?
I know that I made multiple incompatible changes myself and I'm sure that the documentation should probably be enhanced, but I also expect others to help on that! ;-)
Please think about people who have to port 4000 Python projects to
Python 3.10!
---
The best would be to ship a tool which adds Python 3.10 support to existing projects without losing support with Python 3.6. Maybe something like https://github.com/asottile/pyupgrade could be used for that? pyupgrade looks more specific to the Python syntax, than the usage of the stdlib.
I wrote such tool to add Python 3.10 support to C extensions without losing support with Python 2.7. It relies on a header file (pythoncapi_compat.h) which provides new C API functions on old Python versions. https://github.com/pythoncapi/pythoncapi_compat
For example, it replaces "obj->ob_refcnt = refcnt;" with "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python 3.8 and older (function added to Python 3.9)
Victor -- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CVLDV7VI... Code of Conduct: http://python.org/psf/codeofconduct/
I don't know if this is already covered in the discussion and in our processes, but in addition to documenting the instructions in the release in which things break, I think it would also be good to include such instructions in any earlier release in which the thing is merely deprecated. In those cases, the instructions would be the same, but they would be on how to make the deprecation warning go away rather than to unbreak things. That way, people have the information to make their code forward-compatible earlier, as opposed to waiting for the breaking release. Personally, I like to address DeprecationWarnings when I first see them, as opposed to waiting until later. Is there / would it make sense to have a section analogous to "Porting to Python X" that covers "Make All DeprecationWarnings Go Away in X"? If we had such a section, the "Porting to" section could be constructed by copying the relevant bits from that section in the previous release. --Chris On Wed, Jan 20, 2021 at 4:47 PM Kyle Stanley <aeros167@gmail.com> wrote:
Thanks for bringing attention to this, Victor, and to Ken Jin (GH: Fidget-Spinner) for the PR. I've just completed reviewing and merging the PR, so hopefully anyone affected will now have a more clear idea of how to migrate their asyncio code to 3.10. Having the porting method explicitly documented certainly helps to smooth the version transition process and reduce headaches. :-)
Going forward, I'll try to make more of an active effort to ensure any potentially incompatible changes I'm involved with include a clear method of porting documented in their respective whatsnew. It can be easy to forget at times that a seemingly minor fix which is intuitively clear to the authors of a change may not be as clear to those not involved with it, regardless of how difficult the fix actually is.
On Tue, Jan 19, 2021 at 12:03 PM Victor Stinner <vstinner@python.org> wrote:
A PR was proposed to document the removal of the loop parameter: https://github.com/python/cpython/pull/24256
Victor
On Tue, Jan 19, 2021 at 1:34 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to build.
I'm not here to talk about DeprecationWarning, but how we communicate on incompatible changes made on purpose. For example, What's New in Python 3.8 announces: "In asyncio, the explicit passing of a loop argument has been deprecated and will be removed in version 3.10". As expected, the parameter was removed in Python 3.10 (bpo-42392), but I cannot see anything in What's New in Python 3.10. The problem is that I don't know how to fix projects broken by this change. I'm not complaining about this specific change, but more generally.
I strongly suggest to well document incompatible changes. The bare minimum is to mention them in "Porting to Python 3.10" section: https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10
A link to the bpo sometimes helps to understand how to port code. But I would really appreciate if authors of incompatible changes would explain how to add Python 3.10 support to existing projects, without losing support for older Python versions. Not just "this function is now removed, good luck!" :-)
I didn't touch asyncio for at least 1 year, so I don't know what happens if I remove a loop argument. Does an application remain compatible with Python 3.6 without passing loop?
I know that I made multiple incompatible changes myself and I'm sure that the documentation should probably be enhanced, but I also expect others to help on that! ;-)
Please think about people who have to port 4000 Python projects to
Python 3.10!
---
The best would be to ship a tool which adds Python 3.10 support to existing projects without losing support with Python 3.6. Maybe something like https://github.com/asottile/pyupgrade could be used for that? pyupgrade looks more specific to the Python syntax, than the usage of the stdlib.
I wrote such tool to add Python 3.10 support to C extensions without losing support with Python 2.7. It relies on a header file (pythoncapi_compat.h) which provides new C API functions on old Python versions. https://github.com/pythoncapi/pythoncapi_compat
For example, it replaces "obj->ob_refcnt = refcnt;" with "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python 3.8 and older (function added to Python 3.9)
Victor -- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CVLDV7VI... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RGSEPSSP... Code of Conduct: http://python.org/psf/codeofconduct/
Currently, we wait until the first user complains, sometimes after a 3.x.0 final release, before starting to document how to port existing code. I agree that it should be done earlier. I suggest that developers who want to introduce an incompatible change think about how to port existing code, especially if it's possible to write code compatible with the old and the new behavior. It should be done while the incompatible change is designed. I also agree that we document how to port code in the change which introduce the deprecation, not when we introduce the incompatible change (ex: remove a function). I also suggest to communicate on future incompatible changes ;-) Victor
I agree that when we land a feature that introduces incompatible change like this, as part of the PR it should also include updating the documentation on how to migrate. I would think that the feature should not be merged unless the doc has been updated too. Perhaps we should include this explicitly in devguide, as one of the things to consider when reviewing pull requests. Do we have such a guide/doc yet, on how to review CPython pull requests? On Thu, Jan 21, 2021 at 2:49 AM Victor Stinner <vstinner@python.org> wrote:
Currently, we wait until the first user complains, sometimes after a 3.x.0 final release, before starting to document how to port existing code. I agree that it should be done earlier.
I suggest that developers who want to introduce an incompatible change think about how to port existing code, especially if it's possible to write code compatible with the old and the new behavior. It should be done while the incompatible change is designed.
I also agree that we document how to port code in the change which introduce the deprecation, not when we introduce the incompatible change (ex: remove a function).
I also suggest to communicate on future incompatible changes ;-)
Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/F4YKK4ER... Code of Conduct: http://python.org/psf/codeofconduct/
On Jan 20, 2021 at 9:51 PM Chros Jerdonek <chris.jerdonek@gmail.com> wrote:
Is there / would it make sense to have a section analogous to "Porting to Python X" that covers "Make All DeprecationWarnings Go Away in X"? If we had such a section, the "Porting to" section could be constructed by copying the relevant bits from that section in the previous release.
+1 from me to include this section in 3.10 and future whatsnew. On Thu, Jan 21, 2021 at 2:17 PM Mariatta <mariatta@python.org> wrote:
I agree that when we land a feature that introduces incompatible change like this, as part of the PR it should also include updating the documentation on how to migrate. I would think that the feature should not be merged unless the doc has been updated too.
Perhaps we should include this explicitly in devguide, as one of the things to consider when reviewing pull requests. Do we have such a guide/doc yet, on how to review CPython pull requests?
This seems like it would be a good location to include that information: https://devguide.python.org/pullrequest/#how-to-review-a-pull-request Maybe also a brief mention in the checklist for https://devguide.python.org/committing/#is-the-pr-ready-to-be-accepted. Step 9 currently has: "Was “What’s New” updated (as appropriate)?", which could have a sub-point to mention "Incompatible changes should include migration information in their respective "What's New".".
I opened an issue on Devguide: https://github.com/python/devguide/issues/655 On Thu, Jan 21, 2021 at 11:37 AM Kyle Stanley <aeros167@gmail.com> wrote:
On Jan 20, 2021 at 9:51 PM Chros Jerdonek <chris.jerdonek@gmail.com> wrote:
Is there / would it make sense to have a section analogous to "Porting to Python X" that covers "Make All DeprecationWarnings Go Away in X"? If we had such a section, the "Porting to" section could be constructed by copying the relevant bits from that section in the previous release.
+1 from me to include this section in 3.10 and future whatsnew.
On Thu, Jan 21, 2021 at 2:17 PM Mariatta <mariatta@python.org> wrote:
I agree that when we land a feature that introduces incompatible change like this, as part of the PR it should also include updating the documentation on how to migrate. I would think that the feature should not be merged unless the doc has been updated too.
Perhaps we should include this explicitly in devguide, as one of the things to consider when reviewing pull requests. Do we have such a guide/doc yet, on how to review CPython pull requests?
This seems like it would be a good location to include that information: https://devguide.python.org/pullrequest/#how-to-review-a-pull-request
Maybe also a brief mention in the checklist for https://devguide.python.org/committing/#is-the-pr-ready-to-be-accepted. Step 9 currently has: "Was “What’s New” updated (as appropriate)?", which could have a sub-point to mention "Incompatible changes should include migration information in their respective "What's New".".
participants (4)
-
Chris Jerdonek -
Kyle Stanley -
Mariatta -
Victor Stinner