Towards removing asynchat, asyncore and smtpd from the stdlib
The asynchat [1] and asyncore [2] libraries have been deprecated since 3.6 in favor of asyncio [3]. In addition, smtpd [4], which depends on asynchat and asyncore, is deprecated in favor of aiosmtpd [5]. The documentation for each of these libraries mentions that it is deprecated and refers to the suggested replacement. However, they do not yet emit DeprecationWarnings at import, so people may be using them without being aware that they are no longer maintained (there are a few dozen open issues on bpo, some of them reporting bugs). The next step is to add deprecation warnings, so that we can eventually delete them. There is also the issue that some of the stdlib tests are still using these libraries, but this does not need to block removing them from the stdlib because we can move them to test.support until those tests are rewritten. Note that nobody needs to stop using these libraries if they don't want to migrate to the suggested alternatives - the modules are pure python and anyone can include a copy in their own codebase and continue using it in a legacy application (or even continue maintaining it if they want). Barry and I are working on a patch to add deprecation warnings in 3.10 when one of these are imported [6]. Let us know if you have any comments on this plan. [1] https://docs.python.org/3/library/asynchat.html [2] https://docs.python.org/3/library/asyncore.html [3] https://docs.python.org/3/library/asyncio.html [4] https://docs.python.org/3/library/smtpd.html [5] https://aiosmtpd.readthedocs.io/en/latest/ [6] https://github.com/python/cpython/pull/26882
On 23. 06. 21 23:49, Irit Katriel via Python-Dev wrote:
Barry and I are working on a patch to add deprecation warnings in 3.10 when one of these are imported [6]. Let us know if you have any comments on this plan.
With my Fedora Python maintainer hat on, I am not particularly happy about this. DeprecationWarnings have a tendency to break tests. I assumed Python 3.10.0b1 to be the last release that intentionally breaks things that worked with Python 3.9. It's not a strong opinion and if there is a consensus that adding new DeprecationWarnings is not considered "new feature" or "breaking change", I won't fight it, but it feels wrong. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
Miro, what tests (outside of Python itself) do you think may break, and do you have a way to check that? -Barry On Wed, Jun 23, 2021, at 17:15, Miro Hrončok wrote:
On 23. 06. 21 23:49, Irit Katriel via Python-Dev wrote:
Barry and I are working on a patch to add deprecation warnings in 3.10 when one of these are imported [6]. Let us know if you have any comments on this plan.
With my Fedora Python maintainer hat on, I am not particularly happy about this.
DeprecationWarnings have a tendency to break tests. I assumed Python 3.10.0b1 to be the last release that intentionally breaks things that worked with Python 3.9.
It's not a strong opinion and if there is a consensus that adding new DeprecationWarnings is not considered "new feature" or "breaking change", I won't fight it, but it feels wrong.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PIXK73MX... Code of Conduct: http://python.org/psf/codeofconduct/
On 24. 06. 21 0:35, Barry Warsaw wrote:
Miro, what tests (outside of Python itself) do you think may break, and do you have a way to check that?
Any tests that import from asynchat, asyncore or smtpd (in the tests or in the tested code, even transitively trough other projects) if DeprecationWarnings are treated as errors by default -- many projects do this in tests (which is a very good way to get alerted of such deprecations when they happen) or even in setup.py (which I found rather weird). Some examples of such failures with different new DeprecationWarnings from the recent Python 3.10 pre-releases testing (mostly fixed now): - jinja2, subprocess-tee, sqlalchemy and more failed to build with Python 3.10: DeprecationWarning: There is no current event loop (treated as error) - colcon-ros, ruamel-yaml, flake8 and more failed to build with Python 3.10: DeprecationWarning: The distutils package deprecated and slated for removal in Python 3.12 (got it via setuptools usage, treated as error) - cherrypy failed to build with Python 3.10: pytest exited with no clear error message [1] Yes, we have a way to check all Fedora's Python packages by reusing our Python 3.10 pre-releases test-rebuild-everything mechanism, but it takes a few days to finish the builds and analyze the failures. Test failures caused by DeprecationWarnings are sometimes not obvious, e.g. not recognizable directly from the logs -- especially if they obscure some output that's checked for equality or line count (and all you get in the log is AssertionError: 20 != 21), or when they exit entire pytest session without any error message (arguably, that does not happen that often [1]). When such problems are found, it takes some time to report the problem to upstreams, fix the problem or workaround it (e.g. by filtering or ignoring the warning, which is not always trivial, especially when it propagates trough subprocess [2]). At this point of the release cycle, I'd rather recognize, triage and report regressions. [1] https://github.com/cherrypy/cherrypy/issues/1914#issuecomment-848780246 [2] https://github.com/pytest-dev/pytest/pull/8664
-Barry
On Wed, Jun 23, 2021, at 17:15, Miro Hrončok wrote:
On 23. 06. 21 23:49, Irit Katriel via Python-Dev wrote:
Barry and I are working on a patch to add deprecation warnings in 3.10 when
one
of these are imported [6]. Let us know if you have any comments on this plan.
With my Fedora Python maintainer hat on, I am not particularly happy about this.
DeprecationWarnings have a tendency to break tests. I assumed Python 3.10.0b1 to be the last release that intentionally breaks things that worked with Python 3.9.
It's not a strong opinion and if there is a consensus that adding new DeprecationWarnings is not considered "new feature" or "breaking change", I won't fight it, but it feels wrong.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
_______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ <https://mail.python.org/mailman3/lists/python-dev.python.org/> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PIXK73MX... <https://mail.python.org/archives/list/python-dev@python.org/message/PIXK73MX...> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SI4Z777Q... Code of Conduct: http://python.org/psf/codeofconduct/
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
On Jun 23, 2021, at 16:34, Miro Hrončok <mhroncok@redhat.com> wrote:
Yes, we have a way to check all Fedora's Python packages by reusing our Python 3.10 pre-releases test-rebuild-everything mechanism, but it takes a few days to finish the builds and analyze the failures. Test failures caused by DeprecationWarnings are sometimes not obvious, e.g. not recognizable directly from the logs -- especially if they obscure some output that's checked for equality or line count (and all you get in the log is AssertionError: 20 != 21), or when they exit entire pytest session without any error message (arguably, that does not happen that often [1]).
When such problems are found, it takes some time to report the problem to upstreams, fix the problem or workaround it (e.g. by filtering or ignoring the warning, which is not always trivial, especially when it propagates trough subprocess [2]).
At this point of the release cycle, I'd rather recognize, triage and report regressions.
Given that we have a month and change from the first release candidate, that should be enough time to fix any test failures due to these DeprecationWarnings. I defer to the RM but Pablo already approved it. :D https://github.com/python/cpython/pull/26882 adds import time DeprecationWarnings to asyncore, asynchat, and smtpd. Cheers, -Barry
On Wed, Jun 23, 2021 at 10:49:04PM +0100, Irit Katriel via Python-Dev wrote:
The next step is to add deprecation warnings, so that we can eventually delete them. There is also the issue that some of the stdlib tests are still using these libraries, but this does not need to block removing them from the stdlib because we can move them to test.support until those tests are rewritten.
This is a very good idea. I also read in another discussion that offering these as pip installable external packages could be considered, either by the core-dev or by community, just in case there is a reliance on these packages in production code and the developers don't want those the break and aren't ready to upgrade or migrate. Thank you, Senthil
participants (4)
-
Barry Warsaw -
Irit Katriel -
Miro Hrončok -
Senthil Kumaran