Regressions caused the recent work on ceval.c and frame objects
Hi, The recent optimization work on ceval.c and frame objects introduced regressions and the situation is stuck for 4 months (bpo-43760). Right now, maybe the best would be to revert the 2 commits in the 3.10 branch, to get more time in Python 3.11 development cycle to solve these issues. (1) "The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)" https://bugs.python.org/issue43760 This change introduced an incompatible C API change. It's not documented in What's New in Python 3.10 and there is no solution for the 4 broken projects (including Cython). I proposed a C API but so far, nobody implemented it. Another option is to fix each project since the list is short. Right now, only 4 projects are known to be broken. Fixing Cython is not enough, you need to get a new release of broken projects (using Cython) to regenerate the C code with the updated Cython. (2) "Performance regression 3.10b1 and later on Windows: Py_DECREF() not inlined in PGO build" https://bugs.python.org/issue45116 Changes made in bpo-43760 caused a performance regression on the PGO build of Windows. It's a tricky issue about thresholds in compiler PGO optimization, inlining or not static inline functions, number of statements per function, etc. It was proposed to workaround the specific case of the huge _PyEval_EvalFrameDefault() function (3500 lines of C code) by converting again some static inline functions to macros. They were macros in Python 3.8 and were fine in Python 3.9. The performance regression with the Windows PGO build was introduced by the recent ceval.c work. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
Are you sure Cython is still broken? It looks like it was fixed back in May (https://github.com/cython/cython/issues/4153) and all the tests look to be passing on the 3.10-dev CI run for Cython. I think it only affected the profiling feature on Cython (which most people will have turned off) so probably won't cause widespread breakage. Which is not to say that you shouldn't fix the issue, but I don't think it's a disaster from Cython's point of view.
On 19. 09. 21 21:34, dw-git@d-woods.co.uk wrote:
Are you sure Cython is still broken? It looks like it was fixed back in May (https://github.com/cython/cython/issues/4153) and all the tests look to be passing on the 3.10-dev CI run for Cython. I think it only affected the profiling feature on Cython (which most people will have turned off) so probably won't cause widespread breakage.
Which is not to say that you shouldn't fix the issue, but I don't think it's a disaster from Cython's point of view.
Cython is fixed. However all the projects that ship Cython pre-generated C sources in sdists on PyPI that are affected and were not re-generated will not compile on Python 3.10. (Which might already be the case for different changes that affected Cython.) -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
Are projects that ship pre-compiled wheels impacted? Eg twisted-iocpsupport ? On Sun, 19 Sep 2021, 22:09 Miro Hrončok, <mhroncok@redhat.com> wrote:
Are you sure Cython is still broken? It looks like it was fixed back in May (https://github.com/cython/cython/issues/4153) and all the tests look to be passing on the 3.10-dev CI run for Cython. I think it only affected
On 19. 09. 21 21:34, dw-git@d-woods.co.uk wrote: the profiling feature on Cython (which most people will have turned off) so probably won't cause widespread breakage.
Which is not to say that you shouldn't fix the issue, but I don't think
it's a disaster from Cython's point of view.
Cython is fixed. However all the projects that ship Cython pre-generated C sources in sdists on PyPI that are affected and were not re-generated will not compile on Python 3.10. (Which might already be the case for different changes that affected Cython.)
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YHPXTAOH... Code of Conduct: http://python.org/psf/codeofconduct/
Will all packages that use Cython have to upgrade Cython to work with 3.10? That can be a pretty heavy lift. A lot of us pin Cython. -CHB On Sun, Sep 19, 2021 at 4:32 PM Miro Hrončok <mhroncok@redhat.com> wrote:
On 20. 09. 21 0:10, Thomas Grainger wrote:
Are projects that ship pre-compiled wheels impacted? Eg twisted-iocpsupport ?
I guess that if they managed to compile with 3.10, they shouldn't be.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TDGG4VFR... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Will all packages that use Cython have to upgrade Cython to work with 3.10?
For this particular issue you'll only have to upgrade if you use "profiling" (I doubt that many packages routinely build with Cython profiling turned on). However, it's possible there are other 3.10 bugfixes in Cython - I'm not completely sure. I think the easiest way to know is to try to build a package for 3.10.
msgpack 1.0.2 sdist includes c++ code generated by Cython 0.29.21 released at 2020-07-09. Python 3.10 can build it without any source modifications. So I don't expect Python 3.10 requires re-generation for most Cython users. Regards, On Mon, Sep 20, 2021 at 4:11 PM <dw-git@d-woods.co.uk> wrote:
Will all packages that use Cython have to upgrade Cython to work with 3.10?
For this particular issue you'll only have to upgrade if you use "profiling" (I doubt that many packages routinely build with Cython profiling turned on). However, it's possible there are other 3.10 bugfixes in Cython - I'm not completely sure. I think the easiest way to know is to try to build a package for 3.10. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/52VFBHR7... Code of Conduct: http://python.org/psf/codeofconduct/
-- Inada Naoki <songofacandy@gmail.com>
participants (6)
-
Christopher Barker -
dw-git＠d-woods.co.uk -
Inada Naoki -
Miro Hrončok -
Thomas Grainger -
Victor Stinner