cython 3 migration update and next releases
Hi All, I would like to inform you about recent porting of projects to Cython 3. Recently, I participated in migration of 3 bigger projects to Cython 3: * Scipy - scipy main branch is currently compilable with Cython 3 with test suite passing. Cython 3 master branch is used in CI to ensure that all changes are compatible with Cython 3. * Pandas - pandas library is compilable with Cython 3, the test suite is passing except one test case (not caused by Cython in my opinion) - https://github.com/pandas-dev/pandas/issues/53125 * scikit-image - library is compilable with Cython 3 (test suite was not run though) When migrating to Cython 3, I was able to find out several issues in the Cython, all of them are merged in master now. Hence, I would like to ask about next steps. It would help greatly to release Cython 3 beta3. This will allow me to pin scipy CI to real pre-release instead of master branch. Moreover, I would like to ask whether we can do the final Cython 3 release after beta 3. The rationale is that the projects won't start really using Cython 3 until we do the final release. Now, we have 3 big users of Cython migrated, hence I think we have some confidence that Cython 3 is ready. What do you think? Thanks, Matus
On Tue, 16 May 2023 at 22:10, matus valo <matusvalo@gmail.com> wrote:
The rationale is that the projects won't start really using Cython 3 until we do the final release. Now, we have 3 big users of Cython migrated, hence I think we have some confidence that Cython 3 is ready. What do you think?
I would not say that mpi4py is a big project, but it is around 15K lines of Cython code, and it is also ready for Cython 3. I'll update mpi4py requirements to `cython>=3` as soon as Cython 3 is released. -- Lisandro Dalcin ============ Senior Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/
On 17/05/2023 14:44, Lisandro Dalcin wrote:
On Tue, 16 May 2023 at 22:10, matus valo <matusvalo@gmail.com> wrote:
The rationale is that the projects won't start really using Cython 3 until we do the final release. Now, we have 3 big users of Cython migrated, hence I think we have some confidence that Cython 3 is ready. What do you think?
I would not say that mpi4py is a big project, but it is around 15K lines of Cython code, and it is also ready for Cython 3. I'll update mpi4py requirements to `cython>=3` as soon as Cython 3 is released.
IFAIK, 15k lines of Cython makes it among one of the largest Cython projects I'm aware of (I did some research a couple of years ago): https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10... S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Founder & Software Architect - Nua open source PaaS project - https://nua.rocks/ Co-Founder & Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Co-Founder & Board Member, Association Professionnelle Européenne du Logiciel Libre (APELL) - https://www.apell.info/ Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) - https://www.euclidia.eu/
matus valo schrieb am 16.05.23 um 21:09:
I would like to inform you about recent porting of projects to Cython 3. Recently, I participated in migration of 3 bigger projects to Cython 3:
Thanks a lot for doing this, Matúš. It helps Cython as much as it helps these projects.
When migrating to Cython 3, I was able to find out several issues in the Cython, all of them are merged in master now. Hence, I would like to ask about next steps. It would help greatly to release Cython 3 beta3. This will allow me to pin scipy CI to real pre-release instead of master branch.
I'll try to get beta 3 released soon, but need to find a bit of consecutive time to get it out. There are still a couple of PRs that I'd like to look through.
Moreover, I would like to ask whether we can do the final Cython 3 release after beta 3. The rationale is that the projects won't start really using Cython 3 until we do the final release. Now, we have 3 big users of Cython migrated, hence I think we have some confidence that Cython 3 is ready. What do you think?
It's probably a good time to have a final call for merges. Promoting and voting for PRs is welcome. Stefan
On 20/05/2023 10:59, Stefan Behnel wrote:
It's probably a good time to have a final call for merges. Promoting and voting for PRs is welcome.
I've flagged a couple of PRs for possible merging. They're both fairly minor Python-compatibility fixes so it probably isn't a disaster if they're missed though. Unfortunately, I doubt I can get the module state refactor into a usable state in time. David
SageMath has 700K Cython lines, yet not mentioned. On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf@fermigier.com> wrote:
On 17/05/2023 14:44, Lisandro Dalcin wrote:
On Tue, 16 May 2023 at 22:10, matus valo <matusvalo@gmail.com> wrote:
The rationale is that the projects won't start really using Cython 3 until we do the final release. Now, we have 3 big users of Cython migrated, hence I think we have some confidence that Cython 3 is ready. What do you think?
I would not say that mpi4py is a big project, but it is around 15K lines of Cython code, and it is also ready for Cython 3. I'll update mpi4py requirements to `cython>=3` as soon as Cython 3 is released.
IFAIK, 15k lines of Cython makes it among one of the largest Cython projects I'm aware of (I did some research a couple of years ago):
https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10...
S.
--
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/
Founder & Software Architect - Nua open source PaaS project - https://nua.rocks/
Co-Founder & Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/
Co-Founder & Board Member, Association Professionnelle Européenne du Logiciel Libre (APELL) - https://www.apell.info/
Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) - https://www.euclidia.eu/ _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Dima Pasechnik schrieb am 21.05.23 um 11:38:
On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf@fermigier.com> wrote:
IFAIK, 15k lines of Cython makes it among one of the largest Cython projects I'm aware of (I did some research a couple of years ago):
https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10...
SageMath has 700K Cython lines, yet not mentioned.
Certainly worth mentioning, yes. Looking at the numbers, I also noticed that lxml is listed in the 5-10k lines range. It actually has about 18k lines of Cython code (.pyx/.pxi files) and another 1.5k lines in compiled Python (.py) files, according to pygount [1]. I tried sloccount first, but that doesn't seem to have Cython support. Might be worth redoing that count for the other projects as well. Stefan [1] https://pypi.org/project/pygount/
On thing to take care of before the release is to get rid of the hugely outdated advice to ship generated C/C++ files! On Sun, May 21, 2023 at 11:24 AM Stefan Behnel <stefan_ml@behnel.de> wrote:
Dima Pasechnik schrieb am 21.05.23 um 11:38:
On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf@fermigier.com> wrote:
IFAIK, 15k lines of Cython makes it among one of the largest Cython projects I'm aware of (I did some research a couple of years ago):
https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10...
SageMath has 700K Cython lines, yet not mentioned.
Certainly worth mentioning, yes.
Looking at the numbers, I also noticed that lxml is listed in the 5-10k lines range. It actually has about 18k lines of Cython code (.pyx/.pxi files) and another 1.5k lines in compiled Python (.py) files, according to pygount [1]. I tried sloccount first, but that doesn't seem to have Cython support.
Might be worth redoing that count for the other projects as well.
Stefan
[1] https://pypi.org/project/pygount/
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
participants (6)
-
da-woods -
Dima Pasechnik -
Lisandro Dalcin -
matus valo -
Stefan Behnel -
Stefane Fermigier