On Wed, Feb 2, 2022 at 8:53 AM Thomas Caswell <tcaswell@gmail.com> wrote:

I disagree with point (3).

I think it would be better to discourage projects from including the output of cython in their sdists. They should either have cython as a build-time requirement or provide built wheels (which are specific a platform and CPython version). The middle ground of not expecting the user to have cython while expecting them to have a working c-complier is a very narrow case and I think asking those users to install cython is worth the forward compatibility for Python versions you get by requiring people installing from source to re-cythonize.

I don't think having C compiler but not cython is the "narrow" case. Requiring the latest Cython is installed is heavy additional dependency. Note that we don't require the latest C compiler. Users may install C compiler via tools like `dnf` or `apt`. But they can not install the latest Cython via `dnf` or `apt`. So this should be considered case-by-case basis. * If the project can provide wheels for very wide platforms, stop bundling C source is not a big problem. * If the project can not provide wheels for some reason (*), they want to bundle C source. * They can release source package more than a year. * Or they want to use "only public API" mode for better compatibility. (*) For example, my "mysqlclient" library provide wheel only on Windows because user may want to use own libmysqlclient, and I don't want to maintain binary linking OpenSSL. Regards, -- Inada Naoki <songofacandy@gmail.com>

On Tue, Feb 1, 2022 at 4:58 PM Stefan Behnel <stefan_ml@behnel.de> wrote:

I agree. Shipping the generated C sources was a very good choice as long as CPython's C-API was very stable and getting a build time dependency safely installed on user side was very difficult.

These days, it's the opposite way.

Exactly -- shipping the generated C source used to be the standard of practice, but with wheels (and conda) binaries are the way to go for users without a compiler. Cython can be pip installed -- so if you have a compiler that can compile C extensions, and you have Python, then getting Cython is trivial. I can't imagine a system with a compiler that can't pip install cython -- I suppose it's possible on an operational system, but then the user should build a wheel on a develop machine themselves. -CHB

Stefan

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KTWDJGHP... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Victor Stinner schrieb am 02.02.22 um 11:35:

I wish that there would be a 3rd option: ship C code generated by Cython *but* run Cython if this C code "looks" outdated, for example if building the C code fails with a compiler error.

So, one thing I did yesterday was to make sure that .c files get regenerated when a different Cython version is used at build time than what was used to generate them originally. Thinking about this some more now, I'm no longer sure that this is really a good idea, because it can lead to "random" build failures when a package does not pin its Cython version and a newer (or, probably worse, older) one happens to be installed at build time. Not sure how to best deal with this. I'm open to suggestions, although this might be the wrong forum. Let's discuss it in a ticket: https://github.com/cython/cython/issues/4611 Note that what you propose sounds more like a setuptools feature than a Cython feature, though.

So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Python" (Python 3.11) would not have to manually removed generated C code.

That sounds like an environment variable might help? I don't really want to add something like a "last supported CPython version". There is no guarantee that the code breaks between CPython versions, so that would just introduce an artificial support blocker.

In Fedora RPM packages of Python projects, we have to force manually running Cython. For example, the numpy package does: "rm PKG-INFO" with the comment: "Force re-cythonization (ifed for PKG-INFO presence in setup.py)". https://src.fedoraproject.org/rpms/numpy/blob/rawhide/f/numpy.spec#_107

In my pythonci project, I use a worse hack, I search for generated C files and remove them manually with this shell command:

rm -f -v $(grep -rl '/\* Generated by Cython') PKG-INFO

This command searchs for the pattern "/* Generated by Cython".

Right. Hacks like these are just awful. There must be a better way. Stefan

Victor Stinner schrieb am 02.02.22 um 23:23:

On Wed, Feb 2, 2022 at 3:54 PM Stefan Behnel wrote:

...

...

So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Python" (Python 3.11) would not have to manually removed generated C code.

That sounds like an environment variable might help?

Something like CYTHON_FORCE_REGEN=1 would be great :-)

https://github.com/cython/cython/commit/b859cf2bd72d525a724149a6e552abecf9cd... Note that this only applies when cythonize() is actually called. Some setup.py scripts may not do that unless requested to.

My use case is to use a project on the "next Python" version (the main branch) when the project contains outdated generated C code, whereas I have a more recent Cython version installed.

That use case would probably be covered by the Cython version check now, in case that stays in (the decision is pending user feedback). Stefan