On Wed, Jun 3, 2020 at 11:30 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
As I wrote in a previous email, the C API of CPython is evolving slowly to hide more and more implementation details. Sadly, some changes break the backward compatibility. Sometimes, a new function must be used, except that the function doesn't exist in older Python versions.
"Update on C API changes to hide implementation details" https://mail.python.org/archives/list/capi-sig@python.org/thread/HYFQHLGNCRF...
Would it be possible to ship a header file (a single ".h" file) and/or even a dynamic library to provide new functions to old Python versions? I'm not sure if it's possible to ship a header file as a package on PyPI and make it available for C compilers easily without monkey patching distutils. Another option is to encourage Linux distributions to package such header file to make it easy to install on the system.
Another (worse?) option is to advise authors of each C extension module to vendor a copy of the header file in their project and update it from time to time when needed.
If you can provide backward compatibility, the obvious question for a C extension developer is WHY ARE YOU CHANGING THE API IN THE FIRST PLACE?
Now I happen to follow the Python dev and CAPI mailing lists, so yes I do know why you want to change the API, you don't have to explain it to me. But I can imagine this being interpreted as the all too common "churn" rather than actual improvement, breaking people's code just because some dev wants to change stuff.
My suggestion is that for the first, or first few, versions of Python with the new API, this backwards compatibility header and any functions be shipped as part of the standard library. Extension developers can either update their code to the new API, or add a new line #include <PythonCompat.h> and everything works but they get a nag message about needing to update.
(Actually, it might be better if an extension module compiled with the backward compat lib gives a runtime nag warning, so users put pressure on the author.)
Making extension developers jump through hoops to get their old code to work will just suck up time and effort that they could have been using to update the code. My suggestion is to have a grace period for deprecated code, then a clean break.
--
cheers,
Hugh Fisher