[Python-Dev] Usefulness of binary compatibility accross Python versions?

[Antoine Pitrou]

On Sat, 16 Dec 2017 09:34:02 -0800
Guido van Rossum <guido at python.org> wrote:
> I think it's more acceptable to require matching versions now than it was
> 10 years ago -- people are much more likely to use installer tools like pip
> and conda that can check version compatibility.
> 
> I think I'd be okay with dropping the flag-based mechanism you describe if
> we were to introduce a clear mechanism that always rejected a dynamically
> loaded module if it was compiled for a different Python version. This
> should happen without any cooperation from the module. Perhaps in Python.h
> we can introduce a reference to a variable whose name varies by version
> (major.minor, I think) and which is defined only by the interpreter itself.
> Or perhaps the version should be based on a separate ABI version.

Interestingly, Python 2 had such an API version check (though it would
only emit a warning), it was removed as part of PEP 3121:
https://github.com/python/cpython/commit/1a21451b1d73b65af949193208372e86bf308411#diff-4664e4ea04dc636b18070ba01cf42d06L39

I haven't been able to find the pre-approval discussion around PEP
3121, so I'm not sure why the API check was removed.  The PEP (quite
short!) also says nothing about it.

Currently, you can pass a `module_api_version` to PyModule_Create2(),
but that function is for specialists only :-)

("""Most uses of this function should be using PyModule_Create()
instead; only use this if you are sure you need it.""")

And the new multi-phase initialization API doesn't seem to support
passing an API version:
https://docs.python.org/3/c-api/module.html#multi-phase-initialization


Fortunately, nowadays all major platforms (Windows, Linux, macOS) tag
C extension filenames with the interpreter version (*), e.g.:
- "XXX.cpython-35m-darwin.so" on macOS
- "XXX?.cp35-win32.pyd" on Windows
- "XXX?.cpython-37dm-x86_64-linux-gnu.so" on Linux and others

So in practice there should be little potential for confusion, except
when renaming the extension file?

(*) references:
https://bugs.python.org/issue22980
https://docs.python.org/3.7/whatsnew/3.5.html#build-and-c-api-changes


Regards

Antoine.