On Sat, 28 Sep 2019 at 12:56, Victor Stinner <vstinner@python.org> wrote:
Hi,
I dislike having to do that, but I had to make a last minute change in my PEP 587 "Python Initialization Configuration" to allow to modify the structure in the future without breaking the backward compatibility. I added a "struct_size" field to PyPreConfig and PyConfig:
* https://bugs.python.org/issue38304 * https://github.com/python/peps/commit/afa38c0bef7738b8fcc3173daee8488e042083...
The example:
PyConfig config; PyStatus status = PyConfig_InitPythonConfig(&config); ...
must now be written:
PyConfig config; config.struct_size = sizeof(PyConfig); PyStatus status = PyConfig_InitPythonConfig(&config); ...
At the beginning, I used a private "_config_version" field which was initialized statically by a macro. But it was decided to replace macros with functions. I only noticed today that the conversion to function broke the API/ABI future compatibility.
PyConfig_InitPythonConfig() got uninitialized memory and didn't know the size of the config variable.
I don't quite understand the purpose of this change, as there's no stable ABI for applications embedding CPython. As a result, updating to a new X.Y.0 release always requires rebuilding the entire application, not just building and relinking CPython. I could understand a change to require passing in an expected Python version so we can fail more gracefully on a bad link where an application that intended to embed Python 3.8 is incorrectly linked against Python 3.9 (for example), but performing that kind of check would require passing in PY_VERSION_HEX, not the size of the config struct.
With my change, the function now requires the "struct_size" field to be set, and so it can support internally different versions of the PyConfig structure.
Storing the structure size directly in the structure is a common practice in the Windows API which is a good example of long term ABI compatibility.
This analogy doesn't hold, as Microsoft explicitly support running old binaries on new versions of Windows, and hence need a way to determine which version of the structure is being passed in. We don't support that - all our APIs that accept PyObject/PyTypeObject/etc require that the caller pass in structs of the correct size for the version of Python being used. The PyConfig and PyPreConfig structs are no different from PyObject in that regard: if there's a size mismatch, then the developers of the embedding application have somehow messed up their build process. The stable ABI is a different story, but that's why we try very hard to avoid exposing any structs in the stable ABI. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia