On Mon., 30 Sep. 2019, 7:13 am Victor Stinner, <vstinner@python.org> wrote:

Hi Nick,

Le dim. 29 sept. 2019 à 08:47, Nick Coghlan <ncoghlan@gmail.com> a écrit :

...

I don't quite understand the purpose of this change, as there's no stable ABI for applications embedding CPython.

Well, I would like to prepare Python to provide a stable ABI for embedded Python. While it's not a design goal yet (Include/cpython/initconfig.h is currently excluded from Py_LIMITED_API), this change is a step towards that.

...

As a result, updating to a new X.Y.0 release always requires rebuilding the entire application, not just building and relinking CPython.

In Python 3.8, C extensions are no longer linked to libpython which allows to switch between a release build and a debug build of libpython.

Can we imagine the same idea for embedded Python? I checked vim on Linux: it's linked to libpython3.7m.so.1.0: a specific Python version, library built in release mode.

Switching between ABI compatible debug 3.8 and release 3.8 builds isn't the same as allowing switching between ABI incompatible release 3.8 and release 3.9 builds.

...

I could understand a change to require passing in an expected Python version so we can fail more gracefully on a bad link where an application that intended to embed Python 3.8 is incorrectly linked against Python 3.9 (for example), but performing that kind of check would require passing in PY_VERSION_HEX, not the size of the config struct.

It seems simpler to me to pass the structure size rather than the Python version. It avoids the risk of updating the structure without update the Python version. I also avoids to have to change the Python version immediately when PyConfig is modified.

We already change the Python version as soon as the maintenance branch gets created (master is 3.9.0a0, and has been pretty much since 3.8.0b1). The main risk of

sizeof(PyConfig) comes if we *remove* a field and add a new field of the same size: the structure size doesn't change... But in my experience, we only add new ways to configure Pyhon, we never remove old ones :-D

The main risk I see is some *other* unversioned struct in the full C ABI changing size. If the config APIs are only checking for config struct size changes, then changes to anything else will still segfault. If they're instead checking "What version was the calling application built against?" then we can decide how to handle it on the interpreter side (e.g. require that the "X.Y" part of the version match, and report an init error otherwise).

The question is if it's convenient to compute sizeof(PyConfig) in programming languages other than C. Providing a "structure version" or the structure size from a function call would not work. The value must be known a compilation time, not at runtime. The purpose is to compare the version/size between build and runtime (they must match).

You can also compare the build time value of a public integer macro to detect build discrepancies. "sizeof(some_struct)" is just a straightforward way to define such an integer in a way that will always change when a new field is added to a particular struct.

In the first implementation of my PEP, I used an internal "config version" provides by a macro. But it was said that macros are not convenient.

The initialisation macros aren't necessarily convenient (e.g. if you're using a C++ struct rather than a C one). That's an issue with requiring C-style initialisation, though, and less with macros in general. That said, I think the change to make the expected API/ABI version an explicit part of the config API rather than a hidden part of the struct is a good idea, since it lets us replace cryptic segfaults with explicit "Python version mismatch" errors.

PY_VERSION_HEX is provided as a macro, but we are now trying to avoid macros in our C API, no? At least, it's what I understood from the PEP 587 discussion.

The issue with macros is that their behaviour gets locked in at build time, so you can't fix bugs in them or otherwise change their behaviour just by linking against a new version. Instead, you have to recompile the consumer application or module in addition to recompiling the API provider. In this case though, that's exactly what we want, as the whole point would be to detect cases where the CPython runtime library had been recompiled, but the embedding application hadn't (or vice-versa).

...

We don't support that - all our APIs that accept PyObject/PyTypeObject/etc require that the caller pass in structs of the correct size for the version of Python being used.

For PyConfig, it's allocated (on the stack on or on heap) by the application. So the application requires to access to the full structure.

Objects (instances) are allocated by Python (on the heap). Applications usually don't need to know/access the structure.

Python is far from being perfect, static types are still supported and they are an issue for the stable ABI.

The config APIs aren't covered by the stable ABI. If they (or a variant on them) were to be added to it some day though, then Py_VERSION_HEX would still work as a marker to select a specific config struct size, we'd just need to make the processing of any new fields conditional at runtime on the passed in "expected Python version". The PEP 587 versions of the config APIs would still error out on a feature release version mismatch.

...

The PyConfig and PyPreConfig structs are no different from PyObject in that regard: if there's a size mismatch, then the developers of the embedding application have somehow messed up their build process.

In short, PyConfig initialization works like that:

* The application allocates a PyConfig object on the stack * Python calls memset(config, 0, sizeof(PyConfig))

If there is a size mismatch, Python triggers a buffer overflow which is likely to cause issues.

Aye, that's why I agree adding some form of explicit "expected version" check is a good idea.

I prefer to have a clean API which makes buffer overflow impossible.

We'll never change the size of the config structs (or any other public struct) in a maintenance branch, so an "expected Python version" check would serve this purpose just as well as passing in the size of the config structs.

Embedding Python and handling different Python versions is not trivial, especially if we start to add new fields to each PyConfig (which is very likely). If prefer to be extra careful.

As noted above, despite what I wrote on BPO, you no longer need to persuade me that the version check is desirable, only that a narrow check on specific struct sizes is preferable to a broad check on the expected API version. Consider the difference in error messages: "Application expected CPython 3.8, but is attempting to load CPython 3.9" vs "Application provided a 256 byte config struct, CPython expected 264 bytes" (e.g. when a new pointer was added) It wouldn't make sense to try to continue in either case due to the potential for other ABI incompatibilities, but the first offers a much clearer hint to the affected developer as to what is going on. If we ever do create a stable config ABI, then inside the code we can create a range lookup table from different Python versions to different versions of the config struct, whereas if the caller is only passing in the expected size of the config struct, we can't infer an expected Python version from that.

I also expect bad surprises even in CPython with Programs/_testembed: many tests use PyConfig. Depending if _testembed is properly rebuilt or not, bad thing will happen.

That's no worse than bad things happening after changing the compiler, or the ABI versioning scheme. Worst case we work around it with a clean rebuild, best case we figure out why the incremental build didn't do the right thing and fix it.

--

To implement my PEP 445 "Add new APIs to customize Python memory allocators", I added a PyMemAllocator structure which is part of the API.

Quickly, I had to add a new field to PyMemAllocator. But it wasn't possible to detect if a C extension used the old or the new structure... So I decided to rename the structure to PyMemAllocatorEx to ensure that the compilation of all C extensions using the API will fail... :-(

I really dislike this solution. What will happen when we will add another field to the structure, like a new PyMem_Aligned() (similar to posix_memalign()) function? PyMem_Aligned() can be implementation on top of an existing memory allocator which doesn't support it natively. But the problem is again the API and the PyMemAllocatorEx structure...

Passing PY_VERSION_HEX to the pre-init config APIs would address that for all structs in the public API, not just those that are included directly in the config structs. Cheers, Nick.

Victor -- Night gathers, and now my watch begins. It shall not end until my death.

On Mon, 30 Sep 2019 at 09:37, Victor Stinner <vstinner@python.org> wrote:

Le lun. 30 sept. 2019 à 00:33, Nick Coghlan <ncoghlan@gmail.com> a écrit :

...

As noted above, despite what I wrote on BPO, you no longer need to persuade me that the version check is desirable, only that a narrow check on specific struct sizes is preferable to a broad check on the expected API version.

I understand that your main motivation to use the Python version number rather than sizeof(PyConfig) is the error message.

No, my main motivation is to create an API that can emit a useful error message on *ALL* version conflicts between an embedding application and the embedded runtime, not just version conflicts involving versions that change the size of the config structs. The latter option is a good one if all we want to version is the struct itself, but I don't think that's what we really want here: I think we want to version check the entire API/ABI. The improved error message is just a nice bonus, as is the fact that I think the following code is more self-explanatory than the size checks: PyConfig config; config.header_version = PY_VERSION_HEX; PyStatus status = PyConfig_InitPythonConfig(&config); The documentation would say something like "header_version should specify the version of the CPython headers that was used to build and link the embedding application. The simplest way to set it correctly is to set it to ``PY_VERSION_HEX``".

If we implement support for older PyConfig ("stable ABI"), you will simply never see this error: it will just work transparently.

But that's true regardless of whether the consistency check is based on the struct size or the header version.

IMHO the current error message is good enough:

if (config->struct_size != sizeof(PyConfig)) { return _PyStatus_ERR("unsupported PyConfig structure size " "(Python version mismatch?)"); }

That error message won't trigger if the public API struct that changed size is PyTypeObject or PyObject (or any of the other structs in the full API/ABI), whereas a PY_VERSION_HEX based check can be made to fail based on any policy we choose to enforce. For example, we could make the runtime enforce the following pair of rules: * if both the header version and runtime version are final releases or release candidates, then only the major and minor version fields need to match (the X.Y in X.Y.Z) * if either the header version or the runtime version is an alpha or beta release (or an unknown release type), then the entire version number must match exactly Or we could be more permissive, and only ever enforce the first rule, even for alpha and beta releases.

I wrote a proof-of-concept to check if it would be doable to support multiple versions (sizes) of PyConfig: it's doable and it's quite easy to implement, a few lines of code. For example, support Python 3.8 PyConfig in Python 3.9.

That doesn't change all that much with a Py_VERSION_HEX based check - it just needs an extra table for range based lookups to find the relevant struct size for the given version, together with keeping definitions of the old config structs around. typedef struct { uint32_t firstVersion; Py_ssize_t structSize; } _Py_versionToSize; static _Py_versionToSize _PyConfigStructSizes { {0x03090000, sizeof(_PyConfig_3_9)}, {0, sizeof(_PyConfig_3_8)} } If the given version matched, you wouldn't need to do the size lookup. You also wouldn't be constrained to having the lookup table be from versions to struct sizes - it could instead be from versions to function pointers, or some other such thing, based on whatever made the most sense given how the rest of the config system worked in any given release. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Mon, 30 Sep 2019 at 23:50, Victor Stinner <vstinner@python.org> wrote:

Le lun. 30 sept. 2019 à 13:45, Nick Coghlan <ncoghlan@gmail.com> a écrit :

...

...

I understand that your main motivation to use the Python version number rather than sizeof(PyConfig) is the error message.

No, my main motivation is to create an API that can emit a useful error message on *ALL* version conflicts between an embedding application and the embedded runtime, not just version conflicts involving versions that change the size of the config structs.

The latter option is a good one if all we want to version is the struct itself, but I don't think that's what we really want here: I think we want to version check the entire API/ABI.

IMHO this discussion is going too far away from the PEP 587 goals. The PEP 587 is unrelated to the stable API or the stable ABI.

I would be entirely happy with reverting to PEP 587 as originally accepted (no public struct_size field, no hidden struct versioning field), rather than making last minute changes to it right before the first release candidate. PyConfig and PyPreConfig aren't covered by the stable ABI, so there's no current need for them to be able to evolve gracefully across ABI version bumps. If we *do* decide to evolve them in a direction that allows for more explicit ABI version checks, then doing that as a separate PyVersionedConfig and PyVersionedPreConfig API would have the virtue of keeping the simple case simple, while offering a slightly more awkward to use, but also more forward compatible API for the benefits of folks that wanted to use it. However, *if* we were to add a versioning scheme for 3.8.0rc1, then https://github.com/python/cpython/pull/16496/files covers what I think it should look like (a header_version field that gets set to PY_VERSION_HEX by the embedding application). The only outcome I'd consider undesirable is shipping a public API that's more awkward to use than it needs to be, doesn't conform to the accepted version of the PEP, and doesn't protect against most of the potential sources of segfaults arising from an ABI mismatch (and could even cause them if the supplied "struct_size" value were to be trusted directly, rather than being treated as a lookup value into a set of known-valid struct sizes. The currently checked in code at least doesn't do that, but that's only because it ignores the new struct_size field entirely when making the memset() calls to initialise the config structs).

I suggest to develop a new use case and a new solution to handle this case. The solution can be *very different* than "compare two versions at startup".

For example, we may need a CI to ensure that we don't break the stable ABI: https://pythoncapi.readthedocs.io/stable_abi.html#check-for-abi-changes

Another solution is to write a completely new C API from scratch. For example, HPy ("PyHandle") project proposes to write a new API implemented on top of the existing API: https://github.com/pyhandle/hpy

This project is even developed outside CPython. So the solution can be found *outside* CPython as well.

These are far outside the bounds of "Don't segfault when an embedding application tries to do something that's obviously broken", which I consider the only reason to even consider changing the public API away from what was accepted in PEP 587 in the 3.8rc1 time frame. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 2019-09-30 00:33, Nick Coghlan wrote: [...]

We'll never change the size of the config structs (or any other public struct) in a maintenance branch

But we *will* change it in alphas/betas, and then we will ask people to try their software out with these. Of course, there are no actual API/ABI guarantees with alphas/betas: officially, you need to rebuild everything for each new pre-release. But with the goal being to find bugs rather than provide correct software, it's OK to take shortcuts (if you can then distinguish the bugs the shortcuts cause from actual bugs in the code). Issuing proper version mismatch errors (instead of random cryptic segfaults) could make for a much nicer experience when testing several successive alphas/betas.

On 9/29/2019 5:13 PM, Victor Stinner wrote:

It seems simpler to me to pass the structure size rather than the Python version. It avoids the risk of updating the structure without update the Python version. I also avoids to have to change the Python version immediately when PyConfig is modified.

In Win32, Microsoft does this a lot. For example, https://docs.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndcla...

The main risk of sizeof(PyConfig) comes if we *remove* a field and add a new field of the same size: the structure size doesn't change... But in my experience, we only add new ways to configure Pyhon, we never remove old ones :-D

I agree this is unlikely. Eric

On Mon, Sep 30, 2019 at 11:00 PM Brett Cannon <brett@python.org> wrote:

Victor Stinner wrote:

...

Hi Nick, Le dim. 29 sept. 2019 à 08:47, Nick Coghlan ncoghlan@gmail.com a écrit :

...

I don't quite understand the purpose of this change, as there's no stable ABI for applications embedding CPython. Well, I would like to prepare Python to provide a stable ABI for embedded Python. While it's not a design goal yet (Include/cpython/initconfig.h is currently excluded from Py_LIMITED_API), this change is a step towards that.

So then isn't this a very last-minute, premature optimization if it isn't a design goal yet? If that's the case then I would vote not to make the change and wait until there's been feedback on 3.8 and then look at stabilizing the embedding API such that it can be considered stable.

I just want to chime in here and confirm, as a Professional CPython Embedder(tm) that embedders cannot currently rely on a stable ABI, or the limited API, and often not even the public API. It's not just about exposed symbols and struct sizes, it's often also about the semantics of internals (e.g. how importlib handles custom module finders and loaders) that subtly changes. For anything but the simplest PyRun_SimpleString-based embedding this is more the case when embedding than extending, and even than a regular (complex) Python program. (I already showed Victor and a few others some of the hoops we have to jump through at Google to embed CPython correctly, and only half of those things are necessary just because of Google's environment.)

-Brett

...

...

As a result, updating to a new X.Y.0 release always requires rebuilding the entire application, not just building and relinking CPython. In Python 3.8, C extensions are no longer linked to libpython which allows to switch between a release build and a debug build of libpython. Can we imagine the same idea for embedded Python? I checked vim on Linux: it's linked to libpython3.7m.so.1.0: a specific Python version, library built in release mode. I could understand a change to require passing in an expected Python version so we can fail more gracefully on a bad link where an application that intended to embed Python 3.8 is incorrectly linked against Python 3.9 (for example), but performing that kind of check would require passing in PY_VERSION_HEX, not the size of the config struct. It seems simpler to me to pass the structure size rather than the Python version. It avoids the risk of updating the structure without update the Python version. I also avoids to have to change the Python version immediately when PyConfig is modified. The main risk of sizeof(PyConfig) comes if we remove a field and add a new field of the same size: the structure size doesn't change... But in my experience, we only add new ways to configure Pyhon, we never remove old ones :-D The question is if it's convenient to compute sizeof(PyConfig) in programming languages other than C. Providing a "structure version" or the structure size from a function call would not work. The value must be known a compilation time, not at runtime. The purpose is to compare the version/size between build and runtime (they must match). In the first implementation of my PEP, I used an internal "config version" provides by a macro. But it was said that macros are not convenient. PY_VERSION_HEX is provided as a macro, but we are now trying to avoid macros in our C API, no? At least, it's what I understood from the PEP 587 discussion. We don't support that - all our APIs that accept PyObject/PyTypeObject/etc require that the caller pass in structs of the correct size for the version of Python being used. For PyConfig, it's allocated (on the stack on or on heap) by the application. So the application requires to access to the full structure. Objects (instances) are allocated by Python (on the heap). Applications usually don't need to know/access the structure. Python is far from being perfect, static types are still supported and they are an issue for the stable ABI. The PyConfig and PyPreConfig structs are no different from PyObject in that regard: if there's a size mismatch, then the developers of the embedding application have somehow messed up their build process. In short, PyConfig initialization works like that:

The application allocates a PyConfig object on the stack Python calls memset(config, 0, sizeof(PyConfig))

If there is a size mismatch, Python triggers a buffer overflow which is likely to cause issues. I prefer to have a clean API which makes buffer overflow impossible. Embedding Python and handling different Python versions is not trivial, especially if we start to add new fields to each PyConfig (which is very likely). If prefer to be extra careful. I also expect bad surprises even in CPython with Programs/_testembed: many tests use PyConfig. Depending if _testembed is properly rebuilt or not, bad thing will happen. -- To implement my PEP 445 "Add new APIs to customize Python memory allocators", I added a PyMemAllocator structure which is part of the API. Quickly, I had to add a new field to PyMemAllocator. But it wasn't possible to detect if a C extension used the old or the new structure... So I decided to rename the structure to PyMemAllocatorEx to ensure that the compilation of all C extensions using the API will fail... :-( I really dislike this solution. What will happen when we will add another field to the structure, like a new PyMem_Aligned() (similar to posix_memalign()) function? PyMem_Aligned() can be implementation on top of an existing memory allocator which doesn't support it natively. But the problem is again the API and the PyMemAllocatorEx structure... Victor Night gathers, and now my watch begins. It shall not end until my death.

---

Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4ONRL3O6...

-- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!

I agree that passing a struct size as struct member values sounds a bit unidiomatic. Also, it doesn't achieve ABI stability, just allows erroring out in case the user links with the wrong Python version. Regards Antoine. On Sun, 29 Sep 2019 16:47:41 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:

On Sat, 28 Sep 2019 at 12:56, Victor Stinner <vstinner@python.org> wrote:

...

Hi,

I dislike having to do that, but I had to make a last minute change in my PEP 587 "Python Initialization Configuration" to allow to modify the structure in the future without breaking the backward compatibility. I added a "struct_size" field to PyPreConfig and PyConfig:

* https://bugs.python.org/issue38304 * https://github.com/python/peps/commit/afa38c0bef7738b8fcc3173daee8488e042083...

The example:

PyConfig config; PyStatus status = PyConfig_InitPythonConfig(&config); ...

must now be written:

PyConfig config; config.struct_size = sizeof(PyConfig); PyStatus status = PyConfig_InitPythonConfig(&config); ...

At the beginning, I used a private "_config_version" field which was initialized statically by a macro. But it was decided to replace macros with functions. I only noticed today that the conversion to function broke the API/ABI future compatibility.

PyConfig_InitPythonConfig() got uninitialized memory and didn't know the size of the config variable.

I don't quite understand the purpose of this change, as there's no stable ABI for applications embedding CPython. As a result, updating to a new X.Y.0 release always requires rebuilding the entire application, not just building and relinking CPython.

I could understand a change to require passing in an expected Python version so we can fail more gracefully on a bad link where an application that intended to embed Python 3.8 is incorrectly linked against Python 3.9 (for example), but performing that kind of check would require passing in PY_VERSION_HEX, not the size of the config struct.

...

With my change, the function now requires the "struct_size" field to be set, and so it can support internally different versions of the PyConfig structure.

Storing the structure size directly in the structure is a common practice in the Windows API which is a good example of long term ABI compatibility.

This analogy doesn't hold, as Microsoft explicitly support running old binaries on new versions of Windows, and hence need a way to determine which version of the structure is being passed in.

We don't support that - all our APIs that accept PyObject/PyTypeObject/etc require that the caller pass in structs of the correct size for the version of Python being used. The PyConfig and PyPreConfig structs are no different from PyObject in that regard: if there's a size mismatch, then the developers of the embedding application have somehow messed up their build process.

The stable ABI is a different story, but that's why we try very hard to avoid exposing any structs in the stable ABI.

Cheers, Nick.

On Mon, 30 Sep 2019 22:43:40 +0200 Victor Stinner <vstinner@python.org> wrote:

Hi back,

It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not really liked".

I don't see any traction to provide a stable ABI for embedded Python.

Not at the last minute in a rc1, I'd say :-) I think if you wanted to make the PyConfig apt at providing ABI-stability, you should have designed it differently. `PyType_FromSpec` provides a useful model (pass an arbitrary-sized static array of field initializers). Regards Antoine.

On Tue., 1 Oct. 2019, 6:47 am Victor Stinner, <vstinner@python.org> wrote:

Hi back,

It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not really liked".

I don't see any traction to provide a stable ABI for embedded Python. The consensus is more towards: "it doesn't work and we don't want to bother with false promises" (nor add code for that).

Since Lukasz is supposed to tag 3.8.0rc1 today, I propose to remove anything related to "stable API" or "stable ABI" from the PEP 587 right now. I wrote two PRs to remove struct_size:

* CPython: https://github.com/python/cpython/pull/16500 * PEP 587: https://github.com/python/peps/pull/1185

Later, if we decide to start proving a stable ABI for embedded Python, we can still add a "version" or "struct_size" field to PyConfig later (for example in Python 3.9).

Thanks Victor, I think this is the right way for us to go, given the relative timing in the release cycle. The idea of some day expanding the stable ABI to cover embedding applications, not just extension modules, is an intriguing one, but I'm genuinely unsure anyone would ever actually use it. Cheers, Nick.

Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2JAJQA5O...

On 30Sep2019 1625, Nick Coghlan wrote:

After merging your PR and closing mine, I had an idea for Python 3.9: what if we offered a separate public "int Py_CheckVersionCompatibility(uint64_t header_version)" call? (64 bit input rather than 32 to allow for possible future changes to the version number formatting scheme)

The basic variant of that API would be what I had in my PR: release candidates and final releases allow an inexact match, other releases require the hex version to match exactly.

As I posted on the issue tracker, I don't think this kind of version verification is necessary unless we're genuinely going to get into the business of adding built-in shims for API changes. Every supported platform is going to make you link against a dynamic library with the version number in its name. (Every approach that avoids this is not supported or does not support embedding.) So you get run-time checking for free, and you can add a compile-time check in three or four different ways depending on what makes the most sense for your build system. Let's not complicate the embedding API further with unnecessary new APIs. I'm keen to design something broadly useful and simpler than what we have now, and this is not going to help with it. Cheers, Steve

This is too-little, too-late (I was offline this past week), but for those who enjoy digging through historical archives, the Tcl folks had an interesting stubs mechanism that was *supposed* to solve the versioning issue (although I suspect it hasn’t actually done much in that regard) in addition to the “cross-platform dynamic linking is a royal pain” (circa 1999) issue: https://wiki.tcl-lang.org/page/Stubs Basically, you just use a vtable of function pointers exported by version, e.g. Tcl_8_0_Stubs, Tcl_8_1_Stubs, etc. The clients call each function via a layer of indirection, Tcl_8_0_Stubs->Tcl_Tell(), Tcl_8_1_Stubs->Tcl_Tell(), etc.; this is hidden by C macros. (From what I recall of COM, this is a rudimentary implementation of a COM interface.) I’m not sure I would recommend this today -- we were solving different problems back then (CAD workstations from various Unix vendors patched Frankenstein-esque library mismatches), and compiler optimizations weren’t as sophisticated. It had the nice advantage of crashing immediately due to a version mismatch rather than silently corrupting data structures that had undergone changes, etc. Back then, GNU Binutils symbol versioning was just adding to our headaches (again, telling our customers to fix their workstations wasn’t a plausible solution, and it was only on Linux, which was then a very small part of the installed base).

On Oct 1, 2019, at 1:45 AM, Victor Stinner <vstinner@python.org> wrote:

Sorry for the bad timing. I dislike working under pressure. The discussion on python-dev, the bug tracker and pull requests was really interesting.

The issue has been fixed: the whole idea of stable ABI for PyConfig has been abandoned. If you want to switch to a different version of Python when you embed Python, you must recompile your code. It was always like that and the PEP 587 doesn't change anything.

PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() initialize PyConfig to sane default configuration. So if we add new fields in Python 3.9, you don't have to manually initialize these fields, except if you explicitly want to use these new fields to benefit of new configuration options.

Victor

Le mar. 1 oct. 2019 à 00:38, Nick Coghlan <ncoghlan@gmail.com> a écrit :

...

On Tue., 1 Oct. 2019, 6:47 am Victor Stinner, <vstinner@python.org> wrote:

...

Hi back,

It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not really liked".

I don't see any traction to provide a stable ABI for embedded Python. The consensus is more towards: "it doesn't work and we don't want to bother with false promises" (nor add code for that).

Since Lukasz is supposed to tag 3.8.0rc1 today, I propose to remove anything related to "stable API" or "stable ABI" from the PEP 587 right now. I wrote two PRs to remove struct_size:

* CPython: https://github.com/python/cpython/pull/16500 * PEP 587: https://github.com/python/peps/pull/1185

Later, if we decide to start proving a stable ABI for embedded Python, we can still add a "version" or "struct_size" field to PyConfig later (for example in Python 3.9).

Thanks Victor, I think this is the right way for us to go, given the relative timing in the release cycle.

The idea of some day expanding the stable ABI to cover embedding applications, not just extension modules, is an intriguing one, but I'm genuinely unsure anyone would ever actually use it.

Cheers, Nick.

...

Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2JAJQA5O...

-- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/36RTEPNE...