Py_LIMITED_API vs. the Stable ABI
In PEP 384 [1] it says:
"This PEP proposes to define a stable set of API functions whichare guaranteed to be available for the lifetime of Python 3, and which will also remain binary-compatible across versions."
Thus there are two parts that the PEP ties together closely: the limited API (AKA Py_LIMITED_API) and the stable ABI. For the most part, the API stability goal was driven by ABI stability.
Lately the binary compatibility part has seemed less important (as implied by sprint/list discussions), while the limited API has begun to stand on its own conceptually. On top of that, the limited API has started to take on a second, distinct-but-similar role: "an approximation of the minimal future API" or "the basic API that most extensions should stick to". In that case strict compatibility is desirable but not required (like it is with the stable ABI) or quite so important.
Do we want a stand-alone public API that is focused on a minimal set and extra resilient to future changes (but with less strict compatibility guarantees)? If so then we should be deliberate about it:
- introduce a different macro (Py_BASIC_API? Py_MINIMAL_API?)
- assess which parts of the limited API to keep in this new API (and what to add from the full API)
- decide if there are any future-looking gaps we could fill (e.g. with macros)
- document the inclusions explicitly
- make (relative) compatibility guarantees clear
This will provide what folks are after (mostly) without incurring the maintenance cost of binary compatibility. It seems like there's a lot of overlap here with PEP 652, though I'm suggesting a distinct API.
Regardless, I think Py_LIMITED_API should preserve its strong association with the stable ABI.
Finally, is such a "minimal" API worth it? Is the current full "public" API sufficient (and projects like HPy can be the less-constrained, future-oriented adaptations)? Would a minimal API mean we'd be more free to make breaking changes in the rest of the public API?
-eric
On Wed, Mar 17, 2021 at 7:33 PM Eric Snow <ericsnowcurrently@gmail.com> wrote:
Finally, is such a "minimal" API worth it? Is the current full "public" API sufficient (and projects like HPy can be the less-constrained, future-oriented adaptations)?
HPy aims to make the details of the implementation underlying the HPy API more opaque (especially hiding the memory layout of PyObject by replacing it with HPy handles, hiding the details of object lifetimes by removing borrowed and stolen references from the API and by not sharing handles).
Since that's already a big project our other goals are things like "make porting to HPy as easy as we can" (one can port the methods of a type one method at a time, for example).
We are trying to drop old ways of doing things where we can, and trimming down the API where possible, but we're also making a conscious effort not to try to fix everything.
Projects like the existing limited API, or a minimal API are potentially useful in providing a smaller target that contains only the "core" or "most needed" or "not changing too much" parts of the API, but in the end I imagine practicality will win out and HPy with grow whatever APIs it needs to make porting existing extensions work.
Hopefully that was some helpful HPy background.
Yours sincerely, Simon Cross
On Thu, 18 Mar 2021 at 03:33, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Do we want a stand-alone public API that is focused on a minimal set and extra resilient to future changes (but with less strict compatibility guarantees)?
I don't know who the audience for such an API would be.
The stable ABI has a very specific purpose: allowing extension modules to be built once and then used across multiple major versions of Python. The limited API then exists to define what extension modules can do while still building against the stable ABI. There's a concrete benefit to extension module authors in porting: port once, save build and deployment complexity forever.
For porting to multiple Python implementations, an independently updatable abstraction layer like HPy seems like a far more useful approach than a minimal API that only CPython provides. I could see *HPy* some day saying "provide this minimal C API and HPy will work with your Python implementation"
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Eric Snow -
Nick Coghlan -
Simon Cross