[Python-ideas] PEP: Hide implementation details in the C API

[Nick Coghlan]

On 14 July 2017 at 02:29, Brett Cannon <brett at python.org> wrote:
> On Thu, 13 Jul 2017 at 09:12 Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> I don’t understand. Moving too functions instead of macros for some thing
>> doesn’t really help with keeping the public API stable (for the non-stable
>> ABI).
>
>
> Sorry, I didn't specify which ABI/API I was talking about; my point was from
> the stable ABI.
>
> I think this is quickly showing how naming is going to play into this since
> e.g. we say "stable ABI" but call it "Py_LIMITED_API" in the code which is
> rather confusing.

I honestly think we should just change that symbol to Py_STABLE_ABI
(with Py_LIMITED_API retained as a backwards compatibility feature).
Yes, Py_LIMITED_API is technically more correct, but it's confusing in
practice, while "Py_STABLE_ABI" matches the user's intent: "make sure
my binaries only depend on the stable ABI".

> Just to make sure I'm not missing anything, it seems we have a few levels
> here:
>
> 1. The stable A**B**I which is compatible across versions
> 2. A stable A**P**I which hides enough details that if we change a struct
> your code won't require an update, just a recompile

I don't think we want to promise that the portable API will be
completely backwards compatible over time - unlike the stable ABI, it
should be subject to Python's normal deprecation policy (i.e. if an
API emits a deprecation warning in X.Y, we may remove it entirely in
X.Y+1).

Instead, I think the key promises of the portable API should be:

1. It only exposes interfaces that are genuinely portable across at
least CPython and PyPy
2. It adheres as closely to the stable ABI as it can, with additions
made *solely* to support the building of existing popular extension
modules (e.g. by adding back static type declaration support)

> 3. An API that exposes CPython-specific details such as structs and other
> details that might not be entirely portable to e.g. PyPy easily but that we
> try not to break
> 4. An internal API that we use for implementing the interpreter but don't
> expect anyone else to use, so we can break it between feature releases
> (although if e.g. Cython chooses to use it they can)
>
> (There's also an API local to a single file, but since that is never
> exported to the linker it doesn't come into play here.)
>
> So, a portable API/ABI, a stable API, a CPython API, and then an
> internal/core/interpreter API. Correct?

Not quite:

- stable ABI (strict extension module compatibility policy)
- portable API (no ABI stability guarantees, normal deprecation policy)
- public CPython API (no cross-implementation portability guarantees)
- internal-only CPython core API (arbitrary changes, no deprecation warnings)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia