[Python-Dev] Making PyInterpreterState an opaque type

[Victor Stinner]

Le lun. 18 févr. 2019 à 22:34, Jeroen Demeyer <J.Demeyer at ugent.be> a écrit :
> First of all, if everybody can actually #define Py_BUILD_CORE and get
> access to the complete API, I don't mind so much. But then it's
> important that this actually keeps working (i.e. that those headers will
> always be installed).
>
> Still, do we really need so many levels of API:
> (1) stable API (with #define Py_LIMITED_API)
> (2) public documented API
> (3) private undocumented API (the default exposed API)
> (4) internal API (with #define Py_BUILD_CORE)

It's not a matter of documentation. It's a matter of warranty provided to users.

I would like to move towards (1) by default: only provide a stable API
by default. IMHO most users will be just fine with this subset of the
API.

The borders between (2), (3) and (4) are unclear. I created
Include/cpython/ which is not really a "private API" but more "CPython
implementation details". A better definition of (1) would be "portable
stable API" whereas (2)+(3) would be "CPython stable API".

And (4) would be the unstable API.

Summary:

(1) Portable stable API
(2) Portable CPython API
(3) Unstable API

... The border between (2) and (3) is a "work-in-progress".

I modified "make install" to install (3) as well: they are users of
this API. Performance can be a good motivation: Cython for example.
Debuggers and profiles really need to access to the lowest level of
API usually because they can only *inspect* (structure fileds) but no
*execute* code (call functions).

> I'm afraid of hiding actually useful private macros under Py_BUILD_CORE.

Again, it's not a matter of usefulness. It's a matter of backward
compatibility announced to users.

I would like to make it more explicit that if you *opt in* for an
unstable API, you are on your own.

My intent is that in 5 years or 10 years, slowly, most C extensions
will use (1) which will allow Python to experiment new optimizations,
and should help PyPy (cpyext module) to become even more efficient.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.