[Python-Dev] Making PyInterpreterState an opaque type

Victor Stinner vstinner at redhat.com
Tue Feb 19 07:08:18 EST 2019

Le lun. 18 févr. 2019 à 22:34, Jeroen Demeyer <J.Demeyer at ugent.be> a écrit :
> First of all, if everybody can actually #define Py_BUILD_CORE and get
> access to the complete API, I don't mind so much. But then it's
> important that this actually keeps working (i.e. that those headers will
> always be installed).
> Still, do we really need so many levels of API:
> (1) stable API (with #define Py_LIMITED_API)
> (2) public documented API
> (3) private undocumented API (the default exposed API)
> (4) internal API (with #define Py_BUILD_CORE)

It's not a matter of documentation. It's a matter of warranty provided to users.

I would like to move towards (1) by default: only provide a stable API
by default. IMHO most users will be just fine with this subset of the

The borders between (2), (3) and (4) are unclear. I created
Include/cpython/ which is not really a "private API" but more "CPython
implementation details". A better definition of (1) would be "portable
stable API" whereas (2)+(3) would be "CPython stable API".

And (4) would be the unstable API.


(1) Portable stable API
(2) Portable CPython API
(3) Unstable API

... The border between (2) and (3) is a "work-in-progress".

I modified "make install" to install (3) as well: they are users of
this API. Performance can be a good motivation: Cython for example.
Debuggers and profiles really need to access to the lowest level of
API usually because they can only *inspect* (structure fileds) but no
*execute* code (call functions).

> I'm afraid of hiding actually useful private macros under Py_BUILD_CORE.

Again, it's not a matter of usefulness. It's a matter of backward
compatibility announced to users.

I would like to make it more explicit that if you *opt in* for an
unstable API, you are on your own.

My intent is that in 5 years or 10 years, slowly, most C extensions
will use (1) which will allow Python to experiment new optimizations,
and should help PyPy (cpyext module) to become even more efficient.

Night gathers, and now my watch begins. It shall not end until my death.

More information about the Python-Dev mailing list