On Fri, 1 Apr 2022, 6:47 pm Victor Stinner, <vstinner@python.org> wrote:
On Wed, Mar 30, 2022 at 5:42 PM Guido van Rossum <guido@python.org> wrote:

I'm not convinced that advertising an API as being Unstable (in the
documentation?) is going to solve any kind of problem. People love to
use private APIs, and they do it simply because it's technically
possible :-) At the end of the day, we have to help them updating
their code, otherwise we (at least my Red Hat team) cannot update
Python.

I designed the internal C API to be more annoying to be used (need to
define a macro, need more specific #include) in the hope that people
will think twice before using it :-)


The changes you've made have been excellent, and the existing 3 categories (stable public ABI, stable public API, unstable internal API) cover the vast majority of cases.

The final case that isn't quite covered yet is to offer a "semi-stable" API category for use cases that are intrinsically coupled to implementation details that may change between feature releases, but should remain stable within a release series.

The concrete motivating example for the new category is the extra APIs you need in order to provide an alternative eval loop implementation.

The internal API category doesn't properly cover that case, as the APIs there are free to change even in maintenance releases, and setting Py_BUILD_CORE exposes a lot more than what an alternative eval loop would need.

Regular public functions may work in some cases, but aren't necessarily practical in others (such as exposing the internal frame details for use in alternative eval loops).

From an implementation PoV, my own suggestion would be to define a new API tier with an opt-in macro rather than relying solely on documentation or naming conventions.

For example, define "Py_SEMI_STABLE_API" to opt in, with the headers under "Include/cpython/semi_stable/" (I don't like "unstable" as potential terminology here, since the internal API is already unstable - we're splitting the difference between that and the long term stability of the full public API)

Cheers,
Nick.