[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Nathaniel Smith]

On Tue, Nov 20, 2018 at 1:34 AM Petr Viktorin <encukou at gmail.com> wrote:
>
> On 11/19/18 12:14 PM, Victor Stinner wrote:
> > To design a new C API, I see 3 options:
> >
> > (1) add more functions to the existing Py_LIMITED_API
> > (2) "fork" the current public C API: remove functions and hide as much
> > implementation details as possible
> > (3) write a new C API from scratch, based on the current C API.
> > Something like #define newcapi_Object_GetItem PyObject_GetItem"?
> > Sorry, but "#undef <private_function>" doesn't work. Only very few
> > functions are defined using "#define ...".
> >
> > I dislike (1) because it's too far from what is currently used in
> > practice. Moreover, I failed to find anyone who can explain me how the
> > C API is used in the wild, which functions are important or not, what
> > is the C API, etc.
>
> One big, complex project that now uses the limited API is PySide. They
> do some workarounds, but the limited API works. Here's a writeup of the
> troubles they have with it:
> https://github.com/pyside/pyside2-setup/blob/5.11/sources/shiboken2/libshiboken/pep384impl_doc.rst

AFAIK the only two projects that use the limited API are
PySide-generated modules and cffi-generated modules. I guess if there
is some cleanup needed to remove stuff that snuck into the limited
API, then that will be fine as long as you make sure they aren't used
by either of those two projects.

For the regular C API, I guess the PyPy folks, and especially Matti
Picus, probably know more than anyone else about what parts are
actually used in the wild, since they've spent way more time digging
into real projects. (Do you want to know about the exact conditions in
which real projects rely on being able to skip calling PyType_Ready on
a statically allocated PyTypeObject? Matti knows...)

> I hope the new C API will be improvements (and clarifications) of the
> stable ABI, rather than a completely new thing.
> My ideal would be that Python 4.0 would keep the same API (with
> questionable things emulated & deprecated), but break *ABI*. The "new C
> API" would become that new stable ABI -- and this time it'd be something
> we'd really want to support, without reservations.

We already break ABI with every feature release – at least for the
main ABI. The limited ABI supposedly doesn't, but probably does, and
as noted above it has such limited use that it's probably still
possible to fix any stuff that's leaked out accidentally.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org