[python-committers] Redoing the C API? (was: Call For Participants For The 2016 Python Language Summit)

Brett Cannon brett at python.org
Thu Mar 3 12:39:17 EST 2016

On Thu, 3 Mar 2016 at 06:31 Antoine Pitrou <antoine at python.org> wrote:

> Le 03/03/2016 13:40, Nick Coghlan a écrit :
> >>
> >> I would be nice to discuss how to move to a new C API which doesn't
> >> expose implementation details and discuss if libraries will move to it
> >> or not. Implementation "details": GIL, reference counting, C
> >> structures like PyObject, etc.
> >
> > Adding cffi (including its dependencies) to the standard library was
> > approved-in-principle a couple of years ago, and I believe the one
> > technical issue with a lack of support for ahead-of-time compilation
> > of the extension module has since been addressed, so as far as I know
> > that just needs a champion to actually work through the details of
> > getting it added via the PEP process.
> >
> > I'm also not aware of any explicit documentation of the underlying FFI
> > from a C API/ABI perspective, which is what would be needed for tools
> > like SWIG and Cython to support it as an alternative to the full
> > CPython API.
> I don't understand what cffi has to do with the CPython API.  You use
> cffi for binding with third-party libraries.  C code wanting to
> interface with CPython will continue to have to use the CPython API.

You're right, it doesn't directly address the needs of integrators like
Blender who embed CPython as a scripting language. And this doesn't
directly address the needs of Numba where you want low-level access to (I
assume) the code object.

But I do think the spirit of Victor's idea is worth considering. Our C API
does expose *a lot* of low-level detail that isn't necessarily needed and
can hamper our ability to tweak how Python itself operates. We also have
not been good about deprecating parts of the C API and thus getting rid of
old ways of doing things that no longer map well on to how Python operates
now (the [import APIs]( https://docs.python.org/3/c-api/import.html) are an
example of this; I mean how many ways do we really need to expose for
importing Python code?).

So it might behoove us to stop and look at what exactly is needed by
embedders like Blender, library wrappers like NumPy or the various DB
drivers, and accelerators like Numba. That way we can maybe start
discussing things like all external code must go through a function/macro,
PyObject is to be considered opaque to third-parties, and all public APIs
have an error return value. And we discuss how to handle deprecations. And
we see if there's a way to do memory management where INCREF/DECREF is no
longer the direct responsibility of C API users. I mean if you want a
motivation/lens to look at this through, then what would we need to do to
our C API to make it so that anyone following a new API wouldn't be broken
if we dropped the GIL?

> As for integrating cffi into the stdlib, it seems to be doing fine as a
> third-party library.

It is doing fine as an external library, but if something as radical as
heavily trimming back and/or rewriting the C API occurs then having CFFI in
the stdlib to evolve with the internal C changes makes sense. I think
that's where the thought of pulling CFFI in the stdlib stems from.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-committers/attachments/20160303/d1ca0035/attachment.html>

More information about the python-committers mailing list