[pypy-dev] PyPy C API

[Stefan Behnel]

Armin Rigo, 14.07.2012 10:35:
> On Tue, Jul 10, 2012 at 7:57 PM, Amaury Forgeot d'Arc wrote:
>> Do you have an idea what this API would look like?
>> Then I can help with the implementation :)
> 
> If we want to go down this path without caring for compatibility with
> CPython's C API, but instead focusing on what gives the best
> performance, then I would think about something like this:
> 
> pypyobj pypy_wraplong(long value);
> pypyobj pypy_add(pypyobj x, pypyobj y);
> void pypy_close(pypyobj x);
> pypyobj pypy_dup(pypyobj x);
> 
> using handles of type "pypyobj", which are basically opaque stuff (or
> even just integers, with "-1" meaning "exception").  Instead of the
> refcounting approach of CPython, it would be similar to file
> descriptors: a file descriptor refers to a file, but most files don't
> have any open file descriptor, and some files may have more than one.
> Any "object descriptor" must be closed with pypy_close().  pypy_dup()
> just duplicates the object descriptor, so that both descriptors refer
> to the same object but will be pypy_close()d independently.
> 
> This can be implemented efficiently: the C->PyPy direction is just
> doing one array lookup (this minimal indirection is hard to avoid
> anyway with a moving GC); and the PyPy->C direction (like the return
> value from pypy_add) just creates a new object descriptor anyway,
> without needing to look if one already exists.

Assuming that the PyPy->C path takes its object descriptors from a
free-list (because we'll need a lot of them, and usually just for a very
short time), I think this would be acceptable.


> (1) Do I make some sense

Sounds like a really straight forward C-API to me, and also quite efficient.


> and (2) is there any real use case for such
> an API?  E.g. would the expected performance gains of Cython justify
> the rewrite needed to handle such an API, which is quite different
> from CPython's?

Very different, yes. I can't fully estimate the amount of work it would
take to port Cython to it, but it's definitely not small, and there's no
quick win along the path. The API is so different that we can only port it
completely or not at all, no mixing possible.

If Cython wants to continue to support both C-APIs in the same C code file
(and switch at C compile time), this will add a lot of C code overhead.
Basically all Python related code would have to be duplicated, including
object variables and error tests. It will also mean a lot of code overhead
in the code generation part of the compiler.

I'm not convinced that this is worth it solely for the purpose of
interfacing with Cython, but if PyPy wants a suitable C-API for itself (and
you should seriously consider that), then this would be a suitable
long-term solution IMHO.

A more immediate solution would obviously be to bring cpyext up to speed...

Stefan