[Python-Dev] On the METH_FASTCALL calling convention

[Stefan Behnel]

Hi Serhiy!

Serhiy Storchaka schrieb am 07.07.2018 um 10:55:
> There is my idea. Split every of keyword argument parsing functions on two
> parts. The first part linearize keyword arguments, it converts positional
> and keyword arguments (in whatever form they were presented) into a linear
> array of PyObject* (with NULLs for not specified optional arguments). The
> second part is common and similar to _PyArg_ParseStack(), but supports
> NULLs. It converts an array of PyObject* to a sequence of C values. I tried
> to implement this idea, is is not simple, and results were mixed, but I
> don't loss a hope.

That proposal sounds good to me. Cython currently does something similar
/inside/ of its function entry code, in that it executes an unrolled series
of PyDict_GetItem() calls for the expected keyword arguments (instead of
iterating over a dict, which turned out to be slower), and maps those to an
array of arguments, all before it passes over that array to convert the
values to the expected C types. I agree that it makes sense to do the name
matching outside of the callee since the caller knows best in what way
(sequence, dict, ...) the arguments are available and can decide on the
fastest way to map them to a flat array, given the expected argument names.

And I think the proposal would fit nicely into PEP-580.


> And here we return to METH_FASTCALL|METH_KEYWORDS. The first part of
> handling arguments can be made outside of the C function, by the calling
> API. Then the signature of the C function can be simpler, the same as for
> METH_FASTCALL. But we need to expose the list of keyword parameter names as
> an attribute of CFunction.

And names should be expected to be interned, so that matching the keywords
can be done via pointer comparisons in almost all cases. That should make
it pretty fast, and errors can be detected in a slow separate pass if the
pointer matching fails. I think we cannot strictly assume a predictable,
helpful ordering of the keyword arguments on the calling side (that would
allow for a one-pass merge), but it's rather common for users to pass
keyword arguments in the order in which the signature expects them, so I'm
sure there's a fast algorithm (e.g. something like Insertion Sort) to match
both sides in negligible time.

Stefan