Calling an unbound method in C using the Public API

Matthieu Dartiailh m.dartiailh at gmail.com
Wed Aug 29 10:33:28 EDT 2018


Hi,

I am one of the maintainer of the atom library (https://github.com/nucleic/atom <https://github.com/nucleic/atom>). This library provides low-memory footprint Python objects, descriptors and containers enforcing type validation, implements the observer pattern. 

For list, we basically subclass the standard Python list in C++ and add the required checks/notifications to some methods (in particular insert, pop and sort). To call the method of standard list object on our custom object, the original author chose to directly access the c-function inside the MethodDef and call it (by first casting to the proper function pointer type).

Starting with Python 3.7, insert, pop and sort use the FASTCALL calling convention, which means that the signature of the function stored inside the MethodDef have changed. I have a working solution, but it involves to use a lot of the CPython private C-API. In particular I needed to access:
_PyCFunctionFast
_PyCFunctionFastWithKeywords
_PyStack_UnpackDict

I tried to look at the public C API for a way to call an unbound method with a minimal cost (in term of speed and memory). It seems to me, but please correct me if I am wrong, that one cannot call a MethodDef using only the public API. To use the public C API, one has to use PyCFunction_Call (or a variant) that expect a PyCFunctionObject which binds a the MethodDef to an instance. In my case, to avoid creating a temporary PyCFunctionObject each time I call list.insert on my custom subclass instance, I have to store that PyCFunctionObject for each instance. But this means storing  7 PyCFunctionObject per instance (one for each method of list I need to wrap). So I can either use the public API and increase the memory footprint or slow down the code by creating PyCFunctionObject for each call, or use large amount of the private API.

Am I missing something ? 

Best regards

Matthieu




More information about the Python-list mailing list