keeping local state in an C extension module

[Stefan Behnel]

Daniel Franke, 30.06.2011 12:07:
> For a project I implemented a extension module in C. Given the
> excellent documentation everything was straightforward and works fine
> so far.

Don't miss out taking a look at Cython, just in case it's going to be a 
non-trivial project.


> Up until now I kept local state in global static variables, but would
> like to change this to pass a pointer to a state structure around. The
> Py_InitModule4() [1] function seems to allow this by giving the user
> the option to specify the 'self' object passed to the module's
> functions. In this case, the 'self' argument should be a raw C pointer
> malloc'ed elsewhere that should be rolled into a PyObject and be made
> accessible later in the extension functions.

There's a PEP for Py3 that enables this through a general framework. In 
general, to achieve this, you may want to allocate the module content 
(including types etc.) on the heap rather than statically.

http://www.python.org/dev/peps/pep-3121/


> Example code:
>
> static PyObject* module_func(PyObject *self, PyObject *args) {
>    state_t *state = (state_t*) PyRawPointerFromPyObject(self);
>    /* use state */
>    Py_RETURN_NONE;
> }
>
> static PyMethodDef bms_methods[] = {
>    { "func", module_func, METH_VARARGS, "" },
>    { NULL, NULL, NULL, NULL }
> };
>
> void my_module_init(state_t *state) {
>    Py_InitModule4("_mymodule", module_methods,
>                   "My Extension Module",
>                   PyObjectFromRawPointer(state),
>                   PYTHON_API_VERSION);
> }
>
> Here, of course, the functions PyObjectFromRawPointer(void*) and void*
> PyRawPointerFromPyObject(PyObject*) are missing. Is there anything
> like this in the Python C-API? If not, how could it be implemented? Or
> is this approach a stupid idea to begin with? Pointers would be highly
> appreciated.

Spot the pun. ;)

Take a look at PyCapsule, it may (or may not) be enough for your use case. 
However, note the comment about Py_InitModule4() in the docs, where it says 
that the 'self' passing feature isn't used very often. You may or may not 
want to (or need to) use it.

Stefan