[Python-Dev] Pre-PEP: Redesigning extension modules

Stefan Behnel stefan_ml at behnel.de
Sat Aug 31 22:35:42 CEST 2013


Antoine Pitrou, 31.08.2013 21:27:
> On Sat, 31 Aug 2013 21:16:10 +0200
> Stefan Behnel wrote:
>>> Our experience is very different - my perspective is that the normal case
>>> either eschews C level global state in the extension module, because it
>>> causes so many problems, or else just completely ignores subinterpreter
>>> support and proper module cleanup.
>>
>> As soon as you have more than one extension type in your module, and they
>> interact with each other, they will almost certainly have to do type checks
>> against each other to make sure users haven't passed them rubbish before
>> they access any C struct fields of the object. Doing a type check means
>> that at least one type has a pointer to the other, meaning that it holds
>> global module state.
>>
>> I really think that having some kind of global module state is the
>> exceedingly common case for an extension module.
> 
> Since we are eating our own dogfood here (and the work which prompted
> this discussion was indeed about trying to make our extension modules
> more cleanup-friendly), it would be nice to take a look at the Modules
> directory and count which proportion of CPython extension modules have
> state.

There seem to be 81 modules in there currently (grepped for PyMODINIT_FUNC).

16 of them come up when you grep for '(TypeCheck|IsInstance)', all using
global extension type pointers.

32 use some kind of global "static PyObject* something;".

Both add up to 41. That's half of the modules already. I'm sure there's
more if you dig deeper.

Some modules only define functions or only one type (e.g. md5). They would
get away with no global state, I guess - if they all used heap types.


> Caution: "state" is a bit vague here. Depending on which API you use,
> custom extension types can be a part of "module state".

Yep, as I said.

Stefan




More information about the Python-Dev mailing list