[Python-Dev] Strange artifacts with PEP 3121 and monkey-patching sys.modules (in csv, ElementTree and others)

[Stefan Behnel]

Antoine Pitrou, 11.08.2013 13:48:
> On Sun, 11 Aug 2013 07:04:40 -0400 Nick Coghlan wrote:
>> On 11 August 2013 06:33, Antoine Pitrou wrote:
>>> So code can be written like:
>>>
>>>   PyObject *dialects = PyState_GetModuleAttr(
>>>       &_csvmodule, "dialects", &PyDict_Type);
>>>   if (dialects == NULL)
>>>       return NULL;
>>
>> This sounds like a good near term solution to me.
>>
>> Longer term, I think there may be value in providing a richer
>> extension module initialisation API that lets extension modules be
>> represented as module *subclasses* in sys.modules, since that would
>> get us to a position where it is possible to have *multiple* instances
>> of an extension module in the *same* subinterpreter by holding on to
>> external references after removing them from sys.modules (which is
>> what we do in the test suite for pure Python modules).
> 
> Either that, or add a "struct PyMemberDef *m_members" field to
> PyModuleDef, to enable looking up stuff in the m_state using regular
> attribute lookup.

Hmm, yes, it's unfortunate that the module state isn't just a public part
of the object struct.


> Unfortunately, doing so would probably break the ABI. Also, allowing
> for module subclasses is probably more flexible in the long term.

+1000


> We
> just need to devise a convenience API for that (perhaps by allowing to
> create both the subclass *and* instantiate it in a single call).

Right. This conflicts somewhat with the simplified module creation. If the
module loader passed the readily instantiated module instance into the
module init function, then module subtypes don't fit into this scheme anymore.

One more reason why modules shouldn't be special. Essentially, we need an
m_new() and m_init() for them. And the lifetime of the module type would
have to be linked to the (sub-)interpreter, whereas the lifetime of the
module instance would be determined by whoever uses the module and/or
decides to unload/reload it.

Stefan