[Python-Dev] Strange artifacts with PEP 3121 and monkey-patching sys.modules (in csv, ElementTree and others)

[Stefan Behnel]

Stefan Behnel, 11.08.2013 14:53:
> Stefan Behnel, 11.08.2013 14:48:
>> Antoine Pitrou, 11.08.2013 14:32:
>>> On Sun, 11 Aug 2013 14:16:10 +0200 Stefan Behnel wrote:
>>>>> We
>>>>> just need to devise a convenience API for that (perhaps by allowing to
>>>>> create both the subclass *and* instantiate it in a single call).
>>>>
>>>> Right. This conflicts somewhat with the simplified module creation. If the
>>>> module loader passed the readily instantiated module instance into the
>>>> module init function, then module subtypes don't fit into this scheme anymore.
>>>>
>>>> One more reason why modules shouldn't be special. Essentially, we need an
>>>> m_new() and m_init() for them. And the lifetime of the module type would
>>>> have to be linked to the (sub-)interpreter, whereas the lifetime of the
>>>> module instance would be determined by whoever uses the module and/or
>>>> decides to unload/reload it.
>>>
>>> It may be simpler if the only strong reference to the module type is in
>>> the module instance itself. Successive module initializations would get
>>> different types, but that shouldn't be a problem in practice.
>>
>> Agreed. Then the module instance would just be the only instance of a new
>> type that gets created each time the module initialised. Even if module
>> subtypes were to become common place once they are generally supported
>> (because they're the easiest way to store per-module state efficiently),
>> module reinitialisation should be rare enough to just buy them with a new
>> type for each. The size of the complete module state+dict will almost
>> always outweigh the size of the one additional type by factors.
> 
> BTW, this already suggests a simple module initialisation interface. The
> extension module would expose a function that returns a module type, and
> the loader/importer would then simply instantiate that. Nothing else is needed.

Actually, strike the word "module type" and replace it with "type". Is
there really a reason why Python needs a module type at all? I mean, you
can stick arbitrary objects in sys.modules, so why not allow arbitrary
types to be returned by the module creation function?

Stefan