[Python-Dev] Strange artifacts with PEP 3121 and monkey-patching sys.modules (in csv, ElementTree and others)

[Nick Coghlan]

On 10 Aug 2013 21:06, "Eli Bendersky" <eliben at gmail.com> wrote:
>
> n Sat, Aug 10, 2013 at 5:47 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> In a similar vein, Antoine recently noted that the fact the per-module
state isn't a real PyObject creates a variety of interesting lifecycle
management challenges.
>>
>> I'm not seeing an easy solution, either, except to automatically skip
reinitialization when the module has already been imported.
>
> This solution has problems. For example, in the case of ET it would
preclude testing what happens when pyexpat is disabled (remember we were
discussing this...). This is because there would be no real way to create
new instances of such modules (they would all cache themselves in the init
function - similarly to what ET now does in trunk, because otherwise some
of its global-dependent crazy tests fail).

Right, it would still be broken, just in a less horrible way.

>
> A more radical solution would be to *really* have multiple instances of
state per sub-interpreter. Well, they already exist -- it's
PyState_FindModule which is the problematic one because it only remembers
the last one. But I see that it's only being used by extension modules
themselves, to efficiently find modules they belong to. It feels a bit like
a hack that was made to avoid rewriting lots of code, because in general a
module's objects *can* know which module instance they came from. E.g. it
can be saved as a private field in classes exported by the module.
>
> So a more radical approach would be:
>
> PyState_FindModule can be deprecated, but still exist and be documented
to return the state the *last* module created in this sub-interpreter.
stdlib extension modules that actually use this mechanism can be rewritten
to just remember the module for real, and not rely on PyState_FindModule to
fetch it from a global cache. I don't think this would be hard, and it
would make the good intention of PEP 3121 more real - actual intependent
state per module instance.

Sounds promising to me. I suspect handling exported functions will prove to
be tricky, though - they may need to be redesigned to behave more like
"module methods".

>
> Eli
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130811/ee723e56/attachment.html>