[Python-Dev] SVN rev. 85392 broke module handling in py3k

[Stefan Behnel]

Antoine Pitrou, 22.10.2010 16:55:
> On Fri, 22 Oct 2010 09:41:09 -0500
> Benjamin Peterson wrote:
>> 2010/10/22 exarkun:
>>> Instances of classes don't refer to the module their class is defined in.
>>>   It seems more likely that the reason the module is garbage collected is
>>> that there really is nothing which refers to it anymore.
>>
>> Indeed, this is really a Python bug, but there's no good way to deal
>> with it unless dictionaries can know they are module globals.
>
> How about making functions keep a reference to the module they're
> defined in? Is there any reason we shouldn't do that?

I think the general problem is that "the module" can be a pretty broad 
thing, potentially referring to all sorts of stuff such as (usually) 
several other modules. Think of an extreme case where you build and return 
a local function in a module function. Now all you use the module for is to 
get at the outer function and call it once to get an instance of the inner 
function. But the module must stay alive as long as the inner function 
lives, even if its closure is completely self-sufficient. You could 
optimise this specific case, but as soon as as the function refers to any 
non-local name, you'd have to keep the entire module dict alive in case 
someone decides to change it.

It would also mean that each and every function will end up in a reference 
cycle by default that will require garbage collection for cleanup. This 
isn't currently a common case, as only modules refer to their content and 
rarely the other way round.

Augmenting the function closure to include (indirections to) module globals 
won't work very well either, as it would either mean you still keep the 
module dict alive or it would complicate modifications to the module dict 
that would need to be reflected in the closures in some way. That gets us 
back to the usual space-time tradeoff, out of which we currently prefer 
both at the cost of less safe behaviour in the (rather rare) case of module 
unloading.

Stefan