[Python-Dev] Confusing behaviour relating to module objects being GCed.

[Guido van Rossum]

> Anyway, I'm having a problem in a large framework relating to using custom
> __import__ and reload hooks.  I've been able to reduce the code that
> demonstrates this problem to this Python code:
> 
> Python 2.3b1 (#1, Apr 27 2003, 22:07:38)
> [GCC 2.95.3 20010315 (release)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import new
> >>> mone = new.module('mone')
> >>> mone.mtwo = new.module('mtwo')
> >>> mone.mtwo.var = 42
> >>> exec "def func():\n  print var\n  delmtwo()\n  print var\n" in
> mone.mtwo.__dict__
> >>> def delmtwo():
> ...   delattr(mone, 'mtwo')
> ...
> >>> mone.mtwo.delmtwo = delmtwo
> >>> mone.mtwo.func()             # FIRST CASE
> 42
> None
> >>> mone.mtwo = new.module('mtwo')
> >>> mone.mtwo.var = 42
> >>> exec "def func():\n  print var\n  delmtwo()\n  print var\n" in
> mone.mtwo.__dict__
> >>> keepref = mone.mtwo
> >>> mone.mtwo.delmtwo = delmtwo
> >>> mone.mtwo.func()             # SECOND CASE
> 42
> 42
> 
> The problem is that after mone.mtwo.func() calls delmtwo() which
> removes the mtwo module from mone and thus reduces the ref-count to
> 0 (in the first case), func.im_func.func_globals starts showing a
> very peculiar behaviour.  Basically, all of the module-level
> variable *names* stay, but all the *values* become None.  As the
> second call to func() demonstrates, if I keep a ref to the module
> which is deleted from mone, everything is fine.
> 
> I am at a loss explaining this behaviour (not knowing anything about
> Python internals) and I would greatly appreciate enlightment that
> either confirms that this is a bug, or explains why it isn't :-)

The crucial part here is "not knowing anything about Python internals"
:-).  Please take some time to read the Python C source code.  If you
don't have it downloaded, you can view it online here:

  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/

Look at the destructor for module objects (file
Objects/moduleobject.c, function module_dealloc()).  This is what
destroys the module when its refcount goes to zero.  It calls
_PyModule_Clear() which carefully sets all the module's global
variables to None.  The reason this is done is hard to explain; it has
to do with trying to break cyclical references between a module and
the classes and functions it defines, in an attempt at destroying as
many objects as possible when Python exits, in the hope that this will
cause user-defined destructors (__del__ methods) to be callled, which
might do useful things like flushing buffers, removing temporary
files, and releasing other external resources.  (Closing files and
releasing memory are *not* goals here, since this happens anyway when
a process exits.)

--Guido van Rossum (home page: http://www.python.org/~guido/)