[Python-Dev] Problem between deallocation of modules and func_globals

M.-A. Lemburg mal at egenix.com
Fri Jan 19 10:25:58 CET 2007


On 2007-01-18 20:53, Brett Cannon wrote:
> I have discovered an issue relating to func_globals for functions and
> the deallocation of the module it is contained within.  Let's say you
> store a reference to the function encodings.search_function from the
> 'encodings' module (this came up in C code, but I don't see why it
> couldn't happen in Python code).  Then you delete the one reference to
> the module that is stored in sys.modules, leading to its deallocation.
>  That triggers the setting of None to every value in
> encodings.__dict__.
> 
> Oops, now the global namespace for that module has everything valued
> at None.  The dict doesn't get deallocated since a reference is held
> by encodings.search_function.func_globals and there is still a
> reference to that (technically held in the interpreter's
> codec_search_path field).  So the function can still execute, but
> throws exceptions like AttributeError because a module variable that
> once held a dict now has None and thus doesn't have the 'get' method.

That's a typical error situation you get in __del__ methods at
the time the interpreter is shut down.

The main reason for setting everything to None first is to
break circular references and make sure that at least some
of the object destructors can run.

> My question is whether this is at all worth trying to rectify.  Since
> Google didn't turn anything up I am going to guess this is not exactly
> a common thing.  =)  That would lead me to believe some (probably
> most) of you will say, "just leave it alone and work around it".

If you can come up with a better way, sure :-)

> The other option I can think of is to store a reference to the module
> instead of just to its __dict__ in the function.  The problem with
> that is we end up with a circular dependency of the functions in
> modules having a reference to the module but then the module having a
> reference to the functions.  I tried not having the values in the
> module's __dict__ set to None if the reference count was above 1 and
> that solved this issue, but that leads to dangling references on
> anything in that dict that does not have a reference stored away
> somewhere else like encodings.search_function.
> 
> Anybody have any ideas on how to deal with this short of rewriting
> some codecs stuff so that they don't depend on global state in the
> module or just telling me to just live with it?

I'm not exactly sure which global state you are referring to. The
aliase map, the cache used by the search function ?

Note that the search function registry is a global managed
in the thread state (it's not stored in any module).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 19 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


More information about the Python-Dev mailing list