On 2007-01-18 20:53, Brett Cannon wrote:
I have discovered an issue relating to func_globals for functions and the deallocation of the module it is contained within. Let's say you store a reference to the function encodings.search_function from the 'encodings' module (this came up in C code, but I don't see why it couldn't happen in Python code). Then you delete the one reference to the module that is stored in sys.modules, leading to its deallocation. That triggers the setting of None to every value in encodings.__dict__.
Oops, now the global namespace for that module has everything valued at None. The dict doesn't get deallocated since a reference is held by encodings.search_function.func_globals and there is still a reference to that (technically held in the interpreter's codec_search_path field). So the function can still execute, but throws exceptions like AttributeError because a module variable that once held a dict now has None and thus doesn't have the 'get' method.
That's a typical error situation you get in __del__ methods at the time the interpreter is shut down. The main reason for setting everything to None first is to break circular references and make sure that at least some of the object destructors can run.
My question is whether this is at all worth trying to rectify. Since Google didn't turn anything up I am going to guess this is not exactly a common thing. =) That would lead me to believe some (probably most) of you will say, "just leave it alone and work around it".
If you can come up with a better way, sure :-)
The other option I can think of is to store a reference to the module instead of just to its __dict__ in the function. The problem with that is we end up with a circular dependency of the functions in modules having a reference to the module but then the module having a reference to the functions. I tried not having the values in the module's __dict__ set to None if the reference count was above 1 and that solved this issue, but that leads to dangling references on anything in that dict that does not have a reference stored away somewhere else like encodings.search_function.
Anybody have any ideas on how to deal with this short of rewriting some codecs stuff so that they don't depend on global state in the module or just telling me to just live with it?
I'm not exactly sure which global state you are referring to. The aliase map, the cache used by the search function ? Note that the search function registry is a global managed in the thread state (it's not stored in any module). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 19 2007)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::