[Python-Dev] Problem between deallocation of modules and func_globals

[M.-A. Lemburg]

On 2007-01-20 00:01, Brett Cannon wrote:
> On 1/19/07, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 2007-01-19 22:33, Brett Cannon wrote:
>>>> That's a typical error situation you get in __del__ methods at
>>>> the time the interpreter is shut down.
>>>>
>>> Yeah, but in this case this is at the end of Py_Initialize() for the
>>> stuff I am doing to the interpreter.  =)
>> Is that in some error branch of Py_Initialize() ? Otherwise
>> I don't see how the modules could get garbage-collected.
>>
> 
> Nope, it's code I am adding to clean out sys.modules of stuff the user
> didn't import themselves; it's for security reasons.

I'm not sure whether that's really going to increase
security: unloading of modules usually isn't safe and you
cannot be sure that it's possible to reinitialize a C
module once it has been loaded in the process. For Python
modules this is often possible, but there still may be
side-effects of the import that you cannot easily undo.

Perhaps you should just move those modules out to a different
dictionary and keep track of it in the import mechanism, so
that while you can't access the module directly via sys.modules,
the import mechanism still knows that it has been loaded and
reinserts it into sys.modules if it gets imported again.

I think that you get more security by explicitly
limiting which modules and packages you allow to be imported
in the first place and restricting what can be done with
sys.path and sys.modules.

>>>> I'm not exactly sure which global state you are referring to. The
>>>> aliase map, the cache used by the search function ?
>>>>
>>> encodings._cache .
>>>
>>>> Note that the search function registry is a global managed
>>>> in the thread state (it's not stored in any module).
>>>>
>>> Right, but that is not the issue.  If you have deleted the reference
>>> to the encodings module from sys.modules it then sets encodings._cache
>>> to None.  After the deletion, if you try to encode/decode a unicode
>>> string you can an AttributeError about how encodings._cache does not
>>> have a 'get' method since it is now None instead of a dict.  The
>>> function is fine and still runs, it's just that the global state it
>>> depends on is no longer the way it assume it should be.
>> While I could add some tricks to have the cache dictionary stay
>> alive even after the globals were set to None, I doubt that this
>> will really fix the problem.
>>
>> The encoding package relies on the import mechanism, the codecs
>> module and the _codecs builtin module. Any of these could fail
>> to work depending on the order in which the modules get
>> GCed.
>>
>> There's a reason why things in Py_Finalize() are as carefully
>> ordered :-) Perhaps we need to apply some reordering to the
>> steps in Py_Initialize() ?!
>
> 
> Nah, I just  need to not delete the modules.  =)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 20 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::