[ python-Bugs-838140 ] Unloading extension modules not always safe

[SourceForge.net]

Bugs item #838140, was opened at 2003-11-07 21:44
Message generated for change (Comment added) made by ronaldoussoren
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=838140&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Bob Ippolito (etrepum)
Assigned to: Jack Jansen (jackjansen)
Summary: Unloading extension modules not always safe

Initial Comment:
I will look into the solution for this, but the "for now" 
solution would be to never try and unlink bundles because 
they may contain ObjC code.  For several reasons, ObjC 
classes can never be unloaded, so bundles containing ObjC 
code can also never be unloaded.  This is more than a 
problem for just PyObjC, because any arbitrary module may 
contain some ObjC code.  We need to detect this before 
running NSUnlinkModule.  I'll try and put together a patch 
sometime soon if nobody else does, but for now see: http://
sourceforge.net/tracker/?
func=detail&aid=832025&group_id=14534&atid=114534

----------------------------------------------------------------------

Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2003-11-25 15:36

Message:
Logged In: YES 
user_id=580910

A quick workaround would be to let pydoc check the type of 
module it tries to load, it currently checks if the file is a binary 
module by checking "if info and 'b' in info[2]" (pydoc.py:171). If it 
would check if the filetype is not C_EXTENSION (3) the problem 
would not arise with pydoc (which probably accounts for >99% of 
the problematic load_module calls).

----------------------------------------------------------------------

Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2003-11-25 15:18

Message:
Logged In: YES 
user_id=580910

Note that the problem doesn't really have anything to do with 
unloading modules, the problem occurs when trying to unload a 
dylib that doesn't contain the proper init function. See bug 
#848907 (sorry about that one, I didn't look for earlier bug 
reports).

What happens:
- pydoc calls imp.load_module('__temp__', open('_objc.so'), ...)
- load_module loads _objc.so
- load_module tries to find init__temp__ in _objc.so
- nothing is found and therefore load_module calls NSUnlinkModule
- NSUnlinkModule aborts the process because _objc.so contains 
Objective-C definitions

No code in the extension module has any change to fix things 
because the init function is never called.

BTW. The problem also exists in a vanilla Python 2.3 installation on 
Panther, I cleared /Library/Python2.3 and ran 'pydoc -k hello' and 
this also crashed.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-11-14 11:40

Message:
Logged In: YES 
user_id=45365

I think that either the module itself, or the package responsible for 
the module, should forestall unloading (by tucking away an extra 
reference somewhere).

The root of the problem is that the Python refcount doesn't reflect 
the all references to the module: ObjC keeps references too.

OTOH: if all modules containing ObjC code have this problem, and 
it is easy to detect whether a module contains ObjC code then 
adding a safety net to the import code might be a prudent course 
of action.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-11-11 16:16

Message:
Logged In: YES 
user_id=45365

I'm surprised that it does the unload:-)

I think the correct solution would be for the module itself (or 
someone close to it) to stash away a reference. As this is only a 
problem for some modules (those containing ObjC code) I don't 
think a general change is in order.

The real problem is that the "last reference" as Python sees it isn't 
really the last reference: the ObjC runtime also has references to 
stuff in there.


----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-11-11 15:20

Message:
Logged In: YES 
user_id=139309

it does if you del sys.modules['somemodule'] and somemodule's 
reference count goes to zero.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-11-11 11:51

Message:
Logged In: YES 
user_id=45365

Bob, I'm confused. As far as I know Python never unloads 
extension modules...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=838140&group_id=5470