[ python-Bugs-1332869 ] Fatal Python error: Interpreter not initialized

Fri Mar 31 19:46:46 CEST 2006

Bugs item #1332869, was opened at 2005-10-20 01:22
Message generated for change (Comment added) made by tim_one
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1332869&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Andrew Mitchell (ajmitch)
>Assigned to: Neal Norwitz (nnorwitz)
Summary:  Fatal Python error: Interpreter not initialized 

Initial Comment:
When running 'bzr status' on Ubuntu 5.10 with python
2.4.2, I came across the error:
Fatal Python error: Interpreter not initialized
(version mismatch?)
Aborted

The bzr code in question has a __del__ method which
when invoked (eventually) imports socket (which imports
_socket), causing everything to fall over in a heap.

A backtrace is available at http://pastebin.com/399461

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2006-03-31 12:46

Message:
Logged In: YES 
user_id=31435

Sorry for the delay!  I didn't realize this was assigned to
me.  Note that the possiblity for problems here was already
noted in Py_Finalize():

	/* Collect garbage.  This may call finalizers; it's nice to
call these
	 * before all modules are destroyed.
	 * XXX If a __del__ or weakref callback is triggered here,
and tries to
	 * XXX import a module, bad things can happen, because
Python no
	 * XXX longer believes it's initialized.
	 * XXX     Fatal Python error: Interpreter not initialized
(version mismatch?)
	 * XXX is easy to provoke that way.  I've also seen, e.g.,
	 * XXX     Exception exceptions.ImportError: 'No module
named sha'
	 * XXX         in <function callback at 0x008F5718> ignored
	 * XXX but I'm unclear on exactly how that one happens.  In
any case,
	 * XXX I haven't seen a real-life report of either of these.
	 */
	PyGC_Collect();

I don't think it would do much harm to move "initialized =
0" down, although it's impossible to get truly concerned
about code doing imports in __del__ when the interpreter is
tearing itself down.  Note that sys exit funcs have already
been called, and signals have been disabled, by this time,
so there would still be ways for dubious __del__ code to fail.

Anyway, if you want to do this, the XXX comments reproduced
above should be changed too (e.g., deleted).

Assigning to Neal, so that he can get practice following his
new "no changes without a test case" guideline :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-21 02:15

Message:
Logged In: YES 
user_id=33168

Well, what I really hope is that Tim can *make* some time to
*look* at this patch.  Feel free to long over it if you
wish. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-21 01:55

Message:
Logged In: YES 
user_id=33168

Hey Tim, I'm hoping you can take some time to long at this
patch (you too Michael and anyone else listening).

Attached is a patch which seems to fix this problem.  At
least it fixes it from Tim's example in the mail from Nov (I
think).  It moves initialized down after the GC collect.

Is this patch a good or bad thing?

Andrew, it would be interesting if you could test this patch
(you will need to rebuild python) for bzr.  I couldn't
trigger the problem from my version.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-10-20 06:09

Message:
Logged In: YES 
user_id=6656

Are you using daemon threads?

Why is a __del__ method importing things?

----------------------------------------------------------------------

Comment By: Andrew Mitchell (ajmitch)
Date: 2005-10-20 04:51

Message:
Logged In: YES 
user_id=19159

Also,  gdb did not report other modules loaded

----------------------------------------------------------------------

Comment By: Andrew Mitchell (ajmitch)
Date: 2005-10-20 04:04

Message:
Logged In: YES 
user_id=19159

Feedback in #bzr indicated that I was triggering this code:
http://pastebin.com/399491 and also something similar to
http://mail.python.org/pipermail/python-dev/2003-September/038151.html
Another example that triggers it is
http://mail.python.org/pipermail/python-dev/2003-November/040188.html
It's all a stock breezy system, no external modules, and bzr
is retrieved from running:
 rsync -av bazaar-ng.org::bazaar-ng/bzr/bzr.dev .
I was using the latest revision at the time this happened.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-20 01:39

Message:
Logged In: YES 
user_id=33168

Also, where is the bzr code and what version are you running?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-20 01:37

Message:
Logged In: YES 
user_id=33168

Are there any other extension modules in the system that are
not provided with python?  When you went into gdb did it
print a long list of dynamic modules loaded?  Can you
provide that info too?

I'm guessing this is a memory overwrite which can happen
with poorly behaved C extension modules.  You could try
running python under valgrind and see what it reports.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1332869&group_id=5470