[ python-Bugs-1225584 ] crash in gcmodule.c on python reinitialization

[SourceForge.net]

Bugs item #1225584, was opened at 2005-06-22 14:43
Message generated for change (Comment added) made by nascheme
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1225584&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Alexander Miseler (amiseler)
Assigned to: Neil Schemenauer (nascheme)
Summary: crash in gcmodule.c on python reinitialization

Initial Comment:
i have a c++ windows application with embedded python
interpreter.

this code crashs:

Py_Initialize();
PyRun_SimpleString("import gc");
Py_Finalize();
Py_Initialize();
PyRun_SimpleString("import gc");	// <--- BOOM


the problem seems to be the global var "garbage" in
gcmodule.c
the var isn't cleared in the reinitialization and
becomes an invalid pointer.
setting it to NULL in Py_Finalize fixes the crash.
i use python version 2.3.4 but a short look at the
2.4.1 source indicates that the problem is still there.

----------------------------------------------------------------------

>Comment By: Neil Schemenauer (nascheme)
Date: 2005-06-23 18:34

Message:
Logged In: YES 
user_id=35752

Well, the fact that some things are broken is not a good
excuse to introduce more brokeness.  I guess decrefing the
gc.garbage list and letting any reference cycles contained
in it leak is better than sharing objects between interpreters.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-06-23 08:22

Message:
Logged In: YES 
user_id=6656

At this point you might be thinking that Py_NewInterpreter/
Py_EndInterpreter loops are not the best tested things in the world.  You'd 
be right.

Neil's fix *does* fix the crash, but also shares PyObject's between 
interpreter states, which is a bit of a no-no, but otoh, this problem is far 
from unique to gcmodule.c.

> besides: the incref is only in gcinit, but the list can be
> created in gcinit OR in handle_finalizers (and while i was
> looking at it in the debugger it always was handle_finalizers).

That slightly misses the point -- the reason you got a crash was because 
the PyModule_AddObject initgc stole a reference to garbage, so when the 
module got cleaned up, it got deallocated despite there still being a 
reference to it in the module level global.  If initgc is never run, the 
refcount of garbage stays at 1.

> i guess the proper way to fix it would be a cleanup function
> for the gcmodule that is called by Py_Finalize.

Yeah, but quite a lot of modules could do with that, I think.  Do you have 
the energy to work on a 'proper' solution?  I don't.

----------------------------------------------------------------------

Comment By: Alexander Miseler (amiseler)
Date: 2005-06-23 08:11

Message:
Logged In: YES 
user_id=693638

yes, Neil Schemenauer (nascheme) added a Py_INCREF(garbage)
to initgc.
that *MIGHT* fix the crash. but only by simply never
releasing the garbage list. the garbage variable is still
never reset to NULL and therefore no new garbage list is
created after reinitialization.
besides: the incref is only in gcinit, but the list can be
created in gcinit OR in handle_finalizers (and while i was
looking at it in the debugger it always was handle_finalizers).

i guess the proper way to fix it would be a cleanup function
for the gcmodule that is called by Py_Finalize. the cleanup
function should do a decref (if Neils incref stays in, i'm
not sure if it is needed) and then reset the global vars.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-06-22 15:59

Message:
Logged In: YES 
user_id=6656

Bizarrely, I think this got fixed in CVS HEAD just a few days ago.  Are you 
in a position to check that?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1225584&group_id=5470