[ python-Bugs-1075703 ] PyGILState_Ensure() deadlocks (ver 2.4)

[SourceForge.net]

Bugs item #1075703, was opened at 2004-11-29 17:54
Message generated for change (Comment added) made by andivajda
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075703&group_id=5470

Category: Threads
Group: Python 2.4
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Andi Vajda (andivajda)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyGILState_Ensure() deadlocks (ver 2.4)

Initial Comment:
PyLucene (http://pylucene.osafoundation.org) wraps
libgcj with python. Python objects are also wrapped by
java objects. The ref count of the python is
incremented in the wrapping java object constructor and
is decremented in the wrapping java object's finalize()
method. Yes, finalize() is not a reliable means of
cleaning memory up but, so far, I wasn't able to figure
out a better way to do this. The finalize() method runs
in its own libgcj finalizer thread.
It has to acquire the GIL before it can call
Py_DECREF() is my understanding. The java finalize()
calls this code in PyLucene:

void finalizeObject(PyObject *object)
{
#if PY_VERSION_HEX >= 0x02040000
    PyGILState_STATE state = PyGILState_Ensure();

    Py_DECREF(object);
    PyGILState_Release(state);
#endif
}

When compiled with python 2.4c1, this mostly works.
Except that it deadlocks, every now and then. If I
comment this code out, no deadlock. I also verified
that the finalizer is called at most once per such
object. I realize that reproducing this bug maybe hard,
building PyLucene is not particularly easy, unless you
have gcj ready.
If you contact me, I'm more than willing to help in
providing all that's needed to reproduce this. I've
seen the deadlock on Mac OS X 10.3.6, my main
development OS.
For what it's worth, I also tried this with python
2.3.4 (including threadmodule.c patch made in
September, related to GIL function usage).

----------------------------------------------------------------------

>Comment By: Andi Vajda (andivajda)
Date: 2004-12-04 20:15

Message:
Logged In: YES 
user_id=1139346

I finally tracked down this bug to gcj bug <a
href="http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16478">16478</a>
which is fixed in gcj version >= 3.4.2.
I am hence closing this python bug as invalid.

----------------------------------------------------------------------

Comment By: Andi Vajda (andivajda)
Date: 2004-12-03 19:42

Message:
Logged In: YES 
user_id=1139346

I've been running the same test loop on Gentoo/2.6/python
2.4 and was not able to get it to deadlock so far. This bug
could be limited to Mac OS X.

----------------------------------------------------------------------

Comment By: Andi Vajda (andivajda)
Date: 2004-12-03 19:10

Message:
Logged In: YES 
user_id=1139346

I verified this now with python 2.4 final. It still happens
as tested on Mac OS X.
I was also able to narrow it down to a unit test, that when
run repeatedly will eventually lock up. It is attached.
If you need assistance building PyLucene from sources, or
want me to provide a binary, let me know. At the moment, to
reproduce this bug, it is best to build PyLucene from
subversion.
See http://pylucene.osafoundation.org for more information.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1075703&group_id=5470