[Python-bugs-list] [ python-Bugs-660476 ] readline and threads crashes

[SourceForge.net]

Bugs item #660476, was opened at 2002-12-31 14:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660476&group_id=5470

Category: Threads
Group: Python 2.2.2
>Status: Closed
>Resolution: Accepted
Priority: 7
Submitted By: Michael Stone (mbrierst)
Assigned to: Guido van Rossum (gvanrossum)
Summary: readline and threads crashes

Initial Comment:

Python 2.2.2 with linux kernel 2.2.17
paste into the interpreter:


import readline, rlcompleter
readline.parse_and_bind('tab: complete')
import thread
import math
def starter(a):
  for i in range(1000000):
    a = math.cos(i)

thread.start_new_thread(starter, (1,))


now type any letter and hit <tab>
segfault or "Fatal Python error: ceval: tstate mix-up"
will occur, if not the first time, then eventually if you keep hitting tab

this might be related to:
http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=513033
but I'm posting anyway as I'm not sure it's the same, and that one doesn't seem to be getting attention

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-07 15:33

Message:
Logged In: YES 
user_id=6380

Thanks! The fix looks good and works. Checked in as
readline.c 2.58.
Will check into 2.2 maintenance branch too.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-07 13:59

Message:
Logged In: YES 
user_id=6380

I can reproduce this in both 2.2.2 and 2.3CVS, on Linux.

I've looked at the other bug report, and I agree that the
explanation there is correct. The recipe to reproduce it
matches that description.

I'll review the patch shortly.

Boy I am embarrassed that this has sat on SF since last March!!!

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-12-31 15:58

Message:
Logged In: YES 
user_id=6380

Will look after 2.3a1 is released.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-12-31 15:53

Message:
Logged In: YES 
user_id=33168

I cannot replicate this for 2.3, but I get the crash for 2.2.2.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2002-12-31 15:33

Message:
Logged In: YES 
user_id=670441


I don't claim to have a real good understanding of the python internals, but it seems like someone was doing some obviously wrong thread stuff.  The diff below gives a fix that seems to solve my problems.  
After looking at it more, this does seem to be the same bug listed in the link in original comment.

--- readline.c.old      Tue Dec 31 15:15:45 2002
+++ readline.c  Tue Dec 31 15:09:08 2002
@@ -404,16 +404,14 @@
 /* C function to call the Python hooks. */
 
 static int
-on_hook(PyObject *func, PyThreadState *tstate)
+on_hook(PyObject *func, PyThreadState **tstate)
 {
        int result = 0;
        if (func != NULL) {
                PyObject *r;
-               PyThreadState *save_tstate;
                /* Note that readline is called with the interpreter
                   lock released! */
-               save_tstate = PyThreadState_Swap(NULL);
-               PyEval_RestoreThread(tstate);
+               PyEval_RestoreThread(*tstate);
                r = PyObject_CallFunction(func, NULL);
                if (r == NULL)
                        goto error;
@@ -427,8 +425,7 @@
                PyErr_Clear();
                Py_XDECREF(r);
          done:
-               PyEval_SaveThread();
-               PyThreadState_Swap(save_tstate);
+               *tstate = PyEval_SaveThread();
        }
        return result;
 }
@@ -436,14 +433,14 @@
 static int
 on_startup_hook(void)
 {
-       return on_hook(startup_hook, startup_hook_tstate);
+       return on_hook(startup_hook, &startup_hook_tstate);
 }
 
 #ifdef HAVE_RL_PRE_INPUT_HOOK
 static int
 on_pre_input_hook(void)
 {
-       return on_hook(pre_input_hook, pre_input_hook_tstate);
+       return on_hook(pre_input_hook, &pre_input_hook_tstate);
 }
 #endif
 
@@ -455,10 +452,8 @@
        char *result = NULL;
        if (completer != NULL) {
                PyObject *r;
-               PyThreadState *save_tstate;
                /* Note that readline is called with the interpreter
                   lock released! */
-               save_tstate = PyThreadState_Swap(NULL);
                PyEval_RestoreThread(completer_tstate);
                r = PyObject_CallFunction(completer, "si", text, state);
                if (r == NULL)
@@ -478,8 +473,7 @@
                PyErr_Clear();
                Py_XDECREF(r);
          done:
-               PyEval_SaveThread();
-               PyThreadState_Swap(save_tstate);
+               completer_tstate = PyEval_SaveThread();
        }
        return result;
 }

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660476&group_id=5470