[Python-bugs-list] [ python-Bugs-513033 ] unsafe call to PyThreadState_Swap

noreply@sourceforge.net noreply@sourceforge.net
Sun, 17 Mar 2002 19:15:01 -0800

Bugs item #513033, was opened at 2002-02-04 18:16
You can respond by visiting: 

Category: Python Library
Group: Python 2.2.1 candidate
Status: Open
Resolution: None
Priority: 5
Submitted By: Jake McGuire (jamcguir)
Assigned to: Guido van Rossum (gvanrossum)
Summary: unsafe call to PyThreadState_Swap

Initial Comment:
It appears that there is a blatantly unsafe call to 
PyThreadState_Swap in the functions on_hook and 
on_completer in Modules/Readline.c

The diff adding these calls is viewable at 

The call to PyThreadState_Swap is added directly below 
a comment pointing out that readline() is called with 
the interpreter lock released.  Viewing the code shows 
that the interpreter lock is indeed released before 
calling readline (in myreadline.c).

Multithreaded programs that define callback functions 
suffer from intermittent crashes, often Py_FatalError-
ing claiming "tstate mix-up" from ceval.c

Removing the calls to PyThreadState_Swap makes these 
problems go away.

Can someone explain how the call to PyThreadState_Swap 
is indeed the right thing to be doing?


>Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-17 22:15

Logged In: YES 

That's ancient history and will take some time to wrap my
brains around. I don't have time for this tonight, maybe
tomorrow but I doubt the priority can be raised enough to
catch my attention long enough to matter. But it should be
fixed for 2.3.


Comment By: Michael Hudson (mwh)
Date: 2002-03-16 12:40

Logged In: YES 

jamcguir (or indeed anyone else), can you provide a patch
for this pronto?  I'd kind of like to have it fixed in
2.2.1, but this is obscure enough that I'm not going to hold
up the release for it.


Comment By: Jake McGuire (jamcguir)
Date: 2002-02-04 18:55

Logged In: YES 

Unfortunately, the scenario isn't really *simple*.

I think it goes like this:

Thread A defines a readline startup hook.
Thread A calls PyOS_Readline() in myreadline.c
Thread A calls Py_BEGIN_ALLOW_THREADS, saving its thread 
state and setting the global thread state to NULL.
Thread A calls readline.
Thread A gets blocked, and Thread B gets scheduled.
Thread B grabs the global interpreter lock, and restores 
its thread state.
Thread B gets suspended, and Thread A gets scheduled.
-- note: Thread B has the intepreter lock --
Thread A calls PyThreadState_Swap in on_hook(), setting the 
current global thread state to NULL
Thread A calls PyEval_RestoreThread, which blocks waiting 
for the global interpreter lock
Thread B gets scheduled, tries to run, but finds that the 
global thread state is NULL.  Bad things happen.

Proposed solution:
myreadline.c:PyOS_Readline to calls to PyEval_SaveThread 
and PyEval_RestoreThread.


Comment By: Tim Peters (tim_one)
Date: 2002-02-04 18:41

Logged In: YES 

Guido's checkin comment said:

Darn.  When thread support is disabled, the BEGIN/END 
macros don't save and restore the tstate, but explicitly 
calling PyEval_SaveThread() does reset it!  While I think 
about how to fix this for real, here's a fix that avoids 
getting a fatal error.

Therefore I assigned the bug to Guido <wink>.  It would 
help if you could describe a specific simple scenario that 
provokes the problems you're seeing.


You can respond by visiting: