I *did* notice at least one case where using f_tstate might actually be a mistake: theoretically it's possible that two or more threads alternate calling next() on a generator (if they wrap it in a critical section); AFAICT the f_tstate is never updated.
Right, f_tstate is never updated. I think there is another inconsistent situation, which can be created easily. If a generator is run by a different thread than it's creator, then the frame is run in that other thread. eval_fame correctly uses tstate, but if tracing is activated, call_trace uses frame->f_tstate for no obvious reason, which will probably mess up the tracing flags of the wrong thread.
Right.
Could you dig through CVS logs to find out when f_tstate was first introduced? Maybe there's a clue about why there.
Here is a head start on the research. The ceval.c use of tstate goes back to the introduction of generators in 2001. The use in traceback.c and sysmodule.c goes back to 1997 when per-thread globals were factored into a structure to support separate thread-state management. Prior to that, the history is more diffuse and harder to follow. Raymond Hettinger --------------------------------------------------------------------- Ceval.c -------- 2.252 (nascheme 21-Jun-01): PyThreadState *tstate = 2.252 (nascheme 21-Jun-01): * necessarily their creator. */ 2.255 (tim_one 23-Jun-01): Py_XINCREF(tstate->frame); 2.252 (nascheme 21-Jun-01): assert(f->f_back == NULL); 2.252 (nascheme 21-Jun-01): f->f_back = tstate->frame; 2.252 (nascheme 21-Jun-01): revision 2.255 date: 2001/06/23 05:47:56; author: tim_one; state: Exp; lines: +2 -2 gen_iternext(): Don't assume that the current thread state's frame is not NULL. I don't think it can be NULL from Python code, but if using generators via the C API I expect a NULL frame is possible. revision 2.252 date: 2001/06/21 02:41:10; author: nascheme; state: Exp; lines: +27 -14 Try to avoid creating reference cycles involving generators. Only keep a reference to f_back when its really needed. Do a little whitespace normalization as well. This whole file is a big war between tabs and spaces but now is probably not the time to reindent everything. traceback.c ------------ 2.22 (guido 05-May-97): PyThreadState *tstate = frame->f_tstate; 2.22 (guido 05-May-97): tracebackobject *oldtb = (tracebackobjec revision 2.22 date: 1997/05/05 20:56:15; author: guido; state: Exp; lines: +6 -30 Massive changes for separate thread state management. All per-thread globals are moved into a struct which is manipulated separately. The story for their life a globals pre-dates what I can sysmodule.c ------------ 2.45 (guido 02-Aug-97): PyThreadState *tstate = PyThreadState_Ge 2.45 (guido 02-Aug-97): PyObject *sd = tstate->interp->sysdict; 2.57 (guido 05-Oct-99): if (sd == NULL) 2.41 (guido 05-May-97): PyThreadState *tstate; 2.41 (guido 05-May-97): tstate = PyThreadState_Get(); 2.41 (guido 05-May-97): return Py_BuildValue( 2.41 (guido 05-May-97): "(OOO)", 2.41 (guido 05-May-97): tstate->exc_type != NULL ? tstat 2.41 (guido 05-May-97): tstate->exc_value != NULL ? tsta 2.41 (guido 05-May-97): tstate->exc_traceback != NULL ? 2.41 (guido 05-May-97): tstate->exc_traceback : 2.41 (guido 05-May-97): } 2.41 (guido 05-May-97): revision 2.41 date: 1997/05/05 20:56:11; author: guido; state: Exp; lines: +30 -9 Massive changes for separate thread state management. All per-thread globals are moved into a struct which is manipulated separately. Details of the 2.41 checkin ---------------------------- Index: sysmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/sysmodule.c,v retrieving revision 2.40 retrieving revision 2.41 diff -c -r2.40 -r2.41 *** sysmodule.c 29 Apr 1997 20:42:30 -0000 2.40 --- sysmodule.c 5 May 1997 20:56:11 -0000 2.41 *************** *** 48,56 **** #include "osdefs.h" - PyObject *_PySys_TraceFunc, *_PySys_ProfileFunc; - int _PySys_CheckInterval = 10; - #if HAVE_UNISTD_H #include <unistd.h> #endif --- 48,53 ---- *************** *** 98,103 **** --- 95,119 ---- } static PyObject * + sys_exc_info(self, args) + PyObject *self; + PyObject *args; + { + PyThreadState *tstate; + if (!PyArg_Parse(args, "")) + return NULL; + tstate = PyThreadState_Get(); + if (tstate == NULL) + Py_FatalError("sys.exc_info(): no thread state"); + return Py_BuildValue( + "(OOO)", + tstate->exc_type != NULL ? tstate->exc_type : Py_None, + tstate->exc_value != NULL ? tstate->exc_value : Py_None, + tstate->exc_traceback != NULL ? + tstate->exc_traceback : Py_None); + } + + static PyObject * sys_exit(self, args) PyObject *self; PyObject *args; *************** *** 112,123 **** PyObject *self; PyObject *args; { if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(_PySys_TraceFunc); ! _PySys_TraceFunc = args; Py_INCREF(Py_None); return Py_None; } --- 128,140 ---- PyObject *self; PyObject *args; { + PyThreadState *tstate = PyThreadState_Get(); if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(tstate->sys_tracefunc); ! tstate->sys_tracefunc = args; Py_INCREF(Py_None); return Py_None; } *************** *** 127,138 **** PyObject *self; PyObject *args; { if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(_PySys_ProfileFunc); ! _PySys_ProfileFunc = args; Py_INCREF(Py_None); return Py_None; } --- 144,156 ---- PyObject *self; PyObject *args; { + PyThreadState *tstate = PyThreadState_Get(); if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(tstate->sys_profilefunc); ! tstate->sys_profilefunc = args; Py_INCREF(Py_None); return Py_None; } *************** *** 142,148 **** PyObject *self; PyObject *args; { ! if (!PyArg_ParseTuple(args, "i", &_PySys_CheckInterval)) return NULL; Py_INCREF(Py_None); return Py_None; --- 160,167 ---- PyObject *self; PyObject *args; { ! PyThreadState *tstate = PyThreadState_Get(); ! if (!PyArg_ParseTuple(args, "i", &tstate->sys_checkinterval)) return NULL; Py_INCREF(Py_None); return Py_None; *************** *** 202,207 **** --- 221,227 ---- static PyMethodDef sys_methods[] = { /* Might as well keep this in alphabetic order */ + {"exc_info", sys_exc_info, 0}, {"exit", sys_exit, 0}, #ifdef COUNT_ALLOCS {"getcounts", sys_getcounts, 0}, *************** *** 232,238 **** if (list == NULL) return NULL; for (i = 0; _PyImport_Inittab[i].name != NULL; i++) { ! PyObject *name = PyString_FromString(_PyImport_Inittab[i].name); if (name == NULL) break; PyList_Append(list, name); --- 252,259 ---- if (list == NULL) return NULL; for (i = 0; _PyImport_Inittab[i].name != NULL; i++) { ! PyObject *name = PyString_FromString( ! _PyImport_Inittab[i].name); if (name == NULL) break; PyList_Append(list, name);