[Python-Dev] Last chance!

[Christian Tismer]

Guido van Rossum wrote:

...

> Could you dig through CVS logs to find out when f_tstate was first
> introduced?  Maybe there's a clue about why there.

Yeehaa! Quite some ride...

Well, f_tstate was introduced here, in frameobject.c:
-----------------------------------------------------
Revision : 2.29
Date : 1997/5/5 20:55:52
Author : 'guido'
State : 'Exp'
Lines : +13 -2
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

For frameobject.h, it was this version:
---------------------------------------
Revision : 2.23
Date : 1997/5/5 20:55:43
Author : 'guido'
State : 'Exp'
Lines : +3 -1
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

Especially interesting:
There is no single hint mentioning f_tstate.
But, the diff shows

	PyObject *f_exc_type, *f_exc_value, *f_exc_traceback;
	PyThreadState *f_tstate;

added at the same time.

This was Python 1.5, I remember very well, the time
where threading became the standard for Python.

ceval.c was at this revision:
-----------------------------
Revision : 2.118
Date : 1997/5/5 20:55:58
Author : 'guido'
State : 'Exp'
Lines : +263 -151
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

I fetched ceval 2.117 and 2.118 and compared them.
This was exactly the time when the thread state was introduced.
There were exactly two references to f_tstate:

The most interesting diff for this question:

  PyObject *
  PyEval_SaveThread()
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
-		PyObject *res;
-		res = (PyObject *)current_frame;
-		current_frame = NULL;
+		PyThreadState *tstate = PyThreadState_Swap(NULL);
+		PyObject *res = tstate ? (PyObject *) (tstate->frame) : NULL;
  		release_lock(interpreter_lock);
  		return res;
  	}
  #endif
  	return NULL;
  }

  void
  PyEval_RestoreThread(x)
  	PyObject *x;
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
  		int err;
  		err = errno;
  		acquire_lock(interpreter_lock, 1);
  		errno = err;
-		current_frame = (PyFrameObject *)x;
+		PyThreadState_Swap(x ? ((PyFrameObject *)x)->f_tstate : NULL);
  	}
  #endif
  }

PyEval_SaveThread has changed a bit meanwhile,
no longer return the frame, but tstate.
semantics are similar.

The two functions clearly show hwo the transition from
a global current_frame variable was made to the
tstate mechanism. f_tstate was the source for switching
the thread state.

The current ceval version looks like this:

void
PyEval_RestoreThread(PyThreadState *tstate)
{
	if (tstate == NULL)
		Py_FatalError("PyEval_RestoreThread: NULL tstate");
#ifdef WITH_THREAD
	if (interpreter_lock) {
		int err = errno;
		PyThread_acquire_lock(interpreter_lock, 1);
		errno = err;
	}
#endif
	PyThreadState_Swap(tstate);
}

That means, tstate is not fetched from the frame, but
given as a parameter.
I checked all places where this function gets called,
the tstate parameter never comes from a frame.

As a conclusion, f_tstate appears to be an intermediate
step during the move to threading. It was an easy place
to keep the thread state. Its use was reduced to
very few later on.

call_trace introduced today's slightly false behavior
exactly in that 2.118 version. It looks very much like
an optimization, since there was no macro for fetching
tstate at that time.
This was perfectly fine, until generators were introduced.
Until then, a frame was always bound to its thread.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/