[Python-Dev] pystate.c changes for Python 2.4.2

[Michael Hudson]

Gabriel Becedillas <gabriel.becedillas at corest.com> writes:

> Hi,
> At the company I work for, we've embedded Python in C++ application we 
> develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting 
> Py_FatalError("Invalid thread state for this thread") when using debug 
> builds.
> We use both multiple interpreters and thread states.

I know you've said this to me in email already, but I have no idea how
you managed to get this to work with 2.4.1 :)

> I think the problem is that PyThreadState_Delete (which is used when I
> destroy thread states and interpreters) is not making the same clean up
> that PyThreadState_DeleteCurrent is doing (which is used when we create 
> threads from python code). If a thread id is reused (obviously not 
> between two running threads), and the previous thread used 
> PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old 
> and invalid value for autoTLSkey is still stored, and that is
> triggering the Py_FatalError when a call to PyThreadState_Swap is made.
> If I add this code at the end of PyThreadState_Delete:
>
> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
> 	PyThread_delete_key_value(autoTLSkey);
>
> then everything works fine.

I think I begin to understand the problem... but this is still
fragile: it relies on the fact that you delete a thread state from the
OS level thread that created it, but while a thread belonging to a
different *interpreter state* has the GIL (or at least: the
interpreter state of the thread state being deleted *doesn't* hold the
GIL).  Can you guarantee that?

It seems to me that to be robust, you need a way of saying "delete the
thread local value of autoTLSKey from thread $FOO".  But this is all
very confusing...

> Could you please confirm if this is a bug ?

Yes, I think it's a bug.

Cheers,
mwh

-- 
  <etrepum> Jokes around here tend to get followed by implementations.
                                                -- from Twisted.Quotes