Understanding PyEval_InitThreads

[Thomas Heller]

"Donn Cave" <donn at drizzle.com> writes:

> Quoth hansgeunsmeyer at earthlink.net (hg):
> ...
> | As far as I understand it, it doesn't really matter _what_ the thread
> | state is that you swap into the current state before you do the
> | callback, as long as it is _some_ valid thread state (not NULL). So,
> | before you do the callback, at any moment when you're in the main
> | Python thread, you could store a pointer to the then current
> | threadstate.
> 
> Why are there multiple thread states, then?  If we might as well just
> use the existing state of any thread, it seems like we are put to a
> lot of trouble to hunt one up.

The thread state is, well, the state of a thread. So at least they
cannot be shared across threads.

I had multiple other schemes saving thread states when calling out to
C from Python, and getting thread states when calling back into Python
from C, including stacks of thread states in thread local storage.
Some of them worked, some were fragile.

The current scheme (saving it in a local var when calling 'out' and
creating a new one when calling 'in') works well, also for
multithreaded programs. It's also the one used (I believe) in Mark's
win32all.

There are some drawbacks with this approach as far as I can see: First
a performance hit, because thread states are frequently created and
destroyed again.

Second, profiling and setting breakpoints won't work across callouts
and callins because the profiling and tracing is done with objects
stored in the thread state itself, and creating a new one doesn't
inherit them from the saved state. This also affects exceptions, which
are not propagated back into the 'callout' from the 'callin'.

Hm, can anyone understand the last sentence?
What I mean is: When an exception occurs in the callback (which runs
with a newly created thread state), simply PyErr_Print() is called
before the thread state is deleted, because there's no way to pass
the exception somewhere else.

Thomas