Problem with embedded python

[ugodiggi]

Tom Cocagne wrote:
> From looking at your example, it looks like you're making the problem
FAR
> more difficult than it needs to be. The main thing to keep in mind is
that
> Python threads do not correspond to operating system threads. In an
> application using a single OS-level thread, you can use as many
Python
> threads as you like. The thread-switching mechanism in Python is to
> transfer control between threads after the execution of 100
bytecodes. The
> GIL is used to insure that only 1 operating system thread is
executing
> bytecodes at any given time. This allows os threads to sleep on
blocking
> calls (usually I/O) without necessarily stalling the interpreter.
>
>  Your example appears to be written under the assumption that the
spawned
> Python thread will operate concurrently with your main thread. That
will
> not be the case. The PyRun_SimpleString() call will operate entirely
within
> the main thread and will not return until the Python thread has
completed
> operation (so no GIL manipulation is needed). The subsequent Sleep()
call
> will simply halt the process for 300ms.
>
>  Regular threading is difficult enough to get right without throwing
the
> Python-thread / OS-thread interaction into the mix. It's a tough nut
to
> crack so don't feel bad if it takes a while to get everything
straight ;-)
> The quickest and most practical approach would probably be to read
through
> some code that's similar to what you're shooting for. I'd suggest
taking a
> look at some of the modules in the Python source code (particularly
the
> network related ones) to get a better handle on how threads are
managed.
>
>  Cheers,
>
>  Cocagne
>
<removed code>

Tom,

Thank you for your answer.

This code is not the real "problem child", but a slimmed down version
that seems to reproduce a problem that I see in a much larger system.

My understanding of the python GIL works as follows:
1) when I start the embedded python interpreter (Py_Initialize,
PyEval_InitThreads), I own the GIL.
So I need to release it for other threads to be able to do anything.
I don't have other threads, so nothing is happening.

2) when I want to use PyRun_SimpleString, I need to acquire the GIL
before, and release it after. By doing this, no python thread will run
while I'm running my simple string, regardless of anything because the
C is owning the GIL.

3) after I release the GIL, all the python threads are actually free to
run, and they actually will. Since I'm on WXP, python threads ARE
actually os threads, as I can see from my debugger.
I do get my nice pippo.out file with the numbers from 0 to 29 or 30,
which is coherent with the sleeps.

4) when I want to shut down the whole thing, I acquire the GIL and then
I call Py_Finalize, which should pretty much kill all the threads
cleanly.

The crash that I'm getting is during step 4, in the python thread,
while in the main thread I'm calling Py_Finalize.
Maybe I'm doing something wrong in one of the steps, but I cannot see
what.

My best guess is that the words from the ref manual:
"The lock is also released and reacquired around potentially blocking
I/O operations like reading or writing a file, so that other threads
can run while the thread that requests the I/O is waiting for the I/O
operation to complete. "
are haunting me, but I'm not sure how.

Thanks again

Cheers & ciao

Ugo