Use of Py_BEGIN_ALLOW_THREADS?

[Aahz Maruch]

In article <3AF6CE77.47FC4228 at rfa.org>, Bill Eldridge  <bill at rfa.org> wrote:
>Aahz:
>>In article <3AD42A5F.73AE10EC at rfa.org>, Bill Eldridge  <bill at rfa.org> >wrote:
>>>
>>>I'm calling a C function from Python, and that C function in turn calls
>>>another C function #2 with a C callback function.
>>>
>>>Somewhere it seems like with threads it disappears - all of the debug
>>>messages in function #2 come out, but the C callback never gets called.
>>
>>So you're saying that this works correctly if you don't use threads?
>
>I'm saying it works correctly if I have a C main program
>instead of a Python main, with the C being as simple as:
>
>void main() {
>   call_audio_stuff();
>}

Okay, but that's not testing with threads.

>I didn't try using the Py_BEGIN_ALLOW_THREADS until
>my initial try with a Python main program didn't work.

Py_BEGIN_ALLOW_THREADS should only, repeat *ONLY*, be used if you're
writing a C extension where it is safe to have multiple Python threads
concurrently *executing* in that block of code.  Even with that, unless
you're a thread wizard, your Python code should never share instances of
the objects you're creating (e.g. never share a file object across
threads).

>>What happens if you have multiple threads, but only one thread calls the
>>original C function?  My suspicion is that you're getting cross-thread
>>contamination in your C code because it isn't thread-safe.  
>
>I can try doing the call_audio_stuff as an individual thread
>from Python...

Do it.

>>Note that you should *ONLY* use Py_BEGIN_ALLOW_THREADS if your code is
>>"thread hot"; that is, not just thread-safe, but designed to allow
>>multiple threads into the same code at the same time.  In the absence of
>>Py_BEGIN_ALLOW_THREADS, your C code is considered to be one Python
>>bytecode and executes as a single unit.
>
>How exactly do I make it "thread hot"?

Essentially, in a thread-hot code block, there is zero static storage;
everything is purely local on the stack or unique malloc'd data.

>I'm not specifically sure why I need the threads at all,
>as some have indicated to me.  Again, the thing that's
>failing follows:

Well, I don't know why you're using threads, either, but you're
definitely doing something screwy if this works unthreaded and fails
threaded.
-- 
                      --- Aahz  <*>  (Copyright 2001 by aahz at pobox.com)

Androgynous poly kinky vanilla queer het Pythonista   http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6

"Everyone is entitled to an *informed* opinion."  --Harlan Ellison