[C++-sig] Re: Patch to add thread support to invoke function.

[David Abrahams]

"Lijun Qin" <qinlj at solidshare.com> writes:

> Hi,
>
> I have made some changes to the invoke.hpp file, to add
>
> Py_UNBLOCK_THREADS
>  result = f(...);
> Py_BLOCK_THREADS
>
> pair, this is need for too reasons:
> First, if the f(...) is a lengthy operation, it'll block all Python code to
> execute, this is not good,
> second, if f(...) function call some functions that will acquire the python
> global lock, it'll deadlock, one example is the COM inproc server using
> Pythoncom22.dll, every COM interface method will first call
> PyEval_AcquireThread, but the global lock has been held, so deadlock occur.

Hi Lijun,

I'm not ignoring your patches.  Here's my reservation about accepting
this one, though: I don't think it represents a complete solution to
threading support in Boost.Python, and I'd like to solve the problem
all at once.  On way in which this doesn't handle the problem is that
if f takes any object, str, dict, handle<>, etc. parameters by value,
the call may crash the interpreter as reference counts are changed
with threading unblocked.  My guess is that the right solution is to
simply not unblock threads for those functions, since they are
hand-written with Boost.Python in mind and the author can unblock
threads manually if neccessary (with a "unguard" object).  Another way
in which your patch is inadequate is that any code in Py_BLOCK_THREADS
will get skipped if f throws an exception, so we really need guard
objects which use RAII.  Another way is that virtual functions which
call back into Python with call_method need to re-acquire the GIL
somehow -- it's not yet clear to me whether that should happen inside
or outside call_method.

The following are references to some threads in which these issues
have been discussed, should you wish to explore further:

http://aspn.activestate.com/ASPN/Mail/Message/1465959
http://aspn.activestate.com/ASPN/Mail/Message/1603259
http://aspn.activestate.com/ASPN/Mail/Message/1607050
http://aspn.activestate.com/ASPN/Mail/Message/1706806

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com