[C++-sig] Re: Patch to add thread support to invoke function.

[Lijun Qin]

Hi,

I think a better solution is to give the user choice, I mean use a flag
which can be passed when calling 'def', such as
.def('f', f, enableThreadLock)
Currently I'm using boost.python to wrap WTL, and it has hundreds of
functions which are simple SendMessage wrappers, and many of them (I think
half of them, almost all 'write' operation) can cause callback into the
python code again, use the notify message such as WM_COMMAND, WM_NOTIFY. So
write wrapper for every one of them is not acceptable.
The callback will be implemented use call_method<> normally, but the thread
acquiring will not be done is call_method, it'll normally be done earlier in
the very point the control transferred from the 'outside world' to the
python extension module, in the Windows platform, this will normally at the
beginning of COM interface method, Window Procedure, or DLL exported
function (this is rare I think).
If we can implement the 'enableThreadLock' flag, but make it a 'false'
default, it'll be harmless to add this thread support into boost.python
code, but when needed, it'll be very convenience to have them.

I do agree that there should be a try catch block here, like:

Py_UNBLOCK_THREADS
try {
    result = f(...);
} catch(...)
    Py_BLOCK_THREADS
    throw;
}
Py_BLOCK_THREADS
...

In my case, because SendMessage and COM method will never throw an
exception, so I ignored them in the last patch, for a complete solution,
it'll be better to do this.

Lijun Qin
http://www.solidshare.com


----- Original Message ----- 
From: "David Abrahams" <dave at boost-consulting.com>
To: <c++-sig at python.org>
Cc: "Lijun Qin" <qinlj at solidshare.com>
Sent: Tuesday, September 02, 2003 9:53 PM
Subject: [C++-sig] Re: Patch to add thread support to invoke function.


> "Lijun Qin" <qinlj at solidshare.com> writes:
>
> > Hi,
> >
> > I have made some changes to the invoke.hpp file, to add
> >
> > Py_UNBLOCK_THREADS
> >  result = f(...);
> > Py_BLOCK_THREADS
> >
> > pair, this is need for too reasons:
> > First, if the f(...) is a lengthy operation, it'll block all Python code
to
> > execute, this is not good,
> > second, if f(...) function call some functions that will acquire the
python
> > global lock, it'll deadlock, one example is the COM inproc server using
> > Pythoncom22.dll, every COM interface method will first call
> > PyEval_AcquireThread, but the global lock has been held, so deadlock
occur.
>
> Hi Lijun,
>
> I'm not ignoring your patches.  Here's my reservation about accepting
> this one, though: I don't think it represents a complete solution to
> threading support in Boost.Python, and I'd like to solve the problem
> all at once.  On way in which this doesn't handle the problem is that
> if f takes any object, str, dict, handle<>, etc. parameters by value,
> the call may crash the interpreter as reference counts are changed
> with threading unblocked.  My guess is that the right solution is to
> simply not unblock threads for those functions, since they are
> hand-written with Boost.Python in mind and the author can unblock
> threads manually if neccessary (with a "unguard" object).  Another way
> in which your patch is inadequate is that any code in Py_BLOCK_THREADS
> will get skipped if f throws an exception, so we really need guard
> objects which use RAII.  Another way is that virtual functions which
> call back into Python with call_method need to re-acquire the GIL
> somehow -- it's not yet clear to me whether that should happen inside
> or outside call_method.
>
> The following are references to some threads in which these issues
> have been discussed, should you wish to explore further:
>
> http://aspn.activestate.com/ASPN/Mail/Message/1465959
> http://aspn.activestate.com/ASPN/Mail/Message/1603259
> http://aspn.activestate.com/ASPN/Mail/Message/1607050
> http://aspn.activestate.com/ASPN/Mail/Message/1706806
>
> -- 
> Dave Abrahams
> Boost Consulting
> www.boost-consulting.com
>
>
> _______________________________________________
> C++-sig mailing list
> C++-sig at python.org
> http://mail.python.org/mailman/listinfo/c++-sig
>