[C++-sig] Managing the GIL across competing threads

[Niall Douglas]

On 18 Mar 2012 at 12:05, Stefan Seefeld wrote:

> > We didn't go into a lot of depth on the threading, I'm afraid, as one
> > of the problems is that the guy starting the effort - me - doesn't
> > actually know much about threaded programming.  But I am hoping that I
> > can design things in such a way that someone like Niall could easily
> > take it from there.
> 
> I recall seeing a discussion of locking policy being attached to
> individual functions by means of the return-value and argument-passing
> policy traits; in other words, something that's associated with
> from-python and to-python conversion. I found that rather elegant. I'm
> not sure whether anyone has any practical experience with that
> technique, or whether it was just an idea worth exploring.

Indeed, I had argued for a DLL and SO aware type registry to hold 
default BPL<=>Python enter/exit policy (of which locking is an 
example) which can be optionally overridden at the point of call. It 
was from this and other reasons I had argued for a fused compile 
time/runtime registry, and it was at that point that Dave came in to 
argue against such a design for a series of good reasons.

I actually don't think myself and Dave disagreed, rather as usual he 
sees the world one way and I see the world another, and we use 
different vocabulary so we both think we're arguing when often we 
agree. That said, I would be at pains to think twice or thrice about 
an idea if Dave queries it, not least because it encourages me to 
articulate myself much more clearly. He's also generally correct, or 
has had an insight I have missed.

Actually, on the basis of that previous discussion I have raised 
within ISO standards the point that we really ought to do something 
about standardising shared libraries in ISO WG14. It's long overdue 
and every attempt has failed so far. Yet as things stand, BPL is 
simply broken in a multi-extension use context and there is no 
standards-compliant way of fixing it. As the C language is the 
primary interop for all other programming languages, and POSIX's 
approach to shared libraries is broken, I fear we might just have to 
go stomp on some eggshells to get this fixed.

That said, if done right we could seriously improve interop for all 
languages. Imagine a standardised way of talking to Haskell from 
Python via a modern C interop specification? Now that would be cool!

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/