[C++-sig] GIL problems during destruction of a boost::python::wrapper

Niall Douglas s_sourceforge at nedprod.com
Mon May 7 16:41:46 CEST 2012

Only Python may delete a python managed object. You cannot destroy 
any python object explicitly from the C++ side of things.

Solution: allow the python wrapper and C++ instance being wrapped to 
detach from one another. That way if the C++ object is deleted, you 
zombify the python representation of it, and if python refcounts to 
zero the python side, it can be destroyed and can mark the C++ object 
as no longer visible to the python side.

I think BPL used to have explicit special support for auto-detaching 
if you wrap an auto_ptr<> to a C++ object instance? You still need 
some support code your end of course. I remember using a virtual 
factory function taking and returning a void *, you could use this to 
make any arbitrary wrapped C++ object "appear" in python.


On 6 May 2012 at 15:01, Adam Preble wrote:

> I'm running into my old friend the GIL when it comes time to destroy an
> object I've wrapped using boost::python::wrapper.  The object is being
> bandied around as a shared pointer, and it's refcount is ending at the time
> I want.  Unfortunately, I get the old "PyThreadState_Get: no current
> thread" error.  This wrapped class has a lot of virtual functions that
> Python-derived classes have to implement.  In the wrapper my methods fight
> for the GIL like this:
> void TheWrapper::Stuff(blablabla)
> {
> PyGILState_STATE gstate = PyGILState_Ensure();
> this->get_override("Stuff")( blablabla );
> PyGILState_Release(gstate);
> }
> This works fine.  I am assuming I need to write something like this for the
> destructor, but I don't see anything about doing that for the destructor.
> This might be the wrong approach, so maybe I'll give a little more
> information:
> 1. The object was created in Python.
> 2. It implements an interface wrapped from the C++ side (the wrapper as
> mentioned above).
> 3. It gives the object to a container on the C++ side.
> 4. The C++ side has figured out it doesn't need it anymore.
> 5. This is happening in a different thread from where the object was first
> made.
> 6. Destruction is happening when the removal code goes out of scope.
> 7. Once the shared_ptr free code starts, the stack descends into the Python
> runtime, and there's nothing else on the C++ side coming back up that could
> compete with the GIL.  So this is why I believe it is related to the
> destructor.
> I have pondered acquiring the GIL after explicitly resetting the
> shared_ptr, but not all the implementations of that object are actually
> implemented in Python, so that would be something of a hatchet job.  I'd
> also rather--if I could help it--encapsulate the GIL acquisition stuff in
> the wrapper with everything else, rather than remembering to cooperate with
> the GIL in choice places sprinkled around my code.  Any ideas?

Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/

More information about the Cplusplus-sig mailing list