Hi everyone.<br><br>I'm trying to use Boost.Python to expose some of my C++ classes and I encountered an odd behaviour of the to_python_indirect result converter for intrusive_ptr with specific classes.<br>Here is the deal:<br>
<br>I'm using a component-based design for my classes, meaning an aggregation of objects with different features. These objects are ref-counted, and if they are in the same aggregation, they share the same lifetime.<br>
That is to say, as long as one of the components in the "ring" has a strictly positive reference counter, every component in the ring survive even if their ref-count drops to zero.<br>I use boost::intrusive_ptr to manipulate them easily.<br>
<br>There is the basic interface:<br><br>class Component<br>{<br> void AddRef();<br> void Release();<br> unsigned int GetRefCount();<br> bool AddComponent(Component*);<br> bool RemoveComponent(Component*);<br>
Component* ComponentIterator(Component*); //-->Used to parse the ring.<br> <br> virtual some_virtual_fn();<br>};<br><br>To avoid lifetime problems when manipulating an object both from C++ and python side, i want python to manipulate components only through boost::intrusive_ptr. <br>
<br>There is the exposition code:<br>
<br>//-->Object wrapper<br>struct Component_wrapper : SandBoxProj::Component, bp::wrapper< SandBoxProj::Component > {<br><br> Component_wrapper( ) :Component( ), bp::wrapper< SandBoxProj::Component >(){}<br>
<br>
virtual some_virtual_fn() {<br> //overriding code...<br> }<br> <br> default_some_virtual_fn() {<br> Component::some_virtual_fn();<br> }<br>};<br><br>//-->Some declarations that boost.python needs to manipulate boost::intrusive_ptr<br>
namespace boost { namespace python {<br><br> template <class T> struct pointee< intrusive_ptr<T> > //-->pointee struct for intrusive_ptr holder<br> {<br> typedef T type;<br> };<br><br>
struct make_owning_intrusive_ptr_holder //-->ResultConverter to make an intrusive_ptr from a raw one, inspired from to_python_indirect.hpp:79<br> {<br> template <class T><br> static PyObject* execute(T* p)<br>
{<br> typedef objects::pointer_holder<boost::intrusive_ptr<T>, T> holder_t;<br> <br> boost::intrusive_ptr<T> ptr(p);<br> return boost::python::objects::make_ptr_instance<T, holder_t>::execute(ptr);<br>
}<br> };<br> <br> struct return_by_intrusive_ptr //-->corresponding ResultConverterGenerator<br> {<br> template <class T><br> struct apply<br> {<br> typedef to_python_indirect<T, make_owning_intrusive_ptr_holder> type; //--> here is the usage of to_python_indirect<br>
};<br> };<br><br>}}<br><br>bp::class_< Component_wrapper,boost::intrusive_ptr<Component_wrapper>,boost::noncopyable >( "Component", bp::init< >() ) <br> .def( "AddComponent", &Component::AddComponent, ( bp::arg("arg0") ) ) <br>
.def( "ComponentIterator",&Component::ComponentIterator, ( bp::arg("arg0") ), bp::return_value_policy< bp::return_by_intrusive_ptr >() ) //-->this class uses the converter.<br> .def( "some_virtual_fn",&Component::some_virtual_fn,&Component_wrapper::default_some_virtual_fn)<br>
.def( "GetRefCount",&Component::GetRefCount) <br> .def( "RemoveComponent",&Component::RemoveComponent, ( bp::arg("arg0") ) ) ;<br> <br><br>And finally there is a scripts that uses this class and its output:<br>
<br>>>>from pythonbinder import *<br>>>>import gc<br><br>>>>Comp1=Component()<br>>>>Comp2=Component()<br>>>>print Comp2.GetRefCount()<br>1 //-->Ok, I manipulate an intrusive_ptr<br>
>>>Comp2.AddComponent(Comp1)<br>>>>iter=None<br>>>>iter=Comp1.ComponentIterator(iter)<br>>>>print iter<br><pythonbinder.Component object at 0x026DA630> //-->Ok, There is my C++ object<br>
>>>iter=Comp1.ComponentIterator(iter)<br>>>>print iter<br><pythonbinder.Component object at 0x026DA600><br>>>>print iter.GetRefCount()<br>1 //-->What the ? iter is supposed to be an intrusive ptr. After a look at to_python_indirect, it seems that iter==Comp2, the python object wrapping the intrusive_ptr was just addref'ed. Ok, fair enough.<br>
<br>
>>>iter=Comp1.ComponentIterator(iter)<br>>>>print iter<br>None<br> //-->Now the troubles begin.<br>>>>del Comp2<br>>>>gc.collect() //-->ensure deletion, Comp2 is still alive from the C++ side, because Comp1 is alive.<br>
>>>iter=Comp1.ComponentIterator(iter) //May crash<br>>>>print iter<br> //-->if we made it through, a weak_ptr to some random structure.<br><br>After that, i figured out that i never went through the ResultConverters i declared. Let's look at the execute function from to_python_indirect, <br>
<br> template <class U><br> inline PyObject* execute(U const& x, mpl::false_) const<br> {<br> U* const p = &const_cast<U&>(x);<br> if (is_polymorphic<U>::value) //-->My type is polymorphic<br>
{<br> if (PyObject* o = detail::wrapper_base_::owner(p)) //-->Ok, that's why I have a refcount of 1, no matter how many python intrusive_ptr wrappers I seem to have<br> return incref(o);<br>
}<br> return MakeHolder::execute(p); //-->There is the call to my result converter.<br> }<br><br>What happen is that after deletion of the last python wrapping of the intrusive_ptr, which is o in the code above, the Component_wrapper, which is partly a python object, is not deleted (the component ring still holds it), but it still reference its last owner which is the last instance_holder that just got deleted, so the object returned is a random memory garbage.<br>
<br>Changing the holder type fromintrusive_ptr<Component_wrapper> to intrusive_ptr<Component> solves the above problems but then it's not possible to use the class Component_wrapper as a base in python anymore.<br>
I tried to create a class derived from Component in python and the constructor of Component_wrapper was not called. (purpose is overriding virtual functions in python)<br>
<br>I wondered if I'm going the wrong way, or if I discovered a bug. <br>It may be a limitation of the library, which cannot understand that an object survived the deletion of its last holder.<br><br>Maybe if i made my own version of to_python_indirect but removing the is_polymorphic case, it would work?<br>
Is there a fundamental reason to this test, other than saving memory?<br>
<br><br>