weak_ptr to this in C++ classes extended from python
Dear list, how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself? The reason why I want to do this: - using a weak_ptr to itself does not introduce a memory leak. - from a C++-member of that class I can call other functions that expect a shared_ptr<>, in that case I convert the weak_ptr to a shared_ptr and all the memory management works nicely. The way I would suspect this to work is: struct I { void setThisRCP(boost::shared_ptr<I> ptr) { thisRCP = ptr; } boost::shared_ptr<I> getThisRCP() const { return thisRCP.lock(); } boost::weak_ptr<I> thisRCP; }; Then I export this class to C++ and in python I would do: class IDer(I): def __init__(self): I.__init__(self) self.setThisRCP(self) And I would suspect that I can use that class now in C++ and in particular calling getThisRCP() from C++ returns a valid shared pointer. However, immediately after the call of `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP returns an shared pointer to Null (None in python). Attached to this message is a very simple example that demonstrates how this failes for the above implementation, the ouptu for the snipped: print '\ntestcase for thisRCP as from python extended class:' i = IDer() assert i.getThisRCP() is not None fails the assertion. However if I do the equivalent to IDer purely in C++ (which is not really an option) then everything works (see attachment) with the putput testcase for thisRCP as set from C++: 42 destructor for J called ... I suspect that the error comes from the fact that boost python create as temporary shared_ptr that is passes to setThisRCP() which is not the shared pointer that actually holds the instance of IDer in python (is it actually hold as a shared_ptr?). Do you have any proposal of how I can implement the above pattern with boost python? Or is this completly impossible at the moment? I also reported this issue before "storing weak_ptr to this in classes extended from python" but at that time I wasn't using shared_ptr and now I really suffer from the memory leak that was introduced by my workaround which was to store the thisRCP as a "shared_ptr" and I really need to fix that now. So here is a small little example that does not depend on any other parts of my library. -Holger
On 04/22/2012 03:42 PM, Holger Brandsmeier wrote:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
The reason why I want to do this: - using a weak_ptr to itself does not introduce a memory leak. - from a C++-member of that class I can call other functions that expect a shared_ptr<>, in that case I convert the weak_ptr to a shared_ptr and all the memory management works nicely.
The way I would suspect this to work is: struct I { void setThisRCP(boost::shared_ptr<I> ptr) { thisRCP = ptr; }
boost::shared_ptr<I> getThisRCP() const { return thisRCP.lock(); }
boost::weak_ptr<I> thisRCP; };
Then I export this class to C++ and in python I would do: class IDer(I): def __init__(self): I.__init__(self) self.setThisRCP(self)
And I would suspect that I can use that class now in C++ and in particular calling getThisRCP() from C++ returns a valid shared pointer. However, immediately after the call of `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP returns an shared pointer to Null (None in python).
Attached to this message is a very simple example that demonstrates how this failes for the above implementation, the ouptu for the snipped:
print '\ntestcase for thisRCP as from python extended class:' i = IDer() assert i.getThisRCP() is not None
fails the assertion. However if I do the equivalent to IDer purely in C++ (which is not really an option) then everything works (see attachment) with the putput testcase for thisRCP as set from C++: 42 destructor for J called ...
I suspect that the error comes from the fact that boost python create as temporary shared_ptr that is passes to setThisRCP() which is not the shared pointer that actually holds the instance of IDer in python (is it actually hold as a shared_ptr?).
Do you have any proposal of how I can implement the above pattern with boost python? Or is this completly impossible at the moment?
I agree with your suspicion about what's going on here, though I haven't (yet) tested your example. One quick thing to try would be having setThisRCP() take a shared_ptr by reference - that should force Boost.Python to try to return the internal shared_ptr rather than create a temporary one. Even if it fails, you may learn something about what's going on.
I also reported this issue before "storing weak_ptr to this in classes extended from python" but at that time I wasn't using shared_ptr and now I really suffer from the memory leak that was introduced by my workaround which was to store the thisRCP as a "shared_ptr" and I really need to fix that now. So here is a small little example that does not depend on any other parts of my library.
Thanks for the small example. Hopefully we can get to the bottom of this; you don't seem to be the only person who wants to do something like this. Jim
on Sun Apr 22 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Easy, I think: just like any other class you wrap. Are you aware of boost::enable_shared_from_this? It does all of that internally.
The reason why I want to do this: - using a weak_ptr to itself does not introduce a memory leak. - from a C++-member of that class I can call other functions that expect a shared_ptr<>, in that case I convert the weak_ptr to a shared_ptr and all the memory management works nicely.
The way I would suspect this to work is: struct I { void setThisRCP(boost::shared_ptr<I> ptr) { thisRCP = ptr; }
boost::shared_ptr<I> getThisRCP() const { return thisRCP.lock(); }
boost::weak_ptr<I> thisRCP; };
Then I export this class to C++ and in python I would do: class IDer(I): def __init__(self): I.__init__(self) self.setThisRCP(self)
And I would suspect that I can use that class now in C++ and in particular calling getThisRCP() from C++ returns a valid shared pointer. However, immediately after the call of `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP returns an shared pointer to Null (None in python).
If you use boost::shared_ptr<I> as the Holder for your wrapper, you'll be able to do that conversion. I see you've done that with "J" in your example, but not with "I". Why not? If you use enable_shared_from_this, the internal weak_ptr will be initialized from the first shared_ptr that is constructed to manage the object. Everything should work. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
("I", init<>() ) didn't change anything, the getThisRCP() still doesn't work. In fact I
("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used
Dear Dave, dear Jim, thanks for your help in particular to `boost::enable_shared_from_this` which I wasn't aware about in the beginning. Unfortunately your solutions did not yet work. In the meantime I figured out a different way of doing the shared pointer from this in python. Namely, to just make getThisRCP() a virtual function, add it to the Wrapper and to implement it in the python by implementing it as def getThisRCP(self): return self this simply works. As for the issue that I reported, it still persists so out of interest I did try your suggestions and here is what I found out. the difference between the lines class_<IWrapper, boost::noncopyable >("I", init<>() ) and class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable tried both variants before and send you just the first one. Can it be that specifying the holder for IWrapper does not apply to the holder for the subclass IDer from python? In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable this for the Wrapper and not the class itself (see K and KWrapper in the attachment). The behaviour of `enable_shared_from_this` is different from my initual getThisRCP() / setThisRCP() solution but still has seriour problems. Concider the code: class KDer(K): def __init__(self): K.__init__(self) k = KDer() In contrast to my initial solution the lines assert k.getThisRCP2() is not None k2 = k.getThisRCP2() now work, which is nice. Also this call works now: print k2.foo(); However, the following two lines del k print k2.foo(); give a segmentation fault. Note that the destructor for the class `K` has not been called, which would have printed a statement. The `gdb` backtrace is #0 0x00007ffff64425d7 in boost::python::objects::instance_dealloc (inst=0xe51788) at libs/python/src/object/class.cpp:335 #1 0x00007ffff7aba7f6 in subtype_dealloc (self=0xe51788) at Objects/typeobject.c:1014 #2 0x00007ffff7a72413 in instancemethod_dealloc (im=0xae0730) at Objects/classobject.c:2364 #3 0x00007ffff645387e in xdecref<_object> (p=<optimized out>) at ./boost/python/refcount.hpp:36 #4 ~handle (this=<optimized out>, __in_chrg=<optimized out>) at ./boost/python/handle.hpp:211 #5 boost::python::detail::wrapper_base::get_override (this=0xaec988, name=0x7fffe41de992 "foo", class_object=0x71bb10) at libs/python/src/wrapper.cpp:21 #6 0x00007fffe41d95f1 in get_override (name=<optimized out>, this=<optimized out>, name=<optimized out>) at /usr/include/boost/python/wrapper.hpp:29 #7 KWrapper::foo (this=0xaec980) at /home/bholger/HolgerBrandsmeier/parfem.h/python/thisRCPPy.cpp:112 [...] Any ideas? -Holger On Mon, Apr 23, 2012 at 18:01, Dave Abrahams <dave@boostpro.com> wrote:
on Sun Apr 22 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Easy, I think: just like any other class you wrap. Are you aware of boost::enable_shared_from_this? It does all of that internally.
The reason why I want to do this: - using a weak_ptr to itself does not introduce a memory leak. - from a C++-member of that class I can call other functions that expect a shared_ptr<>, in that case I convert the weak_ptr to a shared_ptr and all the memory management works nicely.
The way I would suspect this to work is: struct I { void setThisRCP(boost::shared_ptr<I> ptr) { thisRCP = ptr; }
boost::shared_ptr<I> getThisRCP() const { return thisRCP.lock(); }
boost::weak_ptr<I> thisRCP; };
Then I export this class to C++ and in python I would do: class IDer(I): def __init__(self): I.__init__(self) self.setThisRCP(self)
And I would suspect that I can use that class now in C++ and in particular calling getThisRCP() from C++ returns a valid shared pointer. However, immediately after the call of `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP returns an shared pointer to Null (None in python).
If you use boost::shared_ptr<I> as the Holder for your wrapper, you'll be able to do that conversion. I see you've done that with "J" in your example, but not with "I". Why not?
If you use enable_shared_from_this, the internal weak_ptr will be initialized from the first shared_ptr that is constructed to manage the object. Everything should work.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used
In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable this for the Wrapper and not the class itself (see K and KWrapper in the attachment).
The behaviour of `enable_shared_from_this` is different from my initual getThisRCP() / setThisRCP() solution but still has seriour problems. Concider the code:
class KDer(K): def __init__(self): K.__init__(self)
k = KDer()
In contrast to my initial solution the lines assert k.getThisRCP2() is not None k2 = k.getThisRCP2() now work, which is nice.
Also this call works now: print k2.foo();
However, the following two lines del k print k2.foo(); give a segmentation fault. Note that the destructor for the class `K` has not been called, which would have printed a statement.
Please post a reduced example without any of the "J" business that illustrates exactly the problem you're having for this one particular case. If doing so doesn't immediately reveal the problem to you, I'm sure we can get to the bottom of it here. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Dave, Here is the reduced example with only the `boost::enable_shared_from_this` variant. The issues are all as reported before. -Holger On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used
In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable this for the Wrapper and not the class itself (see K and KWrapper in the attachment).
The behaviour of `enable_shared_from_this` is different from my initual getThisRCP() / setThisRCP() solution but still has seriour problems. Concider the code:
class KDer(K): def __init__(self): K.__init__(self)
k = KDer()
In contrast to my initial solution the lines assert k.getThisRCP2() is not None k2 = k.getThisRCP2() now work, which is nice.
Also this call works now: print k2.foo();
However, the following two lines del k print k2.foo(); give a segmentation fault. Note that the destructor for the class `K` has not been called, which would have printed a statement.
Please post a reduced example without any of the "J" business that illustrates exactly the problem you're having for this one particular case. If doing so doesn't immediately reveal the problem to you, I'm sure we can get to the bottom of it here.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
I don't understand; all the enable_shared_from_this stuff is commented out. Can you please post the complete, correct example that you tried? on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
Dave,
Here is the reduced example with only the `boost::enable_shared_from_this` variant. The issues are all as reported before.
-Holger
On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used
In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable this for the Wrapper and not the class itself (see K and KWrapper in the attachment).
The behaviour of `enable_shared_from_this` is different from my initual getThisRCP() / setThisRCP() solution but still has seriour problems. Concider the code:
class KDer(K): def __init__(self): K.__init__(self)
k = KDer()
In contrast to my initial solution the lines assert k.getThisRCP2() is not None k2 = k.getThisRCP2() now work, which is nice.
Also this call works now: print k2.foo();
However, the following two lines del k print k2.foo(); give a segmentation fault. Note that the destructor for the class `K` has not been called, which would have printed a statement.
Please post a reduced example without any of the "J" business that illustrates exactly the problem you're having for this one particular case. If doing so doesn't immediately reveal the problem to you, I'm sure we can get to the bottom of it here.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
from thisRCPPy import *
print '\ntestcase for thisRCP as from python extended class (using enable_shared_from_this):' class KDer(K): def __init__(self): K.__init__(self)
k = KDer() assert k.getThisRCP2() is not None k2 = k.getThisRCP2() if 1: print k2.foo(); del k print 'with the `del k` statement the following statement will crash the program:'; print k2.foo(); print 'this line is never reached'; #del k del k2
#include <boost/python.hpp> #include "iostream" #include "string" #include "boost/weak_ptr.hpp" #include "boost/enable_shared_from_this.hpp"
using namespace boost::python; using namespace std; //using Teuchos::RCP;
struct K // : public boost::enable_shared_from_this<K> { virtual int foo() = 0;
virtual ~K() { std::cout << "destructor for K called ..." << std::endl; }
/*boost::shared_ptr<K> getThisRCP() { return shared_from_this(); }*/ };
struct KWrapper : public K, public boost::python::wrapper<K>, public boost::enable_shared_from_this<KWrapper> {
KWrapper() {
}
virtual int foo() { if( override f = this->get_override("foo") ) { return f(); } else { return base_foo(); } }
boost::shared_ptr<KWrapper> getThisRCP2() { return shared_from_this(); }
int base_foo() { return 0; } };
BOOST_PYTHON_MODULE(thisRCPPy) { { typedef K ClassT; class_<KWrapper, boost::shared_ptr<KWrapper>, boost::noncopyable >("K", init<>() ) //.def("getThisRCP", &ClassT::getThisRCP) .def("getThisRCP2", &KWrapper::getThisRCP2) .def("foo", &ClassT::foo) ; } } _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
Not all of it is commented out, it is enabled for the wrapper. You can try to enable it for the class itself, but then I get a different error, namely:
In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable>("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used this for the Wrapper and not the class itself (see K and KWrapper in the attachment).
-Holger On Wed, Apr 25, 2012 at 15:32, Dave Abrahams <dave@boostpro.com> wrote:
I don't understand; all the enable_shared_from_this stuff is commented out. Can you please post the complete, correct example that you tried?
on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
Dave,
Here is the reduced example with only the `boost::enable_shared_from_this` variant. The issues are all as reported before.
-Holger
On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <dave@boostpro.com> wrote:
on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
("I", init<>() ) has an issue. When I call getThisRCP() later, and the return value is not a null shared pointer, then I in fact the error TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I> which is quite annoying. I also implemented Dave's suggestion with boost::enable_shared_from_this and because of the above issue I used
In fact the statement class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable this for the Wrapper and not the class itself (see K and KWrapper in the attachment).
The behaviour of `enable_shared_from_this` is different from my initual getThisRCP() / setThisRCP() solution but still has seriour problems. Concider the code:
class KDer(K): def __init__(self): K.__init__(self)
k = KDer()
In contrast to my initial solution the lines assert k.getThisRCP2() is not None k2 = k.getThisRCP2() now work, which is nice.
Also this call works now: print k2.foo();
However, the following two lines del k print k2.foo(); give a segmentation fault. Note that the destructor for the class `K` has not been called, which would have printed a statement.
Please post a reduced example without any of the "J" business that illustrates exactly the problem you're having for this one particular case. If doing so doesn't immediately reveal the problem to you, I'm sure we can get to the bottom of it here.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
from thisRCPPy import *
print '\ntestcase for thisRCP as from python extended class (using enable_shared_from_this):' class KDer(K): def __init__(self): K.__init__(self)
k = KDer() assert k.getThisRCP2() is not None k2 = k.getThisRCP2() if 1: print k2.foo(); del k print 'with the `del k` statement the following statement will crash the program:'; print k2.foo(); print 'this line is never reached'; #del k del k2
#include <boost/python.hpp> #include "iostream" #include "string" #include "boost/weak_ptr.hpp" #include "boost/enable_shared_from_this.hpp"
using namespace boost::python; using namespace std; //using Teuchos::RCP;
struct K // : public boost::enable_shared_from_this<K> { virtual int foo() = 0;
virtual ~K() { std::cout << "destructor for K called ..." << std::endl; }
/*boost::shared_ptr<K> getThisRCP() { return shared_from_this(); }*/ };
struct KWrapper : public K, public boost::python::wrapper<K>, public boost::enable_shared_from_this<KWrapper> {
KWrapper() {
}
virtual int foo() { if( override f = this->get_override("foo") ) { return f(); } else { return base_foo(); } }
boost::shared_ptr<KWrapper> getThisRCP2() { return shared_from_this(); }
int base_foo() { return 0; } };
BOOST_PYTHON_MODULE(thisRCPPy) { { typedef K ClassT; class_<KWrapper, boost::shared_ptr<KWrapper>, boost::noncopyable >("K", init<>() ) //.def("getThisRCP", &ClassT::getThisRCP) .def("getThisRCP2", &KWrapper::getThisRCP2) .def("foo", &ClassT::foo) ; } } _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
-- Dave Abrahams BoostPro Computing http://www.boostpro.com
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Have you tried using boost::enable_shared_from_this? The following example code seems to do what you want. ------------ foo.cpp ------------ #include <boost/python.hpp> #include <boost/shared_ptr.hpp> #include <boost/enable_shared_from_this.hpp> #include <iostream> //Inheriting boost::enable_shared_from_this adds a weak_ptr to the object //That gets automatically initialized when you construct a shared_ptr //holding the object. class A : public boost::enable_shared_from_this<A> { static int counter; public: int id; A() { id = counter++; std::cout << "Constructing A[" << id << "]" << std::endl; } ~A() { std::cout << "Destructing A[" << id << "]" << std::endl; } //This function allows us to get a shared_ptr holding the object boost::shared_ptr<A> ptr() { return shared_from_this(); } }; int A::counter = 0; using namespace boost::python; class B { boost::shared_ptr<A> m_a; static int counter; public: int id; //Notice that B accepts an A object directly //And asks it for the shared_ptr which holds it. B(A& a) : m_a(a.ptr()) { id = counter++; std::cout << "Constructing B[" << id << "]"; std::cout << ", which holds A[" << m_a->id << "]" << std::endl; } ~B() { std::cout << "Destructing B[" << id << "]"; std::cout << ", which holds A[" << m_a->id << "]" << std::endl; } }; int B::counter = 0; BOOST_PYTHON_MODULE(foo) { class_<A, boost::shared_ptr<A> >("A", init<>()); class_<B>("B", init<A&>()); } ----------------------------------- ------------ test.py ------------ import foo class ExtendedA(foo.A): def special(self): return "successfully extended foo.A" a0 = foo.A() a = ExtendedA() print a.special() print "an ExtendedA object is a foo.A object: %s" % isinstance(a, foo.A) b = foo.B(a) ----------------------------------- On my system the output is: Constructing A[0] Constructing A[1] successfully extended foo.A an ExtendedA object is a foo.A object: True Constructing B[0], which holds A[1] Destructing B[0], which holds A[1] Destructing A[1] Destructing A[0]
Bryan, my problems happen as soon as I need to use wrappers. I really need the class that I extend in python to implement some pure virtual function in the C++ class. I extended your code and refined what is going wrong. I tried several Variants with different errors, I used #defines to show you all 4 versions, next to each Variant I put a C++ comment of what is going wrong. So here it is: ------------ foo.cpp ------------ #include <boost/python.hpp> #include <boost/shared_ptr.hpp> #include <boost/enable_shared_from_this.hpp> #include <iostream> using namespace boost::python; class A : public boost::enable_shared_from_this<A> { static int counter; public: int id; A() { id = counter++; std::cout << "Constructing A[" << id << "]" << std::endl; } ~A() { std::cout << "Destructing A[" << id << "]" << std::endl; } boost::shared_ptr<A> ptr() { return shared_from_this(); } #define VARIANT0 #ifdef VARIANT0 // this variant works nicely with VARIANT1 below (compiles and // no error for `aa = a.ptr()`), but it produces wrong results, // i.e. `77` is returned for `aa.foo()` although it should have // been `33`. virtual int foo() { return 77; } #else virtual int foo() = 0; #endif }; struct AWrapper : public A, public boost::python::wrapper<A> { virtual int foo() { if( override f = this->get_override("foo") ) { return f(); } else { return 42; } } }; int A::counter = 0; using namespace boost::python; BOOST_PYTHON_MODULE(foo) { #define VARIANT1 //#define VARIANT2 //#define VARIANT3 #ifdef VARIANT1 // The following line doesn't compites it gives the error: // error: cannot allocate an object of abstract type ‘A’ class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT2 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // RuntimeError: tr1::bad_weak_ptr class_<AWrapper, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT3 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<A> class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
("A", init<>()) #endif .def("ptr", &A::ptr) .def("foo", &A::foo) ; }
------------ foo.py ------------ import foo class ExtendedA(foo.A): def foo(self): return 33; a0 = foo.A() a = ExtendedA() aa = a.ptr(); print a.foo() del a print aa.foo() ----------------------------------- Do you have any suggestion how to get this working? -Holger On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <bcatanzaro@acm.org> wrote:
Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Have you tried using boost::enable_shared_from_this? The following example code seems to do what you want.
I think the problem is that aa is no longer an ExtendedA object, because when you call ptr() it returns a pointer to the base C++ A class, not the derived class. I think at this point it's probably best to reconsider the problem and come up with a simpler solution. Why do you want to expose the ptr method to Python anyway? - bryan On Thu, Apr 26, 2012 at 11:49 AM, Holger Brandsmeier <brandsmeier@gmx.de> wrote:
Bryan,
my problems happen as soon as I need to use wrappers. I really need the class that I extend in python to implement some pure virtual function in the C++ class. I extended your code and refined what is going wrong. I tried several Variants with different errors, I used #defines to show you all 4 versions, next to each Variant I put a C++ comment of what is going wrong.
So here it is:
------------ foo.cpp ------------ #include <boost/python.hpp> #include <boost/shared_ptr.hpp> #include <boost/enable_shared_from_this.hpp> #include <iostream>
using namespace boost::python;
class A : public boost::enable_shared_from_this<A> { static int counter; public: int id; A() { id = counter++; std::cout << "Constructing A[" << id << "]" << std::endl; } ~A() { std::cout << "Destructing A[" << id << "]" << std::endl; } boost::shared_ptr<A> ptr() { return shared_from_this(); } #define VARIANT0 #ifdef VARIANT0 // this variant works nicely with VARIANT1 below (compiles and // no error for `aa = a.ptr()`), but it produces wrong results, // i.e. `77` is returned for `aa.foo()` although it should have // been `33`. virtual int foo() { return 77; } #else virtual int foo() = 0; #endif };
struct AWrapper : public A, public boost::python::wrapper<A> { virtual int foo() { if( override f = this->get_override("foo") ) { return f(); } else { return 42; } } };
int A::counter = 0;
using namespace boost::python;
BOOST_PYTHON_MODULE(foo) { #define VARIANT1 //#define VARIANT2 //#define VARIANT3
#ifdef VARIANT1 // The following line doesn't compites it gives the error: // error: cannot allocate an object of abstract type ‘A’ class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT2 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // RuntimeError: tr1::bad_weak_ptr class_<AWrapper, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT3 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<A> class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
("A", init<>()) #endif .def("ptr", &A::ptr) .def("foo", &A::foo) ; }
------------ foo.py ------------ import foo
class ExtendedA(foo.A): def foo(self): return 33;
a0 = foo.A() a = ExtendedA() aa = a.ptr(); print a.foo() del a print aa.foo() -----------------------------------
Do you have any suggestion how to get this working? -Holger
On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <bcatanzaro@acm.org> wrote:
Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Have you tried using boost::enable_shared_from_this? The following example code seems to do what you want.
Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
On Fri, Apr 27, 2012 at 08:22, Bryan Catanzaro <bcatanzaro@acm.org> wrote:
I think the problem is that aa is no longer an ExtendedA object, because when you call ptr() it returns a pointer to the base C++ A class, not the derived class.
I think you got something confused. A boost::shared_pointer<A> can very well be any pointer to any derived class, no copy should be created. I can still call virtual functions from `A` and should get the result from the implementation of `ExtendedA`.
I think at this point it's probably best to reconsider the problem and come up with a simpler solution. Why do you want to expose the ptr method to Python anyway?
I don't plan to call `ptr()` from python, but I will call the same function from C++ and and at some point the result that C++ produced can find its way back to python. I did find a different solution to this problem: On Mon, Apr 23, 2012 at 20:48, Holger Brandsmeier <brandsmeier@gmx.de> wrote:
In the meantime I figured out a different way of doing the shared pointer from this in python. Namely, to just make getThisRCP() a virtual function, add it to the Wrapper and to implement it in the python by implementing it as
but I still expect the above mentioned approch with `boost::enable_shared_from_this` _should_ have worked with boost python. I seem to run into an issue that even Dave can not explain (well lets wait for his reply). Moreover note that in boost::python usually if C++ returns a `shared_ptr<A>` which has previously been created as an instance of `ExtendedA` inside python, than this class is available as `ExtendedA`. There is some magic with custom deleters that usually makes you able to treat that class as instance of `ExtendedA`. This magic will also stop working when you use `boost::enable_shared_from_this`. With that aspect I can live, though. -Holger
On Thu, Apr 26, 2012 at 11:49 AM, Holger Brandsmeier <brandsmeier@gmx.de> wrote:
Bryan,
my problems happen as soon as I need to use wrappers. I really need the class that I extend in python to implement some pure virtual function in the C++ class. I extended your code and refined what is going wrong. I tried several Variants with different errors, I used #defines to show you all 4 versions, next to each Variant I put a C++ comment of what is going wrong.
So here it is:
------------ foo.cpp ------------ #include <boost/python.hpp> #include <boost/shared_ptr.hpp> #include <boost/enable_shared_from_this.hpp> #include <iostream>
using namespace boost::python;
class A : public boost::enable_shared_from_this<A> { static int counter; public: int id; A() { id = counter++; std::cout << "Constructing A[" << id << "]" << std::endl; } ~A() { std::cout << "Destructing A[" << id << "]" << std::endl; } boost::shared_ptr<A> ptr() { return shared_from_this(); } #define VARIANT0 #ifdef VARIANT0 // this variant works nicely with VARIANT1 below (compiles and // no error for `aa = a.ptr()`), but it produces wrong results, // i.e. `77` is returned for `aa.foo()` although it should have // been `33`. virtual int foo() { return 77; } #else virtual int foo() = 0; #endif };
struct AWrapper : public A, public boost::python::wrapper<A> { virtual int foo() { if( override f = this->get_override("foo") ) { return f(); } else { return 42; } } };
int A::counter = 0;
using namespace boost::python;
BOOST_PYTHON_MODULE(foo) { #define VARIANT1 //#define VARIANT2 //#define VARIANT3
#ifdef VARIANT1 // The following line doesn't compites it gives the error: // error: cannot allocate an object of abstract type ‘A’ class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT2 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // RuntimeError: tr1::bad_weak_ptr class_<AWrapper, boost::noncopyable >("A", init<>()) #endif #ifdef VARIANT3 // the following compiles, but produces an error in python, when executing // aa = a.ptr(); // TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<A> class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
("A", init<>()) #endif .def("ptr", &A::ptr) .def("foo", &A::foo) ; }
------------ foo.py ------------ import foo
class ExtendedA(foo.A): def foo(self): return 33;
a0 = foo.A() a = ExtendedA() aa = a.ptr(); print a.foo() del a print aa.foo() -----------------------------------
Do you have any suggestion how to get this working? -Holger
On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <bcatanzaro@acm.org> wrote:
Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
Dear list,
how is it possible to have a class in C++ that can be extended from python and that stores a weak_ptr to itself?
Have you tried using boost::enable_shared_from_this? The following example code seems to do what you want.
Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
on Fri Apr 27 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
On Mon, Apr 23, 2012 at 20:48, Holger Brandsmeier <brandsmeier@gmx.de> wrote:
In the meantime I figured out a different way of doing the shared pointer from this in python. Namely, to just make getThisRCP() a virtual function, add it to the Wrapper and to implement it in the python by implementing it as
but I still expect the above mentioned approch with `boost::enable_shared_from_this` _should_ have worked with boost python. I seem to run into an issue that even Dave can not explain (well lets wait for his reply).
Unfortunately I don't have the time to do the analysis right now, but I agree with you that it should have worked. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
participants (4)
-
Bryan Catanzaro -
Dave Abrahams -
Holger Brandsmeier -
Jim Bosch