Re: [C++-sig] shared_ptr and register_ptr_to_python
On 10/10/2010 03:58 AM, Marek Denis wrote:
On 10.10.2010 03:47, Jim Bosch wrote:
Pretty much everything you need should automatically work, whether you use shared_ptr or raw pointers with call policies. If you've invoked register_ptr_to_python on all the classes you're using,
That's what I couldn't get while reading Boost tutorial (the example code didn't work either :( ). Let's say I have three classes:
class A {}; class B : public A {}; class C : public B {};
should I later just put: register_ptr_to_python< A >(); register_ptr_to_python< B >(); register_ptr_to_python< C >(); ?
Actually, you want something like this: register_ptr_to_python< boost::shared_ptr<A> >(); If you want to use raw pointers, I don't think you need register_ptr_to_python (see below).
Here is the small code snippet:
+v boost::python::call_method<void>(callback,"addToQueue",c); -v
where callback is a PyObject* pointer,and 'c' is a pointer to the object. Whenever I call this instruction, I can see that the 'c' object is being copied (I just put some cout << "debug" << endl in the definition of the Copy Constructor), so I assumed that Python code doesn't operate on the object that was created in C++ module. Instead, it just operates on the copy.
I'm actually kind of surprised about this - in other circumstances (particularly when returning values in wrapped functions by pointer), Boost.Python doesn't copy pointers or references unless you explicitly ask it to. One thing that may be worth trying: when you wrap C Python, mark it as noncopyable: class_<C,bases<B>,boost::noncopyable>("C") I'd be curious to see what happens with your call_method code snippet with that in place. In any case, if you're willing to use shared_ptr everywhere instead of raw pointers, and switch your register_ptr_to_python calls as I've shown above, I think that will also stop any copying that's going on here. There should be a way to do it with raw pointers too, but I don't see it yet (and it may not exist). Jim
On 10.10.2010 20:15, Jim Bosch wrote:
Actually, you want something like this:
register_ptr_to_python< boost::shared_ptr<A> >();
If you want to use raw pointers, I don't think you need register_ptr_to_python (see below).
You are right.
I'm actually kind of surprised about this - in other circumstances (particularly when returning values in wrapped functions by pointer), Boost.Python doesn't copy pointers or references unless you explicitly ask it to.
I did some tests to check it out (you can check the code at [1] ) and noticed, that when I launch the +v boost::python::call_method<void>(callback,"addToQueue",c); -v piece of code, the object is copied after using self.queue.get() in the python script. This may be the magic (and/or implementation of the Queue.Queue) of Python layer, not Boost.Python itself. So I believe I will have to live with the pointless (in this case) memory copying in my program :)
One thing that may be worth trying: when you wrap C Python, mark it as noncopyable:
class_<C,bases<B>,boost::noncopyable>("C")
I'd be curious to see what happens with your call_method code snippet with that in place.
I will let you know when I find somthing out. [1] http://marek.octogan.net/python/ Using raw pointers is enough in my case, shared_ptrs would be safer, as different threads may delete object. -- Marek Denis
On 10/10/2010 02:16 PM, Marek Denis wrote:
I did some tests to check it out (you can check the code at [1] ) and noticed, that when I launch the +v boost::python::call_method<void>(callback,"addToQueue",c); -v
piece of code, the object is copied after using self.queue.get() in the python script. This may be the magic (and/or implementation of the Queue.Queue) of Python layer, not Boost.Python itself. So I believe I will have to live with the pointless (in this case) memory copying in my program :)
One thing that may be worth trying: when you wrap C Python, mark it as noncopyable:
class_<C,bases<B>,boost::noncopyable>("C")
I'd be curious to see what happens with your call_method code snippet with that in place.
I will let you know when I find somthing out.
[1] http://marek.octogan.net/python/
Using raw pointers is enough in my case, shared_ptrs would be safer, as different threads may delete object.
It is indeed call_method that's generating the copy; because there's no place for a call policy there, it tries to use a by-value converter. If you try it with boost::noncopyable, it refuses to compile for exactly that reason. I'm surprised that's it's behavior (I don't think it's really consistent with the rest of Boost.Python, but it is what it is). Essentially, what you need to do is pass a Python object to call_method (boost::python::object or PyObject *) that already contains your C++ object. In order to get one, you need need to tell Boost.Python to construct A and B with what it calls a "back reference". Your C++ constructors will be given a PyObject* by boost.python, pointing to the Python objects the C++ object will be wrapped in, and you can then store those as data members and pass the PyObject * to call_method. You can find more information in the Boost.Python reference docs on class_, especially the "HeldType Semantics" section. By the way, the call policies on your "gen" function should be return_value_policy<manage_new_object>, not return_internal_reference<>; the latter is for returning data members of wrapped classes and other things that are owned by another existing wrapped object. Jim
On 11.10.2010 00:42, Jim Bosch wrote:
Essentially, what you need to do is pass a Python object to call_method (boost::python::object or PyObject *) that already contains your C++ object. In order to get one, you need need to tell Boost.Python to construct A and B with what it calls a "back reference". Your C++ constructors will be given a PyObject* by boost.python, pointing to the Python objects the C++ object will be wrapped in, and you can then store those as data members and pass the PyObject * to call_method. You can find more information in the Boost.Python reference docs on class_, especially the "HeldType Semantics" section.
I did checked the docs, but I think I don't get how should it be done in my case. What they talk about (as well as in the provided example) They let the user to create Python classes derived from C++ classes and launchind virtual Python function. Would ou mind providing a sample piece of code how should it be done to wrap my C++ object into PyObject* ? Thanks in advance.
By the way, the call policies on your "gen" function should be return_value_policy<manage_new_object>, not return_internal_reference<>; the latter is for returning data members of wrapped classes and other things that are owned by another existing wrapped object.
Good point, thanks! -- regards Marek Denis
On 10/11/2010 10:49 AM, Marek Denis wrote:
On 11.10.2010 00:42, Jim Bosch wrote:
Essentially, what you need to do is pass a Python object to call_method (boost::python::object or PyObject *) that already contains your C++ object. In order to get one, you need need to tell Boost.Python to construct A and B with what it calls a "back reference". Your C++ constructors will be given a PyObject* by boost.python, pointing to the Python objects the C++ object will be wrapped in, and you can then store those as data members and pass the PyObject * to call_method. You can find more information in the Boost.Python reference docs on class_, especially the "HeldType Semantics" section.
I did checked the docs, but I think I don't get how should it be done in my case. What they talk about (as well as in the provided example) They let the user to create Python classes derived from C++ classes and launchind virtual Python function. Would ou mind providing a sample piece of code how should it be done to wrap my C++ object into PyObject* ? Thanks in advance.
I think this is something like what you want to do: ----------------------------------------- namespace bp = boost::python class A { public: std::string getData() { return _data; } void setCallback(bp::object const & callback) { _callback = callback; } bp::object invokeCallback() { return _callback(getSelf()); } virtual PyObject * getSelf() const = 0; virtual ~A() {} explicit A(std::string const & data) : _data(data) {} private: std::string _data; bp::object _callback; }; class B : public A { public: explicit B(std::string const & data) : A(data) {} }; class PyA : public A { public: PyA(PyObject * self, std::string const & data) : A(data), _self(self) {} virtual PyObject * getSelf() const { return _self; } private: PyObject * _self; }; class PyB : public B { public: PyB(PyObject * self, std::string const & data) : B(data), _self(self) {} virtual PyObject * getSelf() const { return _self; } private: PyObject * _self; }; BOOST_PYTHON_MODULE(example) { bp::class_<A,PyA>("A", bp::init<std::string>(bp::arg("data"))) .add_property("data", &A::getData) .def("setCallback", &A::setCallback) .def("invokeCallback", &A::invokeCallback) bp::class_<B,bp::bases<A>,PyB>( "B", bp::init<std::string>(bp::arg("data")) ); } ----------------------------------------- A couple of notes: - I have not tested this. Likely there are some typos, and possibly even a bit of forgotten syntax - hopefully it's easy to correct. - Note that there's no need to re-wrap the methods that B inherits from A, but you do have create both PyA and PyB if you want both A and B to be instantiable from Python (after all, getSelf() is pure virtual). - If you have a C++ function that returns an A or B instance by reference or pointer, this code will still work. Of course, such an object will have to find some other way to implement getSelf() - only objects instantiated in Python will actually be PyA or PyB instances. - I didn't actually use call_method anywhere, because bp::object has an overloaded operator() that does what you need (and using object here is better than PyObject because it handles the reference counting; you don't use it for _self because we don't want reference counting there). - This is a little non-standard because the base classes A and B are Python-aware and are pure virtual - but that makes sense, because your pattern requires that all C++ objects have a back-reference to their Python selves. - I think there's a way to do this that merges each class with it's Python counterpart (i.e., it tells Boost.Python that A has a constructor that takes the special "self" argument), and it has something to do with specializing a template traits class. But I haven't ever done it, and I don't want to lead you astray. HTH! Jim
On 11.10.2010 22:01, Jim Bosch wrote:
A couple of notes:
- I have not tested this. Likely there are some typos, and possibly even a bit of forgotten syntax - hopefully it's easy to correct.
- Note that there's no need to re-wrap the methods that B inherits from A, but you do have create both PyA and PyB if you want both A and B to be instantiable from Python (after all, getSelf() is pure virtual).
- If you have a C++ function that returns an A or B instance by reference or pointer, this code will still work. Of course, such an object will have to find some other way to implement getSelf() - only objects instantiated in Python will actually be PyA or PyB instances.
I still don't think it solves my problem as in original case the objects A and B are created only under C++. I would like to pass the pointer of such object, insted of copying them into Queue.Queue in Python and you wrote that the objects (A,B) need to be instantiated under Python...This time it's more like bad Queue.Queue implementation, not boost.python (as CallPolicies don't work). Anyway, thank you for you help, I appreciate... -- pozdrawiam Marek Denis [marek@octogan.net]
participants (2)
-
Jim Bosch -
Marek Denis