From tr.ernie.l at gmail.com Mon Mar 9 21:26:30 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Mon, 9 Mar 2015 14:26:30 -0600 Subject: [C++-sig] Memory management in Boost.Python Message-ID: Hello, I am working on adding Python binding to C++ project and run into the following problem: For some reason Python tries to delete objects that are still held by smart pointer instances in our C++ code. We are using std::shared_ptr for memory management and classes exposed through Boost::Python have HeldType set to std::shared_ptr. This problem manifests itself in the following scenarios: - in Python i subclass C++ class, which acts as ?callback?, and create an instance of it and pass it to C++ code, where it is held in std::shared_ptr instance. Python script itself does not use this callback object after that point. When Python script terminates i receive memory error, which seems to be triggered by attempting to double-free this callback object. - in Python i subclass C++ class, create an instance of it (call it ?A') and store it in container-like object (call it ?B') (B is defined in C++ and exposed to Python). B stores shared_ptr to A. As soon as Python execution point leaves current scope my object A gets deleted while B still holds shared_ptr to A. It looks to me that ether i have a bug in my wrapping code (any idea what this could be?) or in some cases Python disregards HeldType and tries to determine object lifespan by its own means (in both cases from Python point of view created object is no longer in scope so i am guessing thats why it tries to delete it). If the latter is correct, any idea how to force Python to use smart pointer class specified in HeldType? Thank you, Ernie. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at seefeld.name Tue Mar 10 16:17:58 2015 From: stefan at seefeld.name (Stefan Seefeld) Date: Tue, 10 Mar 2015 11:17:58 -0400 Subject: [C++-sig] Memory management in Boost.Python In-Reply-To: References: Message-ID: <54FF0B26.70105@seefeld.name> Ernie, it appears what you are seeing is the fact that boost.python only supports boost::shared_ptr, but not std::shared_ptr. (See https://svn.boost.org/trac/boost/ticket/6545). Is it possible for you to switch to Boost's shared_ptr, at least for the Python bindings ? Yes, we need to fix this urgently... :-( Stefan -- ...ich hab' noch einen Koffer in Berlin... From tr.ernie.l at gmail.com Tue Mar 10 22:55:31 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Tue, 10 Mar 2015 15:55:31 -0600 Subject: [C++-sig] Memory management in Boost.Python In-Reply-To: <54FF0B26.70105@seefeld.name> References: <54FF0B26.70105@seefeld.name> Message-ID: Thank you for looking this up Stefan! It turned out that we actually using boost::shared_ptr but in rather non-straight-forward way: it got pull in to our own namespace so in the end Boost::Python::class_ is 'called' with held-type set to a formally different type name. I will see if setting held-type to boost::shared_ptr fix this. Best, Ernie. On Tue, Mar 10, 2015 at 9:17 AM, Stefan Seefeld wrote: > Ernie, > > it appears what you are seeing is the fact that boost.python only > supports boost::shared_ptr, but not std::shared_ptr. (See > https://svn.boost.org/trac/boost/ticket/6545). Is it possible for you to > switch to Boost's shared_ptr, at least for the Python bindings ? Yes, we > need to fix this urgently... :-( > > Stefan > > > -- > > ...ich hab' noch einen Koffer in Berlin... > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > https://mail.python.org/mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tr.ernie.l at gmail.com Wed Mar 11 04:24:08 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Tue, 10 Mar 2015 21:24:08 -0600 Subject: [C++-sig] Memory management in Boost.Python In-Reply-To: References: <54FF0B26.70105@seefeld.name> Message-ID: Hi Stefan, I updated my code so it now use 'boost::shared_ptr' class while specifying the held-type and i got exactly the same errors (i guess boost did recognize class even in different namespace). Any other theories of what could be wrong? Could it be that Python in some cases tries to manage memory directly, disregarding SP layer? Thanks, Ernie. On Tue, Mar 10, 2015 at 3:55 PM, Ernie Lee wrote: > Thank you for looking this up Stefan! It turned out that we actually > using boost::shared_ptr but in rather non-straight-forward way: it got pull > in to our own namespace so in the end Boost::Python::class_ is 'called' > with held-type set to a formally different type name. I will see if setting > held-type to boost::shared_ptr fix this. > > Best, > > Ernie. > > > On Tue, Mar 10, 2015 at 9:17 AM, Stefan Seefeld > wrote: > >> Ernie, >> >> it appears what you are seeing is the fact that boost.python only >> supports boost::shared_ptr, but not std::shared_ptr. (See >> https://svn.boost.org/trac/boost/ticket/6545). Is it possible for you to >> switch to Boost's shared_ptr, at least for the Python bindings ? Yes, we >> need to fix this urgently... :-( >> >> Stefan >> >> >> -- >> >> ...ich hab' noch einen Koffer in Berlin... >> >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> https://mail.python.org/mailman/listinfo/cplusplus-sig >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at seefeld.name Wed Mar 11 07:01:09 2015 From: stefan at seefeld.name (Stefan Seefeld) Date: Wed, 11 Mar 2015 02:01:09 -0400 Subject: [C++-sig] Memory management in Boost.Python In-Reply-To: References: <54FF0B26.70105@seefeld.name> Message-ID: <54FFDA25.5070504@seefeld.name> On 10/03/15 11:24 PM, Ernie Lee wrote: > Hi Stefan, > > I updated my code so it now use 'boost::shared_ptr' class while > specifying the held-type and i got exactly the same errors (i guess > boost did recognize class even in different namespace). > > Any other theories of what could be wrong? Could it be that Python > in some cases tries to manage memory directly, disregarding SP layer? In that case I suggest you narrow down the failure into a minimal test case and send that to the list. Otherwise this would be highly speculative and thus inefficient. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From stefano.franchi at gmail.com Thu Mar 12 17:05:52 2015 From: stefano.franchi at gmail.com (stefano franchi) Date: Thu, 12 Mar 2015 11:05:52 -0500 Subject: [C++-sig] [pybindgen] How to wrap char** argv[]? Message-ID: [I posted the same question on Ubuntu's Launchpad's answers site, where earlier discussions of pybindgen took place. However, there seems to be no activity on that site, so i am reposting here] Problem with pybindgen: I stumbled upon a library that includes a function using pointers to the standard main function's arguments, and therefore uses a signature of char** argv[] (that is, a pointer to the standard char* argv[]): in main I have the standard signature with reference to its arguments: int main( int argc, char* argv[] ) { // initialize libstage - call this first Stg::Init( &argc, &argv ); ----------etc.----------------- and then the signature of the called function becomes: void Init( int* argc, char** argv[] ); --------------------------------- Given this situation, which parameter am I supposed to pass to add_function('Init')? I saw the answer how to pass char** argv, but I am not sure it is applicable here, as I don't understand the code in that answer too well, I am afraid. Thanks, Stefano -- __________________________________________________ Stefano Franchi stefano.franchi at gmail.com http://stefano.cleinias.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From tr.ernie.l at gmail.com Mon Mar 16 01:11:17 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Sun, 15 Mar 2015 18:11:17 -0600 Subject: [C++-sig] Memory management in Boost.Python In-Reply-To: <54FFDA25.5070504@seefeld.name> References: <54FF0B26.70105@seefeld.name> <54FFDA25.5070504@seefeld.name> Message-ID: Hi Stefan, I tried to create minimal tests and it turned out that there might be two separate issues here: one related to memory management and other to subclassing C++ classes in Python (i will post the subclassing example in separate mail). Running code below as-is works fine and i can see printed message generated by ?A? class destructor. However commenting out line ?reset_callback()? in Python code will lead to a segfault and also no call to ?A? class destructor occur. Thoughts? Thanks, Ernie. Python code: ???????? from callback2 import * a1 = A() a1.info() set_callback(a1) test_callback() reset_callback() C++ code: ?????????? include #include #include class A { public: virtual ~A() { std::cout << "A destructor for object: " << this << std::endl; } virtual void info() { std::cout << "C++ A info for object" << this << std::endl; } }; static boost::shared_ptr current_callback; void set_callback(boost::shared_ptr a) {current_callback = a; } void reset_callback() { current_callback = boost::shared_ptr(); } void test_callback() { std::cout << "test_callback: "; if(current_callback) current_callback->info(); else std::cout << "Callback is NULL!" << std::endl; } BOOST_PYTHON_MODULE(callback2) { boost::python::class_, boost::noncopyable>("A") .def("info", &A::info) ; boost::python::def("set_callback", set_callback); boost::python::def("reset_callback", reset_callback); boost::python::def("test_callback", test_callback); } ???????????????? On Wed, Mar 11, 2015 at 12:01 AM, Stefan Seefeld wrote: > On 10/03/15 11:24 PM, Ernie Lee wrote: > > Hi Stefan, > > > > I updated my code so it now use 'boost::shared_ptr' class while > > specifying the held-type and i got exactly the same errors (i guess > > boost did recognize class even in different namespace). > > > > Any other theories of what could be wrong? Could it be that Python > > in some cases tries to manage memory directly, disregarding SP layer? > > In that case I suggest you narrow down the failure into a minimal test > case and send that to the list. Otherwise this would be highly > speculative and thus inefficient. > > Regards, > Stefan > > -- > > ...ich hab' noch einen Koffer in Berlin... > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > https://mail.python.org/mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tr.ernie.l at gmail.com Mon Mar 16 01:34:06 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Sun, 15 Mar 2015 18:34:06 -0600 Subject: [C++-sig] Subclassing C++ classes in Python Message-ID: Hi there, I am creating binding for some C++ classes and I need to be able to subclass exposed C++ classes in Python (and pass subclassed instances back in to C++ code). I was able to make this work by following this example: http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions and creating binding for both: original C++ classes and ?wrapper? structs. This however create inconvenience and path for errors because now users need to remember to use different base class when subclassing C++ classes. Is there is a way to achieve subclassing functionality by exposing one class instead of two? When I tries to use class exposed as ?wrapper? as substitute for C++ original class i am getting a segfault error (see example below). Is there is something wrong with a way i create a wrapper-class or it is not intended to work as substitute for a base class? Thanks, Ernie. Python code: ???????? from subclass import * class B(A_Wrapper): def __init__(self): A_Wrapper.__init__(self) def info(self): print 'B info!' a = A(); a.info(); print '1' b = B(); b.info(); print '2' aw = A_Wrapper(); print '3'; aw.info(); print '4' # <-- segfault before '4' got printed C++ code: ???????? #include #include class A { public: virtual ~A() { std::cout << "A destructor for object: " << this << std::endl; } virtual void info() { std::cout << "C++ A info for object" << this << std::endl; } }; struct A_Wrapper : A, boost::python::wrapper { A_Wrapper() : A() {} void info() {this->get_override("info")(); } }; BOOST_PYTHON_MODULE(subclass) { boost::python::class_("A") .def("info", &A::info) ; boost::python::class_("A_Wrapper", boost::python::init <>() ) .def("info", &A_Wrapper::info) ; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From Holger.Joukl at LBBW.de Mon Mar 16 09:04:11 2015 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Mon, 16 Mar 2015 09:04:11 +0100 Subject: [C++-sig] Subclassing C++ classes in Python In-Reply-To: References: Message-ID: Hi Ernie, > ? When I tries to use class exposed as ?wrapper? as substitute for C > ++ original class i am getting a segfault error (see example below). > Is there is something wrong with a way i create a wrapper-class or > it is not intended to work as substitute for a base class? I haven't tried your code sample but you'd normally just expose the wrapper class *instead* of the original class, with the name of "A". Python users don't need to know about the wrapper class intricacies. If you expose a non-pure-virtual method you should also add the default implementation, just as you cited from http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions (necessary to avoid infinite recursion if your Python override implementation calls the C++ base class method) I.e. something like (untested! Stick closely to the tutorial if in doubt) #include #include class A { public: virtual ~A() { std::cout << "A destructor for object: " << this << std::endl; } virtual void info() { std::cout << "C++ A info for object" << this << std::endl; } }; struct A_Wrapper : A, boost::python::wrapper { void info( { if (override info_override = this->get_override("info")) { return info_override(); } return A::info(); } void default_info() { return this->A::info(); } }; BOOST_PYTHON_MODULE(subclass) { boost::python::class_("A", ? boost::python::init <>() ) .def("info", &A::info, &A_Wrapper::default_info) ; } So the wrapper class provides you with the callback-ability to Python and this is the one to expose. As per the segfault you get: I guess that this might stem from your not calling the boost::python::wrapper base class constructor in your A_Wapper constructor (note how I didn't provide a constructor for the wrapper class): A_Wrapper() : A() {} // <-- not calling all base constructors here Again, haven't actually tried it. Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From tr.ernie.l at gmail.com Mon Mar 16 19:05:22 2015 From: tr.ernie.l at gmail.com (Ernie Lee) Date: Mon, 16 Mar 2015 12:05:22 -0600 Subject: [C++-sig] Subclassing C++ classes in Python In-Reply-To: References: Message-ID: Hi Holger, Adding checks for override and default implementation fixed the issue. - Thank you! Best, Ernie. On Mon, Mar 16, 2015 at 2:04 AM, Holger Joukl wrote: > > Hi Ernie, > > > When I tries to use class exposed as ?wrapper? as substitute for C > > ++ original class i am getting a segfault error (see example below). > > Is there is something wrong with a way i create a wrapper-class or > > it is not intended to work as substitute for a base class? > > I haven't tried your code sample but you'd normally just expose the > wrapper class *instead* of the original class, with the name of "A". > > Python users don't need to know about the wrapper class intricacies. > > If you expose a non-pure-virtual method you should also add the > default implementation, just as you cited from > > http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions > (necessary to avoid infinite recursion if your Python override > implementation calls the > C++ base class method) > > I.e. something like (untested! Stick closely to the tutorial if in doubt) > > #include > #include > > class A > { > public: > virtual ~A() { std::cout << "A destructor for object: " << this << > std::endl; } > > virtual void info() { std::cout << "C++ A info for object" << this > << std::endl; } > }; > > > struct A_Wrapper : A, boost::python::wrapper > { > void info( > { > if (override info_override = this->get_override("info")) { > return info_override(); > } > return A::info(); > } > > void default_info() { return this->A::info(); } > }; > > > BOOST_PYTHON_MODULE(subclass) > { > boost::python::class_("A", > boost::python::init <>() ) > .def("info", &A::info, &A_Wrapper::default_info) > ; > } > > So the wrapper class provides you with the callback-ability to Python and > this is the one to expose. > > As per the segfault you get: I guess that this might stem from your not > calling the boost::python::wrapper base class constructor in your > A_Wapper > constructor (note how I didn't provide a constructor for the wrapper > class): > > A_Wrapper() : A() {} // <-- not calling all base constructors here > > Again, haven't actually tried it. > > Holger > > Landesbank Baden-Wuerttemberg > Anstalt des oeffentlichen Rechts > Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz > HRA 12704 > Amtsgericht Stuttgart > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > https://mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From cristeab at gmail.com Wed Mar 18 22:25:06 2015 From: cristeab at gmail.com (Bogdan Cristea) Date: Wed, 18 Mar 2015 22:25:06 +0100 Subject: [C++-sig] Passing std::wstring to Python callback Message-ID: <0FB1C672-F116-492B-B85D-A5E11547CC27@gmail.com> Hi I am using boost python for providing python bindings for a C++ library. I am using Python 2.7. My library needs to call callbacks that are defined in Python. There callbacks can take a string parameter. C++ side I am always using std::wstring. Everything works on as long as std::wstring contains only ASCII chars, but once I get something like ??? my application crashes. This issue seems to be related to the way I pass std::wstring to Python callbacks: as long as I make sure that the passed string has only ASCII chars it is ok. My problem is how do I pass unicode strings to Python. thanks Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dongta.hds at gmail.com Wed Mar 25 01:21:02 2015 From: dongta.hds at gmail.com (Dong Ta) Date: Tue, 24 Mar 2015 17:21:02 -0700 Subject: [C++-sig] Linking with Python3 Libs Message-ID: Hi, I'm trying to figure out how to build Python modules using Boost Python against Python3; and I'm currently stuck at getting cmake to find Python 3. I have already installed Python3 via brew install python3; but cmake still can't find PythonLibs 3. What am I missing here? Here's the header of my *CMakeLists.txt * cmake_minimum_required(VERSION 3.0) find_package(PythonLibs 3 REQUIRED) and here's the error message Could NOT find PythonLibs: Found unsuitable version "2.7.6", but required is at least "3" (found /usr/lib/libpython2.7.dylib) -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl.goode at siemens.com Wed Mar 25 20:45:28 2015 From: carl.goode at siemens.com (Goode, Carl R (PG GT LGT EN MT 4 2)) Date: Wed, 25 Mar 2015 19:45:28 +0000 Subject: [C++-sig] Reuse PyObject for non python instanciated C++ objects Message-ID: <116CC582B05E4548BE5EAC4C7404FB422AC638F3@USLZUA0EM12MSX.ww017.siemens.net> I have a use case where we have a C++ framework that is exposed through boost python. If I use a C++ factory to create objects, I want to always return the same python object once it is exposed. I've done this through a postcall policy, but I was wondering if there is a better way. Below is a simple example of my problem and my proposed solution. This works, with the only drawback that I create a PyObject* that I throw out every time. I do not want my framework (in this case, struct X) to know anything about python. Also, I'm familiar with "has_backed_reference" and using a heldType wrapper, but that would only apply to objects instantiated through python. struct X; std::map > xmap; std::map xpymap; struct X // a container element { std::string id; std::string check; X(std::string s) : id(s) { } void printId() { std::cout << "ID is " << id << std::endl; } void change_string(std::string nstring) { check = nstring; } void print_string() { std::cout << "String is now: " << check << std::endl; } boost::shared_ptr static XFactory(std::string val) { std::map >::iterator it = xmap.find(val); if (it == xmap.end()) { boost::shared_ptr nptr = boost::make_shared(val); xmap[val] = nptr; return nptr; } return xmap[val]; } }; // Custom call policy to overwrite the postcall from C++ back to python // template struct carls_policy : BasePolicy { template PyObject* postcall(const ArgumentPackage& args, PyObject* result) { // Chain to base policy. result = BasePolicy::postcall(args, result); // Extract shared_ptr from result // boost::shared_ptr xptr = boost::python::extract< boost::shared_ptr >(result); if (!xptr) { return result; } std::map< X*, PyObject* >::iterator it = xpymap.find(xptr.get()); if (it == xpymap.end()) { xpymap[xptr.get()] = result; Py_INCREF(xpymap[xptr.get()]); // Increment to store in map } else { Py_DECREF(result); // Not needed anymore } Py_INCREF(xpymap[xptr.get()]); // Inremented to return to python return xpymap[xptr.get()]; } }; BOOST_PYTHON_MODULE(rvp) { boost::python::class_ > ("X", boost::python::no_init) .def("XFactory", &X::XFactory, carls_policy<>()).staticmethod("XFactory") .def("printId", &X::printId) .def("change_string", &X::change_string) .def("print_string", &X::print_string) ; } Implementation: >>> import rvp >>> x1 = rvp.X.XFactory('id1') >>> x2 = rvp.X.XFactory('id2') >>> x3 = rvp.X.XFactory('id1') >>> x1 is x3 True This message and any attachments are solely for the use of intended recipients. The information contained herein may include trade secrets, protected health or personal information, privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you are not an intended recipient, you are hereby notified that you received this email in error, and that any review, dissemination, distribution or copying of this email and any attachment is strictly prohibited. If you have received this email in error, please contact the sender and delete the message and any attachment from your system. Thank you for your cooperation -------------- next part -------------- An HTML attachment was scrubbed... URL: From xavier.warin at gmail.com Fri Mar 27 14:27:11 2015 From: xavier.warin at gmail.com (warin) Date: Fri, 27 Mar 2015 13:27:11 +0000 (UTC) Subject: [C++-sig] =?utf-8?q?Boost_python_exposing_abstract_class_=2C_func?= =?utf-8?q?tion_returning_boost=3A=3Ashared=5Fptr?= Message-ID: > I'm not top posting. I have an abstract class A2, with derived class B2. I want to map a function returning a boost::shared_ptr of A. This abstract class is wrapped : struct A2{ virtual ~A2(){} virtual int ret() =0; }; struct B2: public A2{ virtual ~B2(){} int ret() { return 1; } }; // wrapper struct A2Wrap : A2, wrapper { inline int ret() { return this->get_override("ret")(); } }; // function to map boost::shared_ptr f1() { return boost::shared_ptr(new B2()); } // expose to python BOOST_PYTHON_MODULE(grids) { class_ , boost::noncopyable >("A2") .def("ret",pure_virtual(&A2::ret)); class_,bases>("B2"); def("f1",f1); } I get the following result on execution : >import grids >>> a = grids.f1() Traceback (most recent call last): File "", line 1, in TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr Any idea for this problem ? I imagine to express the conversion from boost::shared_ptr to boost::shared_ptr. From Holger.Joukl at LBBW.de Fri Mar 27 16:45:00 2015 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Fri, 27 Mar 2015 16:45:00 +0100 Subject: [C++-sig] Boost python exposing abstract class , function returning boost::shared_ptr In-Reply-To: References: Message-ID: Hi, > [...] > // expose to python > BOOST_PYTHON_MODULE(grids) > { > class_ , boost::noncopyable >("A2") > .def("ret",pure_virtual(&A2::ret)); > > class_,bases>("B2"); > def("f1",f1); > } > > > I get the following result on execution : > > >import grids > >>> a = grids.f1() > > Traceback (most recent call last): > > File "", line 1, in > TypeError: No to_python (by-value) converter found for C++ type: > boost::shared_ptr What happens if you expose like this: class_ , boost::noncopyable >("A2") ^^ |- original class here, not the callback-wrapper class ? Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From xavier.warin at gmail.com Sat Mar 28 14:58:20 2015 From: xavier.warin at gmail.com (warin) Date: Sat, 28 Mar 2015 13:58:20 +0000 (UTC) Subject: [C++-sig] =?utf-8?q?Boost_python_exposing_abstract_class_=2C_func?= =?utf-8?q?tion_returning_boost=3A=3Ashared=5Fptr?= References: Message-ID: > I'm not top posting. Hi Holger Thank you for the reply. It compiles but same error on execution. Sincerely yours Xavier From Holger.Joukl at LBBW.de Mon Mar 30 11:09:13 2015 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Mon, 30 Mar 2015 11:09:13 +0200 Subject: [C++-sig] Boost python exposing abstract class , function returning boost::shared_ptr In-Reply-To: References: Message-ID: Hi, > Thank you for the reply. > It compiles but same error on execution. If I change the A2 exposure to bp::class_, boost::noncopyable>("A2", bp::no_init) ... then f1() successfully returns a B2 object. But of course we then lose the Python-callback abilities provided by the wrapper class. Looking into the Boost.Python tests i stumbled across shared_ptr.cpp which contains: ... // This is the ugliness required to register a to-python converter // for shared_ptr. objects::class_value_wrapper< shared_ptr , objects::make_ptr_instance,A> > >(); ... So, giving this a try - looks like the following seems to be a working version for your sample (note that I changed the extension module name to shared_ptr_hierarchy in my code): C++ (wrapper) code: ================== #include namespace bp = boost::python; struct A2 { virtual ~A2(){} virtual int ret() =0; }; struct B2: public A2 { virtual ~B2(){} int ret() { return 1; } }; // wrapper struct A2Wrap : A2, bp::wrapper { inline int ret() { return this->get_override("ret")(); } }; // To the best of my knowledge we need to add callback support // through a wrapper class at every level of the hierarchy, so // a bit of code duplication going on. This is due to the fact // that B2 has no inheritance relation whatsoever to A2Wrap. // I haven't so far found an elegant/short solution for this // (e.g. using templates or macros) struct B2Wrap : B2, bp::wrapper { inline int ret() { if (bp::override py_override = this->get_override("ret")) { return py_override(); } return B2::ret(); } inline int default_ret() { return B2::ret(); } }; // function to map boost::shared_ptr f1() { return boost::shared_ptr(new B2()); } // invoke overridden virtual from C++ side (check callback-ability) int invoke_ret(A2 & a2) { return a2.ret(); } // expose to python BOOST_PYTHON_MODULE(shared_ptr_hierarchy) { bp::class_, boost::noncopyable>("A2") .def("ret", bp::pure_virtual(&A2::ret)) ; // taken from boost.python libs/python/test/shared_ptr.cpp: // This is the ugliness required to register a to-python converter // for shared_ptr. bp::objects::class_value_wrapper< boost::shared_ptr , bp::objects::make_ptr_instance,A2> > >(); bp::class_, bp::bases, boost::noncopyable >("B2") .def("ret", &B2::ret, &B2Wrap::default_ret) ; bp::def("f1", f1); bp::def("invoke_ret", &invoke_ret, (bp::arg("a2"))); } Python test code: ================= import shared_ptr_hierarchy class PyB2(shared_ptr_hierarchy.A2): def ret(self): print "PyB2 Python class", self return 5 class PyC2(shared_ptr_hierarchy.B2): def ret(self): print "PyC2 Python class", self return 6 def invoke(factory): print "\nfactory:", factory obj = factory() print "obj =", obj print "obj.ret() ->", obj.ret() if hasattr(shared_ptr_hierarchy, 'invoke_ret'): print "shared_ptr_hierarchy.invoke_ret(obj) ->", \ shared_ptr_hierarchy.invoke_ret(obj) try: invoke(shared_ptr_hierarchy.A2) except Exception as e: print "Exception:", e invoke(shared_ptr_hierarchy.B2) invoke(shared_ptr_hierarchy.f1) invoke(PyB2) invoke(PyC2) Test code output: ================= factory: obj = obj.ret() -> Exception: Pure virtual function called factory: obj = obj.ret() -> 1 shared_ptr_hierarchy.invoke_ret(obj) -> 1 factory: obj = obj.ret() -> 1 shared_ptr_hierarchy.invoke_ret(obj) -> 1 factory: obj = <__main__.PyB2 object at 0x17cb628> obj.ret() -> PyB2 Python class <__main__.PyB2 object at 0x17cb628> 5 shared_ptr_hierarchy.invoke_ret(obj) -> PyB2 Python class <__main__.PyB2 object at 0x17cb628> 5 factory: obj = <__main__.PyC2 object at 0x17cb628> obj.ret() -> PyC2 Python class <__main__.PyC2 object at 0x17cb628> 6 shared_ptr_hierarchy.invoke_ret(obj) -> PyC2 Python class <__main__.PyC2 object at 0x17cb628> 6 I can't really say that I understand why that's needed, but alas... Open issues: - Would be nice to have the abstract class A2 not being initializable from Python, but we can't use bp::no_init and inherit from A2 in Python because the Python-derived needs to call the (A2) base class constructor Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From xavier.warin at gmail.com Mon Mar 30 15:08:16 2015 From: xavier.warin at gmail.com (warin) Date: Mon, 30 Mar 2015 13:08:16 +0000 (UTC) Subject: [C++-sig] =?utf-8?q?Boost_python_exposing_abstract_class_=2C_func?= =?utf-8?q?tion_returning_boost=3A=3Ashared=5Fptr?= References: Message-ID: Hi Holger Thank you very much. A little bit ugly but it works. Even using std::shared_ptr instead of boost::shared_ptr Best regards From a.huebl at hzdr.de Mon Mar 30 17:16:46 2015 From: a.huebl at hzdr.de (Huebl, Axel) Date: Mon, 30 Mar 2015 17:16:46 +0200 Subject: [C++-sig] Boost.Python "contains" compile fix Message-ID: <551968DE.9020702@hzdr.de> Hi, adding more Python 3 features, the "contains" method was added (in 1.55.0?) http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html Unfortunately, I found a nasty compile bug when compiling with nvcc that I tried to fix in https://github.com/boostorg/python/pull/14 Does anyone know how to test my patch to see if this function is still working as expected? Thank you for your help, Axel -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5099 bytes Desc: S/MIME Cryptographic Signature URL: From stefan at seefeld.name Mon Mar 30 17:42:53 2015 From: stefan at seefeld.name (Stefan Seefeld) Date: Mon, 30 Mar 2015 11:42:53 -0400 Subject: [C++-sig] Boost.Python "contains" compile fix In-Reply-To: <551968DE.9020702@hzdr.de> References: <551968DE.9020702@hzdr.de> Message-ID: <55196EFD.3070901@seefeld.name> On 30/03/15 11:16 AM, Huebl, Axel wrote: > Hi, > > > adding more Python 3 features, the "contains" method was added (in 1.55.0?) > http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html > > Unfortunately, I found a nasty compile bug when compiling with nvcc that > I tried to fix in > https://github.com/boostorg/python/pull/14 > > Does anyone know how to test my patch to see if this function is still > working as expected? I have looked at your patch, but am not convinced that this is the right fix. It would be best to really map "contains" to "__contains__" to preserve the precise semantic of that call. Do you see the "incomplete type" error only with nvcc or also with other compilers (clang, notably) ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... From a.huebl at hzdr.de Mon Mar 30 22:37:59 2015 From: a.huebl at hzdr.de (Huebl, Axel) Date: Mon, 30 Mar 2015 22:37:59 +0200 Subject: [C++-sig] Boost.Python "contains" compile fix In-Reply-To: <55196EFD.3070901@seefeld.name> References: <551968DE.9020702@hzdr.de> <55196EFD.3070901@seefeld.name> Message-ID: <5519B427.6050501@hzdr.de> On 30.03.2015 17:42, Stefan Seefeld wrote: > On 30/03/15 11:16 AM, Huebl, Axel wrote: >> Hi, >> >> >> adding more Python 3 features, the "contains" method was added (in 1.55.0?) >> http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html >> >> Unfortunately, I found a nasty compile bug when compiling with nvcc that >> I tried to fix in >> https://github.com/boostorg/python/pull/14 >> >> Does anyone know how to test my patch to see if this function is still >> working as expected? > > I have looked at your patch, but am not convinced that this is the right > fix. It would be best to really map "contains" to "__contains__" to > preserve the precise semantic of that call. > Do you see the "incomplete type" error only with nvcc or also with other > compilers (clang, notably) ? > > Thanks, > Stefan > Hi Stefan, I did only triggered the compile error it with nvcc since that is the target I want to move Boost.Python to. The main problem is actually in accessing the this->attr() method since it's return type "const_object_attribute" is not yet fully typed. Maybe it is therefore enough to just move the implementation of api::object_operators::contains down to a source file, but I did not find an according object_core.cpp Do you have any hints on that? Thanks, Axel -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5099 bytes Desc: S/MIME Cryptographic Signature URL: From a.huebl at hzdr.de Mon Mar 30 22:39:37 2015 From: a.huebl at hzdr.de (Huebl, Axel) Date: Mon, 30 Mar 2015 22:39:37 +0200 Subject: [C++-sig] Boost.Python "contains" compile fix In-Reply-To: <5519B427.6050501@hzdr.de> References: <551968DE.9020702@hzdr.de> <55196EFD.3070901@seefeld.name> <5519B427.6050501@hzdr.de> Message-ID: <5519B489.1090108@hzdr.de> On 30.03.2015 22:37, Huebl, Axel wrote: > On 30.03.2015 17:42, Stefan Seefeld wrote: >> On 30/03/15 11:16 AM, Huebl, Axel wrote: >>> Hi, >>> >>> >>> adding more Python 3 features, the "contains" method was added (in 1.55.0?) >>> http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html >>> >>> Unfortunately, I found a nasty compile bug when compiling with nvcc that >>> I tried to fix in >>> https://github.com/boostorg/python/pull/14 >>> >>> Does anyone know how to test my patch to see if this function is still >>> working as expected? >> >> I have looked at your patch, but am not convinced that this is the right >> fix. It would be best to really map "contains" to "__contains__" to >> preserve the precise semantic of that call. >> Do you see the "incomplete type" error only with nvcc or also with other >> compilers (clang, notably) ? >> >> Thanks, >> Stefan >> > > Hi Stefan, > > > I did only triggered the compile error it with nvcc since that is the > target I want to move Boost.Python to. oh that wasn't precise enough: to answer your question, I did not try it with clang (yet). > > The main problem is actually in accessing the this->attr() method since > it's return type "const_object_attribute" is not yet fully typed. > > Maybe it is therefore enough to just move the implementation of > api::object_operators::contains > > down to a source file, but I did not find an according > object_core.cpp > > Do you have any hints on that? > > > Thanks, > Axel > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5099 bytes Desc: S/MIME Cryptographic Signature URL: From stefan at seefeld.name Mon Mar 30 22:42:32 2015 From: stefan at seefeld.name (Stefan Seefeld) Date: Mon, 30 Mar 2015 16:42:32 -0400 Subject: [C++-sig] Boost.Python "contains" compile fix In-Reply-To: <5519B427.6050501@hzdr.de> References: <551968DE.9020702@hzdr.de> <55196EFD.3070901@seefeld.name> <5519B427.6050501@hzdr.de> Message-ID: <5519B538.3000105@seefeld.name> On 30/03/15 04:37 PM, Huebl, Axel wrote: > On 30.03.2015 17:42, Stefan Seefeld wrote: >> On 30/03/15 11:16 AM, Huebl, Axel wrote: >>> Hi, >>> >>> >>> adding more Python 3 features, the "contains" method was added (in 1.55.0?) >>> http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html >>> >>> Unfortunately, I found a nasty compile bug when compiling with nvcc that >>> I tried to fix in >>> https://github.com/boostorg/python/pull/14 >>> >>> Does anyone know how to test my patch to see if this function is still >>> working as expected? >> I have looked at your patch, but am not convinced that this is the right >> fix. It would be best to really map "contains" to "__contains__" to >> preserve the precise semantic of that call. >> Do you see the "incomplete type" error only with nvcc or also with other >> compilers (clang, notably) ? >> >> Thanks, >> Stefan >> > Hi Stefan, > > > I did only triggered the compile error it with nvcc since that is the > target I want to move Boost.Python to. > > The main problem is actually in accessing the this->attr() method since > it's return type "const_object_attribute" is not yet fully typed. > > Maybe it is therefore enough to just move the implementation of > api::object_operators::contains > > down to a source file, but I did not find an according > object_core.cpp > > Do you have any hints on that? Not yet. I was asking about clang because I believe nvcc is based on clang, and woud thus allow me to attempt to reproduce the problem without having to install nvcc first. (I have run into issues such as the above with my own code using clang, but refactoring the code to make sure all the right pieces are seen in the right order can open up a huge rat-hole, if there are circular dependencies...) > oh that wasn't precise enough: to answer your question, > I did not try it with clang (yet). Trying that would be useful, I think. Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... From axel.huebl at web.de Mon Mar 30 22:37:42 2015 From: axel.huebl at web.de (=?UTF-8?B?QXhlbCBIw7xibA==?=) Date: Mon, 30 Mar 2015 22:37:42 +0200 Subject: [C++-sig] Boost.Python "contains" compile fix In-Reply-To: <55196EFD.3070901@seefeld.name> References: <551968DE.9020702@hzdr.de> <55196EFD.3070901@seefeld.name> Message-ID: <5519B416.10805@web.de> On 30.03.2015 17:42, Stefan Seefeld wrote: > On 30/03/15 11:16 AM, Huebl, Axel wrote: >> Hi, >> >> >> adding more Python 3 features, the "contains" method was added (in 1.55.0?) >> http://www.boost.org/doc/libs/1_55_0/libs/python/doc/news.html >> >> Unfortunately, I found a nasty compile bug when compiling with nvcc that >> I tried to fix in >> https://github.com/boostorg/python/pull/14 >> >> Does anyone know how to test my patch to see if this function is still >> working as expected? > > I have looked at your patch, but am not convinced that this is the right > fix. It would be best to really map "contains" to "__contains__" to > preserve the precise semantic of that call. > Do you see the "incomplete type" error only with nvcc or also with other > compilers (clang, notably) ? > > Thanks, > Stefan > Hi Stefan, I did only triggered the compile error it with nvcc since that is the target I want to move Boost.Python to. The main problem is actually in accessing the this->attr() method since it's return type "const_object_attribute" is not yet fully typed. Maybe it is therefore enough to just move the implementation of api::object_operators::contains down to a source file, but I did not find an according object_core.cpp Do you have any hints on that? Thanks, Axel -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 949 bytes Desc: OpenPGP digital signature URL: