I have a function that reads a file and returns (depending on the content of the file) either a Foo or a Bar wrapped in a PyObject* (both Foo and Bar objects are expensive to copy and have to be passed via a smart pointer). With Boost.Python v1 I was able to do PyObject * readFile(char const * fileName) { PyObject * obj; FileInfo info(fileName); if (info.isFoo()) { std::auto_ptr<Foo> foo(new Foo(info.data())); obj = to_python(foo); } else { std::auto_ptr<Bar> bar(new Bar(info.data())); obj = to_python(bar); } return ref(obj); } and wrap this like so: BOOST_PYTHON_MODULE_INIT(vigracore) { module_builder this_module("reader"); this_module.def(&readFile, "readFile"); } With Boost.Python v2, I have not been able to find an equivalent solution; I tried various approaches gathered from the Boost.Python FAQ and documentation, e.g. template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function(&identity<T>, return_value_policy<manage_new_object>() ); return f(x); } object readFile(char const * fileName) { FileInfo info(fileName); if (info.isFoo()) { Foo * fooPtr = new Foo(info.data()); return get_pointer_reference(fooPtr); } else { Bar * barPtr = new Bar(info.data()); return get_pointer_reference(barPtr); } } BOOST_PYTHON_MODULE(reader) { def("readFile", &readFile); } which compiles fine but crashes at runtime (using gcc 3.2.2 on Linux with a recent CVS version of Boost). Any hints that might help me solving this mystery would be much appreciated! Oliver -------------------------------------------------------------------- F. Oliver Gathmann, Ph.D. Director IT Unit Cenix BioScience GmbH Pfotenhauer Strasse 108 phone: +49 (351) 210-2735 D-01307 Dresden, Germany fax: +49 (351) 210-1309 fingerprint: 8E0E 9A64 A07E 0D1A D302 34C2 421A AE9F 4E13 A009 public key: http://www.cenix-bioscience.com/public_keys/gathmann.gpg --------------------------------------------------------------------
"F. Oliver Gathmann" <gathmann@cenix-bioscience.com> writes:
With Boost.Python v2, I have not been able to find an equivalent solution; I tried various approaches gathered from the Boost.Python FAQ and documentation, e.g.
Nicodemus has nearly finished building/documenting a function which allows you to register conversions from any smart pointer to Python. See recent posts to this list. You could use that.
template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function(&identity<T>, return_value_policy<manage_new_object>() ); return f(x); }
Hmm, *nasty* formatting! template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function( &identity<T>, return_value_policy<manage_new_object>()); return f(x); } Hmm, looks OK to me.
which compiles fine but crashes at runtime (using gcc 3.2.2 on Linux with a recent CVS version of Boost). Any hints that might help me solving this mystery would be much appreciated!
Any hints about the condition of the program at the time of crash would be a big help in helping you. A reproducible test case would be even better. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
"F. Oliver Gathmann" <gathmann@cenix-bioscience.com> writes:
template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function(&identity<T>, return_value_policy<manage_new_object>() ); return f(x); }
Hmm, *nasty* formatting!
Strange - the formatting looks fine in the c++-SIG archives...
template <class T> T identity(T x) { return x; }
template <class T> object get_pointer_reference(T x) { object f = make_function( &identity<T>, return_value_policy<manage_new_object>());
return f(x); }
Hmm, looks OK to me.
which compiles fine but crashes at runtime (using gcc 3.2.2 on Linux with a recent CVS version of Boost). Any hints that might help me solving this mystery would be much appreciated!
Any hints about the condition of the program at the time of crash would be a big help in helping you. A reproducible test case would be even better.
Sorry; I thought my error was so obvious that pseudocode would be enough to find out what's wrong. Here is a complete test case: #include <memory> #include <boost/python.hpp> using namespace boost::python; template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function(&identity<T>, return_value_policy<manage_new_object>() ); return f(x); } struct Foo {}; struct Bar {}; object readFooOrBar(bool fooOrBar) { if (fooOrBar) { Foo * fooPtr = new Foo(); return get_pointer_reference(fooPtr); } else { Bar * barPtr = new Bar(); return get_pointer_reference(barPtr); } } BOOST_PYTHON_MODULE(_problem) { class_<Foo, std::auto_ptr<Foo> >("Foo", no_init) ; class_<Bar, std::auto_ptr<Bar> >("Bar", no_init) ; def("read", &readFooOrBar); } And in Python: Python 2.2.2 (#1, Apr 10 2003, 23:02:08) [GCC 3.3 20030226 (prerelease) (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from _problem import * foo = read(1) bar = read(0) del foo del bar Segmentation fault
When using Nicodemus' register_ptr_to_python template like so BOOST_PYTHON_MODULE(_problem) { class_<Foo>("Foo", no_init) ; register_ptr_to_python<std::auto_ptr<Foo> >(); class_<Bar>("Bar", no_init) ; register_ptr_to_python<std::auto_ptr<Bar> >(); def("read", &readFooOrBar); } the situation is essentially unchanged. BTW, I had to insert a typename directive for gcc to compile the register_ptr_to_python template: namespace boost { namespace python { template <class P> void register_ptr_to_python(P* = 0) { typedef typename boost::python::pointee<P>::type X; ^------^ objects::class_value_wrapper< P , objects::make_ptr_instance< X , objects::pointer_holder<P,X> > >(); } }} // namespace boost::python I guess I am still somewhat at a loss here... - but I am certainly learning a lot ;-) Oliver -------------------------------------------------------------------- F. Oliver Gathmann, Ph.D. Director IT Unit Cenix BioScience GmbH Pfotenhauer Strasse 108 phone: +49 (351) 210-2735 D-01307 Dresden, Germany fax: +49 (351) 210-1309 fingerprint: 8E0E 9A64 A07E 0D1A D302 34C2 421A AE9F 4E13 A009 public key: http://www.cenix-bioscience.com/public_keys/gathmann.gpg --------------------------------------------------------------------
Hi all, I have a written several routines in C++ using blitz++. These routines are called sucessively by the 'main' routine. I want to rewrite the main in python in order to gain development speed and flexibility. The routines use blitz++ matrix template library but the interfaces to the routine are very common C++ calls with basic C++ types (int, float*, short*, etc...) : int foo( short* a, float b, float* c, int N ){ Array<short, 1> A(N); A.data() = a; // Do some work return ( ERROR_CODE ); } Is it straightforward to declare these routines with pyste (or the classic boost method). Does the use of blitz in these routines (which belong to another file than the main, beginning with include of blitz.hh) complicates the process. I'm a bit lost because of the use of "templates in blitz++" versus "integration with boost". Also, as you can see in the example routine above, I started with a creation of a blitz array 'A' from the array 'a'. Because I'm afraid of the work to get back and forth from Numeric array in Python to Blitz++ templated Array in C++ is a complicated and big amount of work. Thanks for any response on these to points, and for any suggestion or criticism on my way of handling the problem. Christophe Grimault -- ------------------------------------------------------ | Christophe Grimault Tel: 02 23 23 52 59 | | NovaGrid SA http://www.novagrid.com | | fax: 02 23 23 62 32 | | mail: christophe.grimault@novagrid.com | | 2, Bd Sebastopol | | 35000 RENNES | | France | |______________________________________________________|
christophe grimault wrote:
Hi all,
I have a written several routines in C++ using blitz++. These routines are called sucessively by the 'main' routine. I want to rewrite the main in python in order to gain development speed and flexibility. The routines use blitz++ matrix template library but the interfaces to the routine are very common C++ calls with basic C++ types (int, float*, short*, etc...) :
int foo( short* a, float b, float* c, int N ){ Array<short, 1> A(N); A.data() = a;
// Do some work
return ( ERROR_CODE ); }
Is it straightforward to declare these routines with pyste (or the classic boost method). Does the use of blitz in these routines (which belong to another file than the main, beginning with include of blitz.hh) complicates the process. I'm a bit lost because of the use of "templates in blitz++" versus "integration with boost".
What problems are you having exactly? And I don't know how you would pass the parameters to this function, except if you create the arrays in C++ too. If you don't care about reading the array values in Python, you can treat them as opaque pointers... check return_opaque_pointer: http://www.boost.org/libs/python/doc/v2/return_opaque_pointer.html
Also, as you can see in the example routine above, I started with a creation of a blitz array 'A' from the array 'a'. Because I'm afraid of the work to get back and forth from Numeric array in Python to Blitz++ templated Array in C++ is a complicated and big amount of work.
Perhaps, but I don't believe it would be such a pain... you can create simple wrappers that would do the job nicely; If there is a lot of functions, then perhaps you could even implement to-python and from-python conversions for Numeric arrays, as demonstrated here (for C++ standard containers): http://www.boost.org/libs/python/doc/v2/faq.html#question2
Thanks for any response on these to points, and for any suggestion or criticism on my way of handling the problem. Christophe Grimault
Hope that helps! Regards, Nicodemus.
Nicodemus wrote:
christophe grimault wrote:
Hi all,
I have a written several routines in C++ using blitz++. These routines are called sucessively by the 'main' routine. I want to rewrite the main in python in order to gain development speed and flexibility. The routines use blitz++ matrix template library but the interfaces to the routine are very common C++ calls with basic C++ types (int, float*, short*, etc...) :
int foo( short* a, float b, float* c, int N ){ Array<short, 1> A(N); A.data() = a;
// Do some work
return ( ERROR_CODE ); }
Is it straightforward to declare these routines with pyste (or the classic boost method). Does the use of blitz in these routines (which belong to another file than the main, beginning with include of blitz.hh) complicates the process. I'm a bit lost because of the use of "templates in blitz++" versus "integration with boost".
What problems are you having exactly? And I don't know how you would pass the parameters to this function, except if you create the arrays in C++ too. If you don't care about reading the array values in Python, you can treat them as opaque pointers... check return_opaque_pointer:
http://www.boost.org/libs/python/doc/v2/return_opaque_pointer.html
The arrays can be created in Python, but could be simple python C-arrays (not numeric), and all the "numerical" work is done in C++ routines.
Also, as you can see in the example routine above, I started with a creation of a blitz array 'A' from the array 'a'. Because I'm afraid of the work to get back and forth from Numeric array in Python to Blitz++ templated Array in C++ is a complicated and big amount of work.
Perhaps, but I don't believe it would be such a pain... you can create simple wrappers that would do the job nicely; If there is a lot of functions, then perhaps you could even implement to-python and from-python conversions for Numeric arrays, as demonstrated here (for C++ standard containers):
I believe this is a better way... I have play a lot more with boost to get more familiar with the possibilities. I'm just lacking time. Thanks for the hints. I will probably have new questions in a near future !
Thanks for any response on these to points, and for any suggestion or criticism on my way of handling the problem. Christophe Grimault
Hope that helps!
Regards, Nicodemus.
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
-- ------------------------------------------------------ | Christophe Grimault Tel: 02 23 23 52 59 | | NovaGrid SA http://www.novagrid.com | | fax: 02 23 23 62 32 | | mail: christophe.grimault@novagrid.com | | 2, Bd Sebastopol | | 35000 RENNES | | France | |______________________________________________________|
Hi Christophe, We've previously implemented a Numeric array wrapper, num_util, available for download at: http://www.eos.ubc.ca/research/clouds/software.html Feel free to give it a try ;-) This wrapper generally works with Boost's numeric::array class. It allows users to access certain array properties and to instantiate Python numeric array in C++. There are some example usage in the num_test.cpp file. Your example, utilizing num_util, will probably look something like... ... namespace nbpl = num_util; namespace py = boost::python; py::long_ foo( py::numeric::array a, float b, py::numeric::array c, int N ){ Array<short, 1> A(N); A.data() = (short*) nbpl::data(a); // Do some work return ( py::long_(ERROR_CODE) ); } I hope this helps. Cheers, Charles
Charles Leung writes:
Your example, utilizing num_util, will probably look something like...
... namespace nbpl = num_util; namespace py = boost::python;
py::long_ foo( py::numeric::array a, float b, py::numeric::array c, int N ){ !!> Array<short, 1> A(N); !!> A.data() = (short*) nbpl::data(a);
// Do some work
Note that Blitz has a flag to prevent it from invoking delete on a's data pointer when A goes out of scope -- the lines marked !! need to be replaced with: short* data = (short*) nbpl::data(a); Array<short, 1> A(data,shape(N),neverDeleteData); Regards, Phil
"F. Oliver Gathmann" <gathmann@cenix-bioscience.com> writes:
David Abrahams wrote:
"F. Oliver Gathmann" <gathmann@cenix-bioscience.com> writes:
template <class T> T identity(T x) { return x; } template <class T> object get_pointer_reference(T x) { object f = make_function(&identity<T>, return_value_policy<manage_new_object>() ); return f(x); }
Hmm, *nasty* formatting!
Strange - the formatting looks fine in the c++-SIG archives...
template <class T> T identity(T x) { return x; }
template <class T> object get_pointer_reference(T x) { object f = make_function( &identity<T>, return_value_policy<manage_new_object>());
return f(x); }
Hmm, looks OK to me.
which compiles fine but crashes at runtime (using gcc 3.2.2 on Linux with a recent CVS version of Boost). Any hints that might help me solving this mystery would be much appreciated!
Any hints about the condition of the program at the time of crash would be a big help in helping you. A reproducible test case would be even better.
Sorry; I thought my error was so obvious that pseudocode would be enough to find out what's wrong.
I'm not that smart. Sometimes I need to crash it myself and look at it with a debugger <wink> Also, as in this case, sometimes there's a much easier solution when you see it all in context... Here's the fix for this approach: template <class T> object get_pointer_reference(T x) { object f = make_function( &identity<T>, return_value_policy<manage_new_object>() ); return f(ptr(x)); // ^^^^ ^ } The problem is that x is copied by default when calling back into Python, so the copy being deleted by the auto_ptr was already owned by another Python object.
BOOST_PYTHON_MODULE(_problem) { class_<Foo, std::auto_ptr<Foo> >("Foo", no_init) ; class_<Bar, std::auto_ptr<Bar> >("Bar", no_init) ; def("read", &readFooOrBar); }
And in Python:
Python 2.2.2 (#1, Apr 10 2003, 23:02:08) [GCC 3.3 20030226 (prerelease) (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from _problem import * foo = read(1) bar = read(0) del foo del bar Segmentation fault
When using Nicodemus' register_ptr_to_python template like so
BOOST_PYTHON_MODULE(_problem) { class_<Foo>("Foo", no_init) ; register_ptr_to_python<std::auto_ptr<Foo> >(); class_<Bar>("Bar", no_init) ; register_ptr_to_python<std::auto_ptr<Bar> >(); def("read", &readFooOrBar); }
the situation is essentially unchanged.
No surprise. Those converters are registered in exactly the same way when auto_ptr<T> is a parameter to class_<...>. Since you have those converters registered, why not simply: object readFooOrBar(bool fooOrBar) { if (fooOrBar) { std::auto_ptr<Foo> p(new Foo()); return object(p); } else { std::auto_ptr<Bar> p(new Bar()); return object(p); } } ?? That seems much easier to me. -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Since you have those converters registered, why not simply:
object readFooOrBar(bool fooOrBar) { if (fooOrBar) { std::auto_ptr<Foo> p(new Foo()); return object(p); } else { std::auto_ptr<Bar> p(new Bar()); return object(p); } }
??
That seems much easier to me.
Doh! I thought I had tried this - works like a charme. Many thanks for pointing this out (particularly given the fact that you are on vacation this week...). Oliver -- -------------------------------------------------------------------- F. Oliver Gathmann, Ph.D. Director IT Unit Cenix BioScience GmbH Pfotenhauer Strasse 108 phone: +49 (351) 210-2735 D-01307 Dresden, Germany fax: +49 (351) 210-1309 fingerprint: 8E0E 9A64 A07E 0D1A D302 34C2 421A AE9F 4E13 A009 public key: http://www.cenix-bioscience.com/public_keys/gathmann.gpg --------------------------------------------------------------------
participants (6)
-
Charles Leung -
christophe grimault -
David Abrahams -
F. Oliver Gathmann -
Nicodemus -
Philip Austin