From chris.schnaufer at eagleview.com  Mon Apr  3 11:58:58 2017
From: chris.schnaufer at eagleview.com (Chris Schnaufer)
Date: Mon, 3 Apr 2017 15:58:58 +0000
Subject: [C++-sig] Convert OpenCV class reference return value
Message-ID: <CO1PR20MB0790D9E7ABAD87C320603BEDEC080@CO1PR20MB0790.namprd20.prod.outlook.com>

I have a C++ library where const references to OpenCV types are returned. Now I'm porting the interface to Python and have hit upon what looks to be an old issue; there doesn't appear to be a way to have boost.python intrinsically convert a const reference OpenCV type to a Python object (tuple or numpy). My guess is that I'm missing something, but maybe not. So I'm providing more details

Right now I'm writing wrapper classes/functions to handle the interface but would prefer to define a conversion of the types in a way that boost.python can just do the conversion without individual class/function wrappers explicitly declared for everything.

For example, if a method returns a member variable of type cv::Vec3d as 'const cv::Vec3d&', I would like to declare one "const cv:Vec3d& to Python" converter and have that be used/specified where needed; and another for Python to const cv:Vec3d&. Sans individual wrapper classes/functions.

Going over the library I've come closest with one of: 1) declare things in such a way as to get a [paraphrasing] "unable to convert pointer to const cv::Vec3d&() type" error [in as_to_python_function.hpp - so close!], or 2) use return_value_policy<copy_const_ref> which Python subsequently can't convert.

I think declaring something along the lines of "class_<cv::Vec3d>..." might work but then I'm redefining the OpenCV class which I don't like to do since my library shouldn't make OpenCV types visible in Python.

I am new to boost.python so I'm probably wrong, but it appears the best solution would be to have as_to_python_function.cpp split into reference and copy-constructor classes to handle the appropriate differences. Right now the same class handles both cases.

Any help is appreciated.

I am including a simple example:

#include <boost/python.hpp>
namespace p = boost::python;

class Vec3d
{
                public:
                  Vec3d() {v[0] = v[1] = v[2] = 0.0;}
                  virtual ~Vec3d() {}
                private:
                  double v[3];
};

class UseVec3d
{
                public:
                                UseVec3d() {}
                                UseVec3d(const Vec3d& vec) : v(vec) {}
                                virtual ~UseVec3d() {};
                                const Vec3d& get_vec() const {return v;}
                                void set_vec(const Vec3d& vec) {v = vec;}
                private:
                  Vec3d                 v;
};

BOOST_PYTHON_MODULE(test)
{
                p::class_<UseVec3d>("UseVec3d")
                                .def("get_vec", &UseVec3d::get_vec)   // Error's here
                ;
}

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20170403/64e52cee/attachment.html>

From stefan at seefeld.name  Tue Apr  4 08:28:32 2017
From: stefan at seefeld.name (Stefan Seefeld)
Date: Tue, 4 Apr 2017 08:28:32 -0400
Subject: [C++-sig] Convert OpenCV class reference return value
In-Reply-To: <CO1PR20MB0790D9E7ABAD87C320603BEDEC080@CO1PR20MB0790.namprd20.prod.outlook.com>
References: <CO1PR20MB0790D9E7ABAD87C320603BEDEC080@CO1PR20MB0790.namprd20.prod.outlook.com>
Message-ID: <f3f972af-9e95-a7ed-cbbc-080aedc25472@seefeld.name>

Hi Chris,

On 03.04.2017 11:58, Chris Schnaufer wrote:
>
> I have a C++ library where const references to OpenCV types are
> returned. Now I?m porting the interface to Python and have hit upon
> what looks to be an old issue; there doesn?t appear to be a way to
> have boost.python intrinsically convert a const reference OpenCV type
> to a Python object (tuple or numpy). My guess is that I?m missing
> something, but maybe not. So I?m providing more details
>
>  
>
> Right now I?m writing wrapper classes/functions to handle the
> interface but would prefer to define a conversion of the types in a
> way that boost.python can just do the conversion without individual
> class/function wrappers explicitly declared for everything.
>

Boost.Python was designed specifically to be fully unintrusive, i.e. to
allow Python bindings to be added to C++ APIs without requiring any
changes to the API being wrapped. Hand-written wrapper types shouldn't
be needed, as generating them is precisely what Boost.Python automatizes.

>  
>
> For example, if a method returns a member variable of type cv::Vec3d
> as ?const cv::Vec3d&?, I would like to declare one ?const cv:Vec3d& to
> Python? converter and have that be used/specified where needed; and
> another for Python to const cv:Vec3d&. Sans individual wrapper
> classes/functions.

Yes, of course !

>  Going over the library I?ve come closest with one of: 1) declare
> things in such a way as to get a [paraphrasing] ?unable to convert
> pointer to const cv::Vec3d&() type? error [in
> as_to_python_function.hpp ? so close!], or 2) use
> return_value_policy<copy_const_ref> which Python subsequently can?t
> convert.
>

I think you should spend a moment to fully understand these errors. (It
might help if you could provide the full errors here, instead of
paraphrasing them, so I can help with that. The first looks suspicious
as it suggests that you want to convert a function pointer (returning a
const reference to Vec3d). The second error simply tells you that you
need a "return-value policy" (see for example
http://boostorg.github.io/python/doc/html/tutorial/tutorial/functions.html#tutorial.functions.call_policies).

>  
>
> I think declaring something along the lines of ?class_<cv::Vec3d>??
> might work but then I?m redefining the OpenCV class which I don?t like
> to do since my library shouldn?t make OpenCV types visible in Python.
>

If you don't expose cv::Vec3d to Python then you can't expect function
arguments (and return values) to be automatically converted between
Python and C++.
(But I'm a little confused by your phrasing: Something like
`class_<cv::Vec3d>` doesn't "redefine" anything, and it doesn't make a
C++ type "visible" in Python.
Rather, it generates a Python type to/from the given C++ type can be
converted automatically. You have full control over all of its
attributes, from its name to its members and methods down to how the
Python ("wrapper") object stores the C++ object, i.e. whether by-value,
or as a shared pointer, or something else.

>  
>
> I am new to boost.python so I?m probably wrong, but it appears the
> best solution would be to have as_to_python_function.cpp split into
> reference and copy-constructor classes to handle the appropriate
> differences. Right now the same class handles both cases.
>
>  
>
> Any help is appreciated.
>
>  
>
> I am including a simple example:
>

(I'm not sure what you want to show with your example. It would be
better to make it minimally functional to demonstrate what you are
trying to do, or what doesn't work as you'd expect.)

Best,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From chris.schnaufer at eagleview.com  Tue Apr  4 16:58:57 2017
From: chris.schnaufer at eagleview.com (Chris Schnaufer)
Date: Tue, 4 Apr 2017 20:58:57 +0000
Subject: [C++-sig] Convert OpenCV class reference return value
Message-ID: <CO1PR20MB0790ADB5C40BF08140140641EC0B0@CO1PR20MB0790.namprd20.prod.outlook.com>

Thank you for the quick response. I have included the full code (at the bottom of the message) of how I think it "should" work. I've put "should" in quotes because it's based upon my understanding of how to put a converter from C++ to Python together. Compiling this produces two errors (first about no match and second for no get_pytype() declared:

/usr/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_<false, true>, const RC&, F&, TC&) [with RC = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>; F = const Vec3d& (UseVec3d::*)() const; TC = boost::python::arg_from_python<UseVec3d&>; PyObject = _object]':
/usr/include/boost/python/detail/caller.hpp:218:46:   required from 'PyObject* boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::operator()(PyObject*, PyObject*) [with F = const Vec3d& (UseVec3d::*)() const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2<const Vec3d&, UseVec3d&>; PyObject = _object]'
/usr/include/boost/python/object/py_function.hpp:38:33:   required from 'PyObject* boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller<const Vec3d& (UseVec3d::*)() const, boost::python::default_call_policies, boost::mpl::vector2<const Vec3d&, UseVec3d&> >; PyObject = _object]'
/root/RAF/RAF/Python/raf.cpp:80:1:   required from here
/usr/include/boost/python/detail/invoke.hpp:88:14: error: no match for call to '(const boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>) (const Vec3d&)'
     return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) );

and

/usr/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject* boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>; PyTypeObject = _typeobject]':
/usr/include/boost/python/detail/caller.hpp:240:19:   required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::signature() [with F = const Vec3d& (UseVec3d::*)() const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2<const Vec3d&, UseVec3d&>]'
/usr/include/boost/python/object/py_function.hpp:48:35:   required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller<const Vec3d& (UseVec3d::*)() const, boost::python::default_call_policies, boost::mpl::vector2<const Vec3d&, UseVec3d&> >]'
/root/RAF/RAF/Python/raf.cpp:80:1:   required from here
/usr/include/boost/python/detail/caller.hpp:102:98: error: 'struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>' has no member named 'get_pytype'
         return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype();

> ...
> (But I'm a little confused by your phrasing: Something like
> `class_<cv::Vec3d>` doesn't "redefine" anything, and it doesn't make a
> C++ type "visible" in Python.
> ...
Here I'm trying to understand if I have an OpenCV Vec3d type (that I've defined using p::class_<cv::Vec3d>) getting returned to Python from a function call, how does one prevent collision with the OpenCV Vec3d defined by another library (that uses OpenCV and may be using the same mechanism)? The two declarations may not be compatible despite their apparent similarities (the other library is several revisions behind on OpenCV, for example).

Here is the code with converter. I don't know if there's a preferred way of doing what I'm attempting:

#include <boost/python.hpp>
namespace p = boost::python;

class Vec3d
{
                public:
                  Vec3d() {v[0] = v[1] = v[2] = 0.0;}
                  virtual ~Vec3d() {}
                  double operator[](size_t idx) const {return v[idx];}
                  double& operator[](size_t idx) {return v[idx];}
                private:
                  double v[3];
};

class UseVec3d
{
                public:
                                UseVec3d() {}
                                UseVec3d(const Vec3d& vec) : v(vec) {}
                                virtual ~UseVec3d() {};
                                const Vec3d& get_vec() const {return v;}
                                void set_vec(const Vec3d& vec) {v = vec;}
                private:
                  Vec3d                 v;
};

struct vec3d_to_py
{
static PyObject* convert(const Vec3d& src)
{
  return Py_BuildValue("(ddd)", src[0], src[1], src[2]);
}

static PyTypeObject const *get_pytype () // Could use converter::to_python_target_type
{
  return &PyTuple_Type;
}
};

struct vec3d_from_py
{
  vec3d_from_py()
  {
    boost::python::converter::registry::push_back(
        &convertible,
        &construct,
        boost::python::type_id<Vec3d>()
        );
  }

  static void* convertible(PyObject* obj)
  {
                                return obj;
  }

  static void construct(PyObject* self, p::converter::rvalue_from_python_stage1_data* data)
  {
                typedef p::converter::rvalue_from_python_storage<Vec3d> storage_t;

                storage_t* store = reinterpret_cast<storage_t *>(data);
                void* mem = store->storage.bytes;
                new (mem) Vec3d();

                Vec3d* pv = reinterpret_cast<Vec3d *>(mem);
                PyArg_ParseTuple(self, "ddd", &((*pv)[0]), &((*pv)[1]), &((*pv)[2]));
                data->convertible = mem;
  }
};


BOOST_PYTHON_MODULE(raf)
{
                vec3d_from_py();
                p::to_python_converter<Vec3d, vec3d_to_py, true>();

                p::class_<UseVec3d>("UseVec3d")
                                .def("get_vec", &UseVec3d::get_vec)
                ;
}


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20170404/7be93b98/attachment-0001.html>

From chris.schnaufer at eagleview.com  Tue Apr  4 15:12:08 2017
From: chris.schnaufer at eagleview.com (Chris Schnaufer)
Date: Tue, 4 Apr 2017 19:12:08 +0000
Subject: [C++-sig] Convert OpenCV class reference return value
Message-ID: <CO1PR20MB079060B5C0BA7DD23B7C892AEC0B0@CO1PR20MB0790.namprd20.prod.outlook.com>

Thank you for the quick response. I have included the full code (at the bottom of the message) of how I think it "should" work. I've put "should" in quotes because it's based upon my understanding of how to put a converter from C++ to Python together. Compiling this produces two errors (first about no match and second for no get_pytype() declared:

/usr/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_<false, true>, const RC&, F&, TC&) [with RC = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>; F = const Vec3d& (UseVec3d::*)() const; TC = boost::python::arg_from_python<UseVec3d&>; PyObject = _object]':
/usr/include/boost/python/detail/caller.hpp:218:46:   required from 'PyObject* boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::operator()(PyObject*, PyObject*) [with F = const Vec3d& (UseVec3d::*)() const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2<const Vec3d&, UseVec3d&>; PyObject = _object]'
/usr/include/boost/python/object/py_function.hpp:38:33:   required from 'PyObject* boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller<const Vec3d& (UseVec3d::*)() const, boost::python::default_call_policies, boost::mpl::vector2<const Vec3d&, UseVec3d&> >; PyObject = _object]'
/root/RAF/RAF/Python/raf.cpp:80:1:   required from here
/usr/include/boost/python/detail/invoke.hpp:88:14: error: no match for call to '(const boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>) (const Vec3d&)'
     return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) );

and

/usr/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject* boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>; PyTypeObject = _typeobject]':
/usr/include/boost/python/detail/caller.hpp:240:19:   required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::signature() [with F = const Vec3d& (UseVec3d::*)() const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2<const Vec3d&, UseVec3d&>]'
/usr/include/boost/python/object/py_function.hpp:48:35:   required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller<const Vec3d& (UseVec3d::*)() const, boost::python::default_call_policies, boost::mpl::vector2<const Vec3d&, UseVec3d&> >]'
/root/RAF/RAF/Python/raf.cpp:80:1:   required from here
/usr/include/boost/python/detail/caller.hpp:102:98: error: 'struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<const Vec3d&>' has no member named 'get_pytype'
         return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype();

> ...
> (But I'm a little confused by your phrasing: Something like
> `class_<cv::Vec3d>` doesn't "redefine" anything, and it doesn't make a
> C++ type "visible" in Python.
> ...
Here I'm trying to understand if I have an OpenCV Vec3d type (that I've defined using p::class_<cv::Vec3d>) getting returned to Python from a function call, how does one prevent collision with the OpenCV Vec3d defined by another library (that uses OpenCV and may be using the same mechanism)? The two declarations may not be compatible despite their apparent similarities (the other library is several revisions behind on OpenCV, for example).

Here is the code with converter. I don't know if there's a preferred way of doing what I'm attempting:

#include <boost/python.hpp>
namespace p = boost::python;

class Vec3d
{
                public:
                  Vec3d() {v[0] = v[1] = v[2] = 0.0;}
                  virtual ~Vec3d() {}
                  double operator[](size_t idx) const {return v[idx];}
                  double& operator[](size_t idx) {return v[idx];}
                private:
                  double v[3];
};

class UseVec3d
{
                public:
                                UseVec3d() {}
                                UseVec3d(const Vec3d& vec) : v(vec) {}
                                virtual ~UseVec3d() {};
                                const Vec3d& get_vec() const {return v;}
                                void set_vec(const Vec3d& vec) {v = vec;}
                private:
                  Vec3d                 v;
};

struct vec3d_to_py
{
static PyObject* convert(const Vec3d& src)
{
  return Py_BuildValue("(ddd)", src[0], src[1], src[2]);
}

static PyTypeObject const *get_pytype () // Could use converter::to_python_target_type
{
  return &PyTuple_Type;
}
};

struct vec3d_from_py
{
  vec3d_from_py()
  {
    boost::python::converter::registry::push_back(
        &convertible,
        &construct,
        boost::python::type_id<Vec3d>()
        );
  }

  static void* convertible(PyObject* obj)
  {
                                return obj;
  }

  static void construct(PyObject* self, p::converter::rvalue_from_python_stage1_data* data)
  {
                typedef p::converter::rvalue_from_python_storage<Vec3d> storage_t;

                storage_t* store = reinterpret_cast<storage_t *>(data);
                void* mem = store->storage.bytes;
                new (mem) Vec3d();

                Vec3d* pv = reinterpret_cast<Vec3d *>(mem);
                PyArg_ParseTuple(self, "ddd", &((*pv)[0]), &((*pv)[1]), &((*pv)[2]));
                data->convertible = mem;
  }
};


BOOST_PYTHON_MODULE(raf)
{
                vec3d_from_py();
                p::to_python_converter<Vec3d, vec3d_to_py, true>();

                p::class_<UseVec3d>("UseVec3d")
                                .def("get_vec", &UseVec3d::get_vec)
                ;
}


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20170404/7415a594/attachment-0001.html>

From chris.schnaufer at eagleview.com  Wed Apr  5 09:40:47 2017
From: chris.schnaufer at eagleview.com (Chris Schnaufer)
Date: Wed, 5 Apr 2017 13:40:47 +0000
Subject: [C++-sig] Convert OpenCV class reference return value
Message-ID: <SN2PR20MB0797B888B82E2438BCBE1723EC0A0@SN2PR20MB0797.namprd20.prod.outlook.com>

I think I have been able to answer my own question. As Stephan had pointed out, it came down to the return_value_policy.

By defining the conversions (as in my last post) and using return_value_policy<return_by_value> I was able to have the translation handled appropriately when returning const Vec3d& types. The conversions also handled the case with a const Vec3d& as a parameter (and all other Vec3d references).

Many thanks to Stephan for pointing me in the right direction.

Here is the test in its final state:

#include <boost/python.hpp>
namespace p = boost::python;

class Vec3d {
                public:
                  Vec3d() {v[0] = v[1] = v[2] = 0.0;}
                  virtual ~Vec3d() {}
                  double operator[](size_t idx) const {return v[idx];}
                  double& operator[](size_t idx) {return v[idx];}
                private:
                  double v[3];
};

class UseVec3d {
                public:
                                UseVec3d() {}
                                UseVec3d(const Vec3d& vec) : v(vec) {}
                                virtual ~UseVec3d() {};
                                const Vec3d& get_vec() const {return v;}
                                void set_vec(const Vec3d& vec) {v = vec;}
                private:
                  Vec3d                 v;
};

struct vec3d_to_py {
static PyObject* convert(const Vec3d& src) {
  return Py_BuildValue("(ddd)", src[0], src[1], src[2]);
}

static PyTypeObject const *get_pytype ()  {
  return &PyTuple_Type;
}
};

struct vec3d_from_py  {
  vec3d_from_py()   {
    boost::python::converter::registry::push_back(
        &convertible,
        &construct,
        boost::python::type_id<Vec3d>()
        );
  }

  static void* convertible(PyObject* obj)   {
                                return obj;
  }

  static void construct(PyObject* self, p::converter::rvalue_from_python_stage1_data* data)   {
                typedef p::converter::rvalue_from_python_storage<Vec3d> storage_t;

                storage_t* store = reinterpret_cast<storage_t *>(data);
                void* mem = store->storage.bytes;
                new (mem) Vec3d();

                Vec3d* pv = reinterpret_cast<Vec3d *>(mem);
                PyArg_ParseTuple(self, "ddd", &((*pv)[0]), &((*pv)[1]), &((*pv)[2]));
                data->convertible = mem;
  }
};


BOOST_PYTHON_MODULE(raf)
{
                vec3d_from_py();
                p::to_python_converter<Vec3d, vec3d_to_py, true>();

                p::class_<UseVec3d>("UseVec3d")
                                .def(p::init<Vec3d>())
                                .def("get_vec", &UseVec3d::get_vec, p::return_value_policy<p::return_by_value>())
                ;
}

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20170405/8b2ab355/attachment.html>