[C++-sig] more examples of automatic type conversion

[Albert Strasheim]

Hello all

On Thu, 03 May 2007, Roman Yakovenko wrote:

> On 5/3/07, Albert Strasheim <fullung at gmail.com> wrote:
<snip>
> >Anyway, I've more or less got pass-by-value and return-by-value working
> >using Roman's example, but I can't quite figure out how to make
> >Boost.Python convert pointer and reference arguments or return values
> >into PyObject*'s using the same scheme. Should this just work if I have
> >my automatic conversion set up correctly, or is there additional magic
> >that needs to be added? Again, it would be *very* useful if copying
> >could be avoided here.
> 
> 
> Post small example, may be I will be able to help

To recap, the idea is to wrap classes with member functions that take 
uBLAS vectors or matrices by value, (const) reference and (const) 
pointer. 

However, on the Python side one always wants to pass in a NumPy array 
which should be converted on the fly. Using an appropriate array 
adaptor, one should be able to construct a uBLAS matrix that reused the 
NumPy array's underlying data buffer without copying (except in special 
cases which I won't go into now).

Similarly, for (const) pointer or (const) reference returns, one wants 
to wrap a new NumPy array around the return value so that the Python 
side always sees a NumPy array. Again, copying should be avoided.
For the return_internal_reference case, it would be nice if the object 
"owning" the matrix being returned could be kept alive until all NumPy 
arrays that refer to the data are deleted.

In the manage_new_object case, the array_adaptor in the array being 
returned can be told to give up the ownership of the data and then 
uBLAS matrix can be deallocated immediately (effectively only 
destroying a few bits and pieces).

I expect some complications here if you have a NumPy array taking over 
ownership of a data buffer allocated in C++ code, so I guess one would 
have to look at an array_adaptor that allocates the matrix's storage as 
a NumPy array which can be "extracted" when returning to Python.

Here's some code that shows what I'm trying to using tuples instead of 
NumPy arrays and array_t instead of uBLAS matrices.

#include <boost/python.hpp>
#include <iostream>

namespace py = boost::python;

struct array_t {
    int rows;
    int cols;
    void* data;
    array_t(int rows_, int cols_) : rows(rows_), cols(cols_), data(0){}
    array_t() : rows(0), cols(0), data(0){}
};

struct array_converter {
    static void register_to_and_from_python() {
        register_from_python();
        register_to_python();
    }

    static void register_to_python() {
        py::to_python_converter<array_t, array_converter>();
    }

    static void register_from_python() {
        py::converter::registry::push_back(
            &array_converter::convertible,
            &array_converter::construct,
            py::type_id<array_t>());
    }

    static void* convertible(PyObject* obj) {
        return PyTuple_Check(obj) != 0 ? obj : 0;
    }

    static void construct(PyObject* obj, py::converter::rvalue_from_python_stage1_data* data) {
        // get the storage
        typedef py::converter::rvalue_from_python_storage<array_t> storage_t;
        storage_t* the_storage = reinterpret_cast<storage_t*>(data);
        void* memory_chunk = the_storage->storage.bytes;
        // placement new
        int rows = PyInt_AsLong(PyTuple_GET_ITEM(obj, 0));
        int cols = PyInt_AsLong(PyTuple_GET_ITEM(obj, 1));
        // XXX get data pointer here
        array_t* a = new (memory_chunk) array_t(rows, cols);
        data->convertible = memory_chunk;
    }

    static PyObject* convert(const array_t& a) {
        return py::incref(py::make_tuple(a.rows, a.cols).ptr());
    }
};

struct array_test {
    array_test() : data_(3, 4){}
    array_t value_return() const {
        return data_;
    }
    array_t const* const_pointer_return() const {
        return &data_;
    }
    array_t* pointer_return() {
        return &data_;
    }
    array_t const& const_ref_return() const {
        return data_;
    }
    array_t& ref_return() {
        return data_;
    }
    void value_arg(array_t a) {
        print(a);
    }
    void pointer_arg(array_t* a) {
        print(*a);
    }
    void const_pointer_arg(array_t const* a){}
    void ref_arg(array_t& a){}
    void const_ref_arg(array_t const& a){}
private:
    void print(array_t const& a) {
        std::cout << "(" << a.rows << ", " << a.cols << ")" << std::endl;
    }
    array_t data_;
};

BOOST_PYTHON_MODULE(pyublas)
{
	array_converter::register_to_and_from_python();
	py::class_<array_test, boost::noncopyable>("array_test")
            .def("value_return", &array_test::value_return)
            .def("const_pointer_return", &array_test::const_pointer_return, py::return_internal_reference<>())
#if 0
            .def("pointer_return", &array_test::pointer_return)
            .def("const_ref_return", &array_test::const_ref_return)
            .def("ref_return", &array_test::ref_return)
#endif
            .def("value_arg", &array_test::value_arg)
            .def("pointer_arg", &array_test::pointer_arg)
#if 0
            .def("const_pointer_arg", &array_test::const_pointer_arg)
            .def("ref_arg", &array_test::ref_arg)
            .def("const_ref_arg", &array_test::const_ref_arg)
#endif
        ;
}

And the Python code

import pyublas
print pyublas
print dir(pyublas)
at = pyublas.array_test()
print at
print at.value_return()
# XXX this doesn't work yet
print at.const_pointer_return()
a = (10, 20)
at.value_arg(a)
# XXX this doesn't work yet
at.pointer_arg(a)

The call to const_pointer_return raises the following error:

Traceback (most recent call last):
  File "test_pyublas.py", line 24, in ?
    print at.const_pointer_return()
TypeError: No Python class registered for C++ class struct array_t

The call to pointer_arg raises the following error:

Traceback (most recent call last):
  File "test_pyublas.py", line 28, in ?
    at.pointer_arg(a)
Boost.Python.ArgumentError: Python argument types in
    array_test.pointer_arg(array_test, tuple)
did not match C++ signature:
    pointer_arg(struct array_test {lvalue}, struct array_t *)

In both cases, I would like conversions to happen from the PyObjects to 
the expected C++ data structures. Is it possible to convince 
Boost.Python to do this?

Thanks for any feedback.

Cheers,

Albert