Re: [C++-sig] [python] python + phoenix
I have some code that takes member function pointers of the form R Obj::*( A1, A2, A3, ..., An ) and converts them to a function object Q with the following signature: Rnew Q( Obj&, B1, B2, ..., Bn ) where Rnew = result_conversion_metafunction<R>::type Bi = arg_conversion_metafunction<Ai>::type and Q is exposed via boost.python as a member of Obj.
For example, Ai could be a fixed-point number, with Bi being a double so that the python side does not know anything about fixed-point numbers.
I gather that custom converters aren't preferred because:
1. It is a hassle to talk directly to the underlying raw PyObject pointers and manage storage in the converter methods
2. The converted type still leaks out to python in docstrings
Are there others? Or am I completely off?
Neither of the above is a big issue. In fact, python-side knowledge of the converted type would even be mildly useful. The main problem is the number of types: my_fixed_point_type< int bit_width, // 1 to 63 int binary_pt_location, // -63 to 63 bool signed, property_tag // policies for rounding, etc. > std::complex< my_fixed_point_type<...> > boost::numeric::ublas::array< ... > boost::numeric::ublas::matrix< ... > There are simply far too many types to create converters for and to register explicitly. Therefore, I use inline functions/functors to do all the conversions on the C++ side, and python knows only about real and complex doubles.
An instance of Q would convert eh floating point numbers passed from the python side into fixed-point numbers, call the member function, and convert the returned value to a double for use on the python side.
For these "converters", metafunctions from boost.function_types are used to obtain the mpl::vector specifying result type and argument types. Extending the technique above to function objects which take Obj* as their first argument, I have a protocol which relies on the presence of a typedef called 'components' in the function object so that I can use the converter when exposing via boost.python:
struct my_functor { typedef mpl::vector<R, Obj*, A1, A2, A3, A4> components; R operator()( Obj*, A1, A2, A3, A4 ) { ... } };
To be sure I'm following you, correct me:
* for each member function signature that you wrap, you have a matching my_functor that calls it * my_functor does nothing but call the member function it wraps * my_converter<my_functor>::type does the conversion from Rnew (B1, B2, ... Bn) to R (A1, A2, ...An) and calls my_functor.
Have I got it?
Yes.
It is all very interesting, and to a certain degree duplicates some functionality in detail/caller.hpp. How do you generate these my_functors... preprocessor? fusion?
Mostly boost.preprocessor (with some template metaprogramming using mpl). Fusion doesn't work very well for me since the effort needed to get my functors into a form usable from fusion is greater than just coding my own invocation functions via boost.preprocessor; pseudo-code: template <typename Func> struct my_converter { Func f_; typedef typename ft::components<F>::type components; typedef typename arg_converter<components>::type arg_types; typedef typename result_converter<components>::type result_type; // Shown for non-void return with one argument; can be generalized // using enable_if for void types and using boost.preprocessor for // different numbers of arguments result_type operator()( typename at_c<1,arg_types>::type a1 ) { return convert_result<result_type>( f_( convert_arg<typename at_c<1,components>::type>( a1 ) ) ); } }; template <typename Func> my_converter<Func> make_converted_func( Func f ) { return my_converter<Func>( f ); }
I'm wondering if there isn't motivation here to cleanly integrate a general facility for additional c++-side type conversions. The following came to mind, which imitates a function type style that boost::proto uses extensively:
struct hidden_type; // python shouldn't see this at all
struct int_2_hidden // converts python-side 'int' to hidden { typedef hidden_type result_type;
hidden_type operator()(int) const; };
// fnobj takes a hidden, doesn't know it is wrapped in python struct fnobj { typedef void result_type; void operator()(float, hidden_type); };
def("myfunctor", as<void(float, int_to_hidden(int))>(fnobj()));
where int_to_hidden(int) is a function type (not a function call, but it later becomes a function call), indicating that what python passes should be converted to int, then the int converted to hidden_type via an instance of int_to_hidden, then the hidden_type passed to the underlying instance of fnobj.
I realize this doesn't involve using the mpl::vectors you've already calculated, just throwing it out there.
Such a general facility would probably be better than my code and would be greatly useful for me, but are there enough people out there with this use case? I don't care about using mpl::vector if there is a more general solution.
// my_converter uses the components typedef typedef typename my_converter<my_functor>::type Q;
so Q could have a nested typedef (note I say Rnew, not R):
typedef mpl::vector<Rnew, Obj*, B1, B2, B3, B4> components;
or use that in combination with function_types to synthesize
typedef Rnew(Obj*, B1, B2, B3, B4) signature;
Yes. Regards, Ravi
I'm wondering if there isn't motivation here to cleanly integrate a general facility for additional c++-side type conversions.
I got distracted with boost.cmake stuff for a while but just got some things working. Simple example: // type hidden from python struct Hidden { double value; }; // // c++ interface that uses hidden type // std::string takes_hidden(Hidden h) { return std::string("Got an Hidden containing ") + boost::lexical_cast<std::string>(h.value); } // // converter object // struct dbl_to_Hidden { typedef Hidden result_type; result_type operator()(double v) const { Hidden h; h.value = v; return a; } }; // the module BOOST_PYTHON_MODULE(udconversions_ext) { def("takes_hidden", as<std::string(dbl_to_Hidden(double))>(&takes_hidden)); } ^^^^^^^^^^^^^^^^^^^^^ And the python:
from udconversions_ext import * takes_a(13) 'Got an A containing 13'
Here's another contrived example: // Here we have a class that takes optional<int> arguments that // python ought not know about: struct S { void set(optional<int> v) { i = v; } void add(boost::optional<int> oi) { if (i && oi) (*i) += (*oi); } std::string show() { if (i) return boost::lexical_cast<std::string>(*i); else return "<unset>"; } optional<int> i; }; // And a converter function object that converts a python::object // to an optional<T> (holding a T) if the object is // convertible to T, otherwise it returns an empty optional<T>: template <typename T> struct to_optional { typedef optional<T> result_type; result_type operator()(const bp::object& o) { bp::extract<T> ex(o); if (ex.check()) return optional<T>(ex()); else return optional<T>(); } }; BOOST_PYTHON_MODULE(opt_ext) { bp::class_<S>("S") .def("set", bp::as<void(S*, to_optional<int>(bp::object))>(&S::set)) .def("add", bp::as<void(S*, to_optional<int>(bp::object))>(&S::add)) .def("show", &S::show) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ; }; And the result:
from opt_ext import * s = S() s.set(13) s.show() '13' s.set(None) s.show() '<unset>' s.set(None) s.add(13) s.show() '<unset>' s.set(13) s.add(13) s.show() '26'
There are plenty of issues yet. Currently, the converter type still leaks out to python in the signature: class S(Boost.Python.instance) | add(...) | add( (S)arg1, (object)arg2) -> None : | | C++ signature : | void add(S*,to_optional<int> (*)(boost::python::api::object)) converters won't nest: def("foo", as<returnvalue(arg1, conv1(conv2(arg2)))>(&somefn)); // nah I dont have a mechanism for hooking a user defined converter into the *return* type. I'm going to have to think about that one a bit more. specifying a function type, whose return value is a pointer to a function, is kinda nasty anyway: def("foo", as<(converter(*)(int))(bool, converter(float))>(&fn)); I'm just not sure it gets one anything other than terseness. Maybe a special return value policy is in order. Also, there isn't a way to use an object's constructor as a converter, That is, I'd like to support this syntax: struct X { X(double); }; void takes_X(X x); def("takes_x", as<void(X(double))>(&takes_x)); But I think this involves adding some scaffolding so that boost.python can differentiate between what in proto-ese is a Callable Transform (what now works) and an Object Transform (convert via constructor). -t
On Friday 06 November 2009 14:52:49 troy d. straszheim wrote:
Currently, the converter type still leaks out to python in the signature:
class S(Boost.Python.instance) | add(...) | add( (S)arg1, (object)arg2) -> None : | | C++ signature : | void add(S*,to_optional<int> (*)(boost::python::api::object))
converters won't nest:
def("foo", as<returnvalue(arg1, conv1(conv2(arg2)))>(&somefn)); // nah
I dont have a mechanism for hooking a user defined converter into the return type. I'm going to have to think about that one a bit more.
Did you have any ideas on this, Troy? This is the last bit I think I'd need in order to replace my code with yours. Regards, Ravi
Ravi wrote:
On Friday 06 November 2009 14:52:49 troy d. straszheim wrote:
Currently, the converter type still leaks out to python in the signature:
class S(Boost.Python.instance) | add(...) | add( (S)arg1, (object)arg2) -> None : | | C++ signature : | void add(S*,to_optional<int> (*)(boost::python::api::object))
converters won't nest:
def("foo", as<returnvalue(arg1, conv1(conv2(arg2)))>(&somefn)); // nah
I dont have a mechanism for hooking a user defined converter into the return type. I'm going to have to think about that one a bit more.
Did you have any ideas on this, Troy? This is the last bit I think I'd need in order to replace my code with yours.
Hi Ravi, I haven't looked, I got pulled away. I hope to be back to this soon though. -t
participants (3)
-
lists_ravi@lavabit.com -
Ravi -
troy d. straszheim