[C++-sig] V2: to_python/from_python questions

[David Abrahams]

----- Original Message -----
From: "Pearu Peterson" <pearu at cens.ioc.ee>

> Hi Dave,

> > The upshot is that you can ignore coerce and just overload your
__add__
> > and __radd__ method to work with floats.
>
> Instead of defining a punch of add methods like
>
> const GiNaC::ex ex_add(const GiNaC::ex&,double);
> const GiNaC::ex ex_add(const GiNaC::ex&,int);
> const GiNaC::ex ex_add(const GiNaC::ex&,const GiNaC::ex&);
> const GiNaC::ex ex_add(const GiNaC::ex&,const GiNaC::basic&);
> const GiNaC::ex ex_add(const GiNaC::ex&,complex);
>
> I am thinking of having a single one
>
> const GiNaC::ex ex_add(const GiNaC::ex&,PyObject*);
>
> It ignores V2 overloading stuff. Is there any downside of doing this
> except having my own bugs?

Not that I can think of offhand.

> And I am a bit confused about to_python and from_python that, I think,
> understand from V1 perspective. Is there a simple correspondence
between
> these facilities in V1 and V2?

Unfortunately not. The v2 facilities are more sophisticated, and thus
(unfortunately) more complicated.

> In V2 I have used, for example
>
>   boost::python::to_python_value<const GiNaC::ex&>()(e)
>
> to construct a PyObject of e. Is this the correct way?

Not exactly. I haven't provided a nice mechanism for users yet. In v2,
we don't use exceptions to implement overloading anymore, which means
that conversions used in wrapped function calls explicitly check whether
the conversion can succeed (basically, it tests whether such a
conversion has been registered).

So the first problem is that you need to call the to_python_value<>
object's convertible() member function and "throw a Python exception" if
it returns false. That means, for the time being, setting the Python
exception information through the usual Python "C" API, and calling
boost::python::throw_error_already_set(). You can see the checking in
action in boost/python/preprocessed/returning_non_void.hpp, though
there's no exception thrown since we might be doing overload resolution:

...
    if(!cr.convertible())return 0;
    if(!policies->precall(args_))return 0;
    PyObject*result=cr(((
                       c0(PyTuple_GET_ITEM(args_,0)))->*pmf)());
...

Where cr might be an instance of to_python_value<>, depending on the
type of pmf and the policies class used.

The second problem is that how references are converted depends on
context. You can see this in boost/python/preprocessed/call.hpp:

...
    converter::callback_from_python<R>converter;
    return converter(
        PyEval_CallFunction(
            callable,const_cast<char*>("(" "O" ")")
            ,converter::callback_to_python<A0>(a0).get()));
...

This is code used to call *into* Python, so callback_to_python<> acts
differently from to_python_value. First, it /does/ throw C++ exceptions
when no converter can be found. Secondly, callback_from_python's
behavior w.r.t. references and pointers is determined by the logic in
select_callback_to_python<> (see boost/python/converter/callback.hpp).
The rationale is outlined in libs/python/doc/v2/callbacks.txt.

After all of this, maybe you can see why I haven't settled on a simple
mechanism for general use yet. I'm not sure what the best behavior for
users is, but I'm inclined to think that callback_to_python<T> is pretty
close to being the right mechanism. Perhaps just wrapping it in a simple
function is enough.

> There is also to_python_indirect and to_python_converter. When one
should
> use them and how?

these are detailed in libs/python/doc/v2/to_python_indirect.html and
libs/python/doc/v2/to_python_converter.html; does that material answer
your question?

> Also I have used
>
>   boost::python::converter::reference_from_python<const
GiNaC::basic&>(obj)(obj)
>
> to get GiNaC::basic& from PyObject obj. This looks clumsy (using two
> times obj). Is there a more appropriate way for getting C++ objects
for
> Python objects?

I guess I would tend to direct you toward
converter::callback_from_python<>, since that will throw a C++ exception
upon failure. It seems to match user needs pretty well.

> I see that there are also from_python, value_from_python.
> How and when they should be used?

I'm not sure from_python is directly appropriate for users. from_python
hides a large quantity of converter selection logic (see
select_from_python in boost/python/converter/from_python.hpp), and you
can see examples of its use in
boost/python/preprocessed/returning_non_void.hpp.

I don't see value_from_python anywhere (?)

I think that answering this post is pushing me in the direction of
renaming the existing to_python/from_python so the names can be used for
the user-friendly functions based on the callback_* functionality.

> It took some time for me to figure out how to use this
> reference_from_python and I am still not sure that I am using it
> correctly.

I hope the above helps, and isn't too much to wade through.

-Dave