[C++-sig] Re: conversion of python dictionary to shared_ptrs

[David Abrahams]

Raoul Gough <RaoulGough at yahoo.co.uk> writes:

> David Abrahams <dave at boost-consulting.com> writes:
>
>> "aashish" <aashish at vrac.iastate.edu> writes:
>>
>>> Hi, 
>>>
>>> I need to pass a dictionary from python to C++ function which is defined as 
>>>
>>> void setVariables(ValueMapPtr variables);
>>>
>>> where I have typedef ValueMapPtr as 
>>>
>>> //a shared ptr of a value map
>>> typedef boost::shared_ptr< ValueMap > ValueMapPtr;
>>>
>>> // And 
>>> typedef std::map< std::string, std::vector<double> > ValueMap;
>>>
>>>
>>> Is it possible?? 
>>
>> Yes.
>>
>>> Do I need to use register_ptr_to_python ? 
>>
>> No; if it's truly a Python dictionary on the Python side, you need to
>> build a custom rvalue from-python converter, per
>>
>> http://www.boost.org/libs/python/doc/v2/faq.html#question2
>> http://article.gmane.org/gmane.comp.python.c%2B%2B/2161
>>
>> I wish it were easier; sorry!
>
> There is an interesting trade off here, which I've thought about on a
> few occasions. If you want to share a container between Python and C++
> code, I believe you have three choices:
>
> 1. Use a Python container and access it from C++ using the Python/C
>    APIs like (e.g.) PyDict_GetItem
>
> 2. Use a C++ container and access it from Python using (e.g.) the
>    indexing suite.
>
> 3. Use native containers on each side and convert from/to the
>    different types at the interface.
>
> In terms of performance, which one is appropriate depends on the
> application, where most of the processing takes place and probably
> also on the size of the container.
>
> Option 1. means dealing with PyObjects from C++, instead of having the
> compiler know the static types of the elements. It introduces
> overheads on the C++ side, so isn't much good if the heavy processing
> is done here.
>
> Option 2. means the C++ code knows the static types of the elements,
> which gives you the fastest possible code for what you do in C++. It
> introduces overheads on the Python side, since every operation on the
> container requires a call through the Boost.Python code. 

I'm not convinced these overheads you're referring to are real.
Accessing a native Python dict or list from Python requires a call
through Python's 'C' code that's much like Boost.Python's C++ code.
There's probably one extra level of function indirection, but I doubt
that counts for much in most cases.

> For example, this wouldn't make much sense if you spend all the time
> filling the container from Python and then do one simple operation
> on it in C++.

Probably the reason, though, is the amount of effort you expend
writing wrapping code in that case.

> Option 3. means converting the whole container for each call (either
> from a C++ container to Python or vice versa). I guess it also
> introduces a problem if you want code on the non-native side to modify
> the container, since you'd then have to convert the container back and
> replace the original.

Right.

> As for which is the easiest to use, I would say probably option 2
> would be, except that I haven't yet documented the new indexing suite
> properly, and I don't think the original suite has much std::map
> support.

Mmm, I'm eagerly awaiting those docs.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com