[Cython] Automatic C++ conversions

[Robert Bradshaw]

On Thu, Jun 28, 2012 at 5:10 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 28.06.2012 12:07:
>> On Thu, Jun 28, 2012 at 2:54 AM, Stefan Behnel wrote:
>>> Robert Bradshaw, 28.06.2012 10:59:
>>>> I've been looking how painful it is to constantly convert between
>>>> Python objects and string in C++.
>>>
>>> You mean std::string (as I think it's called)? Can't we just special case
>>> that in the same way that we special case char* and friends?
>>
>> Yes, we could.
>
> Then I think it makes sense to do that. Basically, the std::string type
> would set its is_string flag and then we need the actual coercion code for it.

I just leveraged the object <-> char* conversion in our utility code.

>> If we do that it'd make sense to special case list and
>> vector and pair and and map and set as well, though perhaps those are
>> special enough to hard code them, and it makes the language simpler to
>> not have more special methods.
>
> Ok, then it's
>
> std::string <=> bytes
> std::vector <=> list
> std::map <=> dict
> std::set <=> set
>
> Potentially also:
>
> std::pair => tuple (maybe 2-tuple => std::pair with a runtime length test?)

I implemented

std::string <=> bytes
std::map <=> dict
iterable => std::vector => list
iterable => std::list => list
iterable => std::set => set
2-iterable => std::pair => 2-tuple

> What about allowing list(<C++ iterator>) etc.? As long as the item type can
> be coerced at compile time, this should be doable:
>
> <C++ iterator> => Python iterator
>
> and it would even be easy to implement in Cython code using a generator
> function.

The tricky part is memory management; one would have to make sure the
iterable is valid as long as the Python object is around (whereas its
usually bound to the lifetime of its container).

Even more useful, however, would be supporting the "for ... in" syntax
for C++ iterators, which I plan to implement soon if no one beats me
to it.

> The other direction (Python iterator => <C++ iterator>) would be
> trickier but could also be made to work when the C++ item type on the LHS
> of the assignment that triggers the coercion is known at compile time.

Yes, this would be actually probably be easier.

> We might want to look for a way to make these coercions a "thing" in the
> code (maybe through a registry or dedicated class) rather than adding
> special casing code everywhere.

Perhaps, but that's a rather vague idea with less immediate benefit.
The list of obvious cases to support turns out to be rather clear and
small. (We already have the from/to_py_function framework.)

> I think a CEP would be a good way to specify the above coercions.

Though user-extensibility would be a larger topic and certainly
deserve a CEP, though I'm not claiming we want to support it.

> I also think that this is large enough a feature to openly ask for sponsorship.

That depends on the CEP.

- Robert