[Cython] aritmetic with arrays in Cython

[Ian Henriksen]

I'd like to work on it. That was why I asked. It's been bothering me for
some time that we don't have this feature yet. This will take me some time
though since I'm not at all familiar with Cython's internals. I've used it
in relatively small settings for speeding up array operations and wrapping
external functions, but I've never had occasion to dig through much of what
it does internally. Some help navigating Cython's internals would be
greatly appreciated.

I'll also have to take some time to become more familiar with eigen itself.
Most of my work has used numpy arrays.

Eigen seems to be ideal for this since it allows us to offload the
expression analysis to another package where it is already
well-implemented. It looks like they have an array class for n-dimensional
array operations as well. My guess is that that would allow a fairly
natural translation from memory views to eigen arrays.

For now, I'll work on learning some of the details of how to use eigen.
Where would be a good place to start from there?

Would it be more helpful for me to start working on Cython bindings for
eigen, or to work on adding this directly as a part of Cython? Where should
I look in Cython's source code to do that? What kinds of modifications
would be necessary?

Thanks!

-Ian


On Fri, Aug 8, 2014 at 4:28 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Matěj Laitl schrieb am 08.08.2014 um 11:07:
> > On Friday 08 of August 2014 07:05:08 Stefan Behnel wrote:
> >> Eigen can apparently do these things already:
> >>
> >> http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html
> >>
> >> It might not be all that much work to integrate this with Cython to
> enable
> >> syntax support. Or have some kind of preprocessor, but I guess that's
> not
> >> going to be simpler than have it in Cython as an optional AST transform.
> >> Basically, whenever an arithmetic expression consists of a mixture of
> >> memory views and (optionally) scalars, it could be mapped to the kind of
> >> code that Ceygen uses, just at an arbitrarily complex expression length.
> >
> > Exactly! That was one of my ideas when I decided to create Ceygen, but I
> > resorted to take the easier route and made uglier static API because I
> didn't
> > (and still don't) have enough knowledge of Cython internals and Mark's
> "array
> > expressions" still looked like they are going to be merged.
>
> There are a couple of points in favour of Eigen/Ceygen. It's a way simpler
> approach than moving the entire vectorised code generator into Cython. And
> it lets Eigen do the hard work, leaving only the interfacing part to
> Cython. Basically, both tools can do what they're good at. We're always
> happy that there's a C compiler cleaning up after us to which we can
> delegate the low level optimisation and tuning stuff. The same would apply
> to Eigen. It's a much more cythonic approach than what's implemented in the
> pie-in-the-sky array expressions branch.
>
> It requires a C++ compiler, but I think that's an acceptable dependency
> these days in places where array computation performance matters. People
> even accept a dependency on huge tool stacks like Numba/LLVM for the same
> kind of tasks. An array expression syntax is just so much simpler than
> writing dumb nested loops.
>
>
> > Unfortunately, I no longer work on the project (my master's thesis) that
> > triggered work on Ceygen, and now I can invest only little maintenance
> work to
> > it.
>
> I don't see me being the one to implement this, either. :)
>
>
> > *However*, if such project is found worth to tacking by Cython devs, I
> can
> > imagine in my wild dreams that there may be a student at my former
> university
> > interested in doing it as their project. (this is rather optimistic
> statement,
> > please don't get too excited yet)
>
> I'm sure there are many people who would benefit from this. Some of them
> might even be able to provide funding, if someone wants to pick up this
> idea. But past experience shows that we shouldn't count on this either.
> It's definitely a nice project to tackle for someone who likes coding.
>
> Stefan
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20140808/e5eef7c5/attachment.html>