[Numpy-discussion] Anyone have a well-tested SWIG-based C++ STL valarray <=> numpy.array typemap to share?

[Charles R Harris]

On 9/10/07, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> Thanks for you input Xavier.
>
> Xavier Gnata wrote:
> > std:valarray are quite strange containers because they are not well
> > integrated in the STL.
>
> > I always use vector when I have to deal with arrays.
>
> > ps : There are no real performance differences between vector and
> > valarray (in my use cases...)
>
> Probably not for our use cases either. However, we're not doing a lot of
> STL either, so I'm not sure there is any downside to valarray. It looks
> like neither one supports any kind of "view" semantics, so for the
> purposes of numpy array wrapping, they really aren't any different.


I think that the originator of valarray saying it was misguided might be
considered a downside.

Now on to your other issue:

HOPKINS:

> As I see it, *valarray* was designed by BS [Bjarne Stroustrup]

BUDGE:

No, Bjarne should not be blamed for *valarray*. If any one person should
be blamed for *valarray*, it's me. That's why I tend to sign my postings
to this reflector as Kent "Frankenstein" Budge. -- Oops, I guess I've
tipped my hand already ...

HOPKINS:
exactly for this purpose; to
> allow aggressive optimisation by compilers in various ways e.g. because
> it is guaranteed to be alias-free. In theory, this should make it
> potentially faster than ANSI C until the restrict keyword is
> implemented, should it not?

BUDGE:

Yes, that was the idea. I wanted *valarray* to provide a mechanism for
expressing any one-loop operation as a single expression which could be
highly optimized. I also had a vague notion that nested-loop
expressions could in turn be expressed as single expressions on nested
template classes, but the experience just wasn't there to see all the
implications -- you should know that *valarray* was originally *not* a
class template, but a pair of classes based on int and double for which
there *was* some experience. This is because implementations of
templates were not widely available at the time *valarray* was first
proposed.

HOPKINS:

>
> However, there does seem to be a body of (informed?) opinion that
> *valarray* is 'broken' or at least not working well enough to be worth the
> effort.

BUDGE:

Yes, that is probably a fair assessment, and probably the assessment of
the vast majority (though not a unanimous view.) It has become fairly
clear that the aliasing guarantees provided by *valarray* simply aren't
strong enough to be useful, and that the market incentives for taking
advantage of them aren't strong enough even if they were.

*valarray* was written at a time when vector supercomputers were still
the sexy leading edge of computing. Unfortunately, the best
optimization strategy for a vector supercomputer is almost the opposite
of the best optimization strategy for modern hierarchical-storage
machines. On a vector supercomputer, you wanted to run the largest
possible data set past each instruction, so that the vector pipeline
remained full. On a hierarchical-memory machine, you want to throw the
largest possible number of instructions at a particular working set of
data, so that you keep your data in cache (or paged into memory or on
processor, depending on which level of the memory hierarchy you are
concerned with.) *valarray* might conceivably have been helpful for
optimization on vector machines, because it assumes operations are best
treated atomically. It's hopeless on modern machines.

In any case, it's not at all clear that *valarray* is the right philosophy.
*valarray* was meant to replace loops with expressions, but *STL* has shown
that loops can be beautiful.

HOPKINS:

> Does anyone here have experience of the runtime speed of a
> carefully constructed (i.e. avoiding the C++ efficiency pitfalls)
> *valarray*-based set of BLAS, LAPACK or whatever?

BUDGE:

I'm not aware of anyone attempting to implement BLAS or LAPACK using
*valarray*.

HOPKINS:

If so, are you using
> slices, iterators etc and doing it fully *STL*-style or doing access with
> more traditional Fortran-style (i,j) operators.
>
> Compiler writers; are you taking full advantage of *valarray*? Does it
> offer what it suggests?

BUDGE:

Arch Robison can answer this better than I, but the short answer to both
questions is No.

HOPKINS:

> Numerical methods people; have you compared it (on a level playing
> field) with C, Fortran, C++ template-based libraries (Blitz, MTL,
> POOMA)?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20070916/b9e6219b/attachment.html>