[Numpy-discussion] Proposed Roadmap Overview

[Mark Wiebe]

On Sun, Feb 19, 2012 at 3:16 AM, David Cournapeau <cournape at gmail.com>wrote:

> On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> > On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau <cournape at gmail.com>
> > wrote:
> >>
> >> On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >>
> >> >
> >> > Well, we already have code obfuscation (DOUBLE_your_pleasure,
> >> > FLOAT_your_boat), so we might as well let the compiler handle it.
> >>
> >> Yes, those are not great, but on the other hand, it is not that a
> >> fundamental issue IMO.
> >>
> >> Iterators as we have it in NumPy is something that is clearly limited
> >> by C. Writing the neighborhood iterator is the only case where I
> >> really felt that C++ *could* be a significant improvement. I use
> >> *could* because writing iterator in C++ is hard, and will be much
> >> harder to read (I find both boost and STL - e.g. stlport -- iterators
> >> to be close to write-only code). But there is the question on how you
> >> can make C++-based iterators available in C. I would be interested in
> >> a simple example of how this could be done, ignoring all the other
> >> issues (portability, exception, etc…).
> >>
> >> The STL is also potentially compelling, but that's where we go into my
> >> "beware of the dragons" area of C++. Portability loss, compilation
> >> time increase and warts are significant there.
> >> scipy.sparse.sparsetools has been a source of issues that was quite
> >> high compared to its proportion of scipy amount code (we *do* have
> >> some hard-won experience on C++-related issues).
> >
> >
> > These standard library issues were definitely valid 10 years ago, but all
> > the major C++ compilers have great C++98 support now.
>
> STL varies significantly between platforms, I believe it is still the
> case today. Do you know the status of the STL on bluegen, on small
> devices ? We unfortunately cannot restrict ourselves to one well known
> implementation (e.g. STLPort).


Is there anyone who uses a blue gene or small device which needs up-to-date
numpy support, that I could talk to directly? We really need a list of
supported platforms on the numpy wiki we can refer to when discussing this
stuff, it all seems very nebulous to me.

> Is there a specific
> > target platform/compiler combination you're thinking of where we can do
> > tests on this? I don't believe the compile times are as bad as many
> people
> > suspect, can you give some simple examples of things we might do in NumPy
> > you expect to compile slower in C++ vs C?
>
> Switching from gcc to g++ on the same codebase should not change much
> compilation times. We should test, but that's not what worries me.
> What worries me is when we start using C++ specific code, STL and co.
> Today, scipy.sparse.sparsetools takes half of the build time  of the
> whole scipy, and it does not even use fancy features. It also takes Gb
> of ram when building in parallel.
>

Particular styles of using templates can cause this, yes. To properly do
this kind of advanced C++ library work, it's important to think about the
big-O notation behavior of your template instantiations, not just the big-O
notation of run-time. C++ templates have a turing-complete language (which
is said to be quite similar to haskell, but spelled vastly different)
running at compile time in them. This is what gives template
meta-programming in C++ great power, but since templates weren't designed
for this style of programming originally, template meta-programming is not
very easy.

Cheers,
Mark


>
> David
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120219/3a3fa493/attachment.html>