[Numpy-discussion] Proposed Roadmap Overview

[David Cournapeau]

On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Sat, Feb 18, 2012 at 4:24 PM, David Cournapeau <cournape at gmail.com>
> wrote:
>>
>> On Sat, Feb 18, 2012 at 9:40 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>>
>> >
>> > Well, we already have code obfuscation (DOUBLE_your_pleasure,
>> > FLOAT_your_boat), so we might as well let the compiler handle it.
>>
>> Yes, those are not great, but on the other hand, it is not that a
>> fundamental issue IMO.
>>
>> Iterators as we have it in NumPy is something that is clearly limited
>> by C. Writing the neighborhood iterator is the only case where I
>> really felt that C++ *could* be a significant improvement. I use
>> *could* because writing iterator in C++ is hard, and will be much
>> harder to read (I find both boost and STL - e.g. stlport -- iterators
>> to be close to write-only code). But there is the question on how you
>> can make C++-based iterators available in C. I would be interested in
>> a simple example of how this could be done, ignoring all the other
>> issues (portability, exception, etc…).
>>
>> The STL is also potentially compelling, but that's where we go into my
>> "beware of the dragons" area of C++. Portability loss, compilation
>> time increase and warts are significant there.
>> scipy.sparse.sparsetools has been a source of issues that was quite
>> high compared to its proportion of scipy amount code (we *do* have
>> some hard-won experience on C++-related issues).
>
>
> These standard library issues were definitely valid 10 years ago, but all
> the major C++ compilers have great C++98 support now.

STL varies significantly between platforms, I believe it is still the
case today. Do you know the status of the STL on bluegen, on small
devices ? We unfortunately cannot restrict ourselves to one well known
implementation (e.g. STLPort).

> Is there a specific
> target platform/compiler combination you're thinking of where we can do
> tests on this? I don't believe the compile times are as bad as many people
> suspect, can you give some simple examples of things we might do in NumPy
> you expect to compile slower in C++ vs C?

Switching from gcc to g++ on the same codebase should not change much
compilation times. We should test, but that's not what worries me.
What worries me is when we start using C++ specific code, STL and co.
Today, scipy.sparse.sparsetools takes half of the build time  of the
whole scipy, and it does not even use fancy features. It also takes Gb
of ram when building in parallel.

David