[Numpy-discussion] Time for beta1 of NumPy 1.0

[David Cournapeau]

Albert Strasheim wrote:
> Hey Chuck
>
>   
>> -----Original Message-----
>> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy-
>> discussion-bounces at lists.sourceforge.net] On Behalf Of Charles R Harris
>> Sent: 01 July 2006 19:57
>> To: Robert Kern
>> Cc: numpy-discussion at lists.sourceforge.net
>> Subject: Re: [Numpy-discussion] Time for beta1 of NumPy 1.0
>>
>> All,
>>
>> This is bit off topic, but a while ago there were some complaints about
>> the usefulness of distutils. I note that KDE has gone over to using cmake
>> after trying scon. I am not familiar with cmake, but perhaps someone here
>> knows more and can comment on its suitability.
>>     
>
> CMake definately warrants investigation, but I think SCons might be a better
> way to go. I think it would make it easier to reuse large parts of the
> existing build code (e.g. conv_template.py could be converted into a SCons
> builder without too much effort). Reusing parts of distutils and setuptools
> would also be easier if the new tool is somehow Python-aware.
>
> I think the main problem with distutils in the NumPy context is that it was
> never designed to build C/Fortran code over so many platforms with to many
> possible build configurations. python setup.py install works pretty well,
> but any kind of non-default configuration can become a bit hairy, despite
> the excellent work on NumPy extensions to distutils.
>
> I'd like to take a stab at doing something with SCons in a few weeks' time.
> Does anybody want to brainstorm on some ideas for what is needed from a
> better build system for NumPy? Maybe a wiki page?
>   
I have a small experience with scons, as a replacement of the auto* 
tools for small packages
on my own (requirements for cross-building, library and header 
dependency, build of libraries, etc...). So I am willing to share my 
somewhat limited experience with scons (the code I am building scons 
with is using cblas/clapack, and has libraries + some unit testing, so 
we would not start from scratch). Also, I have access to x86 and ppc 
linux + mac os x + windows easily, which makes it easy to test on some 
common platforms,

David

P.S: Some comments on scons :

I don't know distutils, so I can only compare to autotools: from *my* 
experience, you should think about scons as a Makefile replacement, and 
as a build framework to build onto. The main pro of scons:

 - having a real language for build rules programming is a real plus. It 
makes it much easier to extend that autoconf, for example (debugging m4 
macro is not something I can enjoy much, and I am sure I am not alone).
- the dependency checking works great
- parallel build is explicitly handled
- scons knows how to build library (static and shared) on the plateforms 
it supports
- can be included in the project so scons does not need to be installed 
if needed (I have never used this feature myself).

The main cons:
 - configuration part: there are some tools to test library/header a la 
autoconf, but this is far from great in the present situation, mainly 
because of the next point
 - option handling from the command line: there is some support, but 
nothing is automatic. On the long run, this is painful.
 - No support for library versioning; I am not sure about rpath support, 
which is useful for non-standard path installation. I don't know how 
difficult it would be to implement for all platforms (I have some
 - can be slow for big projects ? I have seen quite big projects (eg 
ardour: several hundred of .c and .h files) using scons, and it was not 
really slow, and I don't think it would be a problem for something like 
numpy which size is nothing compared to kde.

To sum it up: as a make replacement, from a developer POV, it works 
great. As a tool for *distribution*, I am less convinced. For people 
familiar with autotools, scons is a great automake replacement. 
Everything else has to be implemented: autoconf, libtool, etc... My 
understanding is that those two tools (autoconf and libtool) are the 
ones most needed for numpy, so there is a lot of work to do if we want 
to use scons.