[Numpy-discussion] For review: first milestone of scons support in numpy

[Pearu Peterson]

David Cournapeau wrote:
> Pearu Peterson wrote:
>> I think this is good.
>> Does scons require python-dev? If not then this will solve one of the
>> frequent issues that new users may experience: not installed distutils.
> Isn't distutils included in python library ?

Not always. For example, in debian one has to install python-dev 
separately from python.

> Anyway, scons does not 
> require anything else than a python interpreter. Actually, an explicit 
> requirement of scons is to support any python starting at version 1.5.2 
> (this is another important point which I consider important for a 
> replacement of numpy.distutils).

Though, it is irrelevant as numpy/scipy packages require python
versions starting at 2.3.

>> I think numpy is quite stable now that it's safe to develop in a branch
>> (if trunk is very actively developed then merging branches
>> can be a nightmare). However, IMHO using a branch makes other
>> developers to stay aside from branch development and in time it is
>> more and more difficult to merge.
> I don't have strong experience in subversion, so I was afraid of that. 
> Do I understand correctly that you suggest opening a new dev branch, and 
> then do all subsequent dev (including non distutils/scons related ones) 
> there ?

No, my original suggestion was that I don't mind if you would develop
scons support in trunk as it does not affect the current state
of numpy/scipy builds. Don't know if other developers would have
objections in that.

>> numpy.distutils setup.py script gathers the information from
>> subpackage setup.py scripts recursively and then passes all the
>> information to one setup function call.
>> I think setupscons.py should do the same. If scons does not support
>> recursive reading of scons scripts then the corresponding feature
>> should be implemented, I guess it would not be difficult.
> Scons supports recursive calls. To be more clear about the possibilities 
> of scons wrt to this, let's take a simplified example:
> 
> root/Sconstruct
> root/numpy/Sconscript
> root/numpy/core/Sconscript
> root/numpy/linalg/Sconscript
> 
> If you are in root and call scons (> is a shell prompt):
> 
> root > scons
> 
> Then it will call recursively all the sconscript (as long as you request 
> it in the sconscript files). The other supported method is
> 
> root/numpy/linalg > scons -u
> 
> this will look every parent directory to find the "root" Sconstruct.

My point was that

   root/numpy/linalg > scons

should work (without the -u option). A subpackage may not require all
the stuff that other subpackages require and therefore scons should
not configure everything - it's a waste of time and efforts - especially
if something is broken in upper packages but not in the subpackage.

>> I meant that Configuration class could have a method, say 
>> toscons(<filename>),  that will generate SConstruct script
>> from the information that Configuration instance holds. I thought that 
>> this would just ease creating SConstruct scripts from
>> existing setup.py files.
> I don't think it would worth the effort for numpy (the main work really 
> is to implement and test all the checkers: blas/lapack, fortran). Now, 
> as a general migration tool, this may be useful. But since we would 
> still use distutils, it would only be useful if it is easy to develop 
> such as tool.

Yes, that's a low priority feature.

> P.S: would it be easy for you to make a list of requirements for fortran 
> ? By requirement, I mean things like name mangling and so on ? Something 
> like the autoconf macros: 
> http://www.gnu.org/software/autoconf/manual/autoconf-2.57/html_node/autoconf_66.html

To use f2py succesfully, a fortran compiler must support flags that make
fortran symbol names lowercase and with exactly one underscore at the 
end of a name. This is required when using numpy.distutils.

f2py generated modules make use of the following CPP macros:
-DPREPEND_FORTRAN -DNO_APPEND_FORTRAN -DUPPERCASE_FORTRAN -DUNDERSCORE_G77
and therefore the above requirement would not be needed if
scons could figure out how some particular compiler mangles
the names of fortran symbols. This would be especially useful
since some fortran compiler vendors change the compiler flags
between compiler versions and one has to update numpy.distutils
files accordingly.

Note that using hardcoded name mangeling flags may be still required
for certian Fortran 90 compilers (which ones exactly, I don't member 
now) that by default produce symbol names with special characters like $ 
or . for F90 modules and making these names unaccessible to C programs.

Pearu