[SciPy-dev] Re: [Numpy-discussion] Trying out Numeric3

[Pearu Peterson]

On Thu, 24 Mar 2005, Travis Oliphant wrote:

> Michiel Jan Laurens de Hoon wrote:
>
>> Arnd's comment raises the question of how to try out or contribute to 
>> Numeric3 if the code base is changing from day to day. It may be a good 
>> idea to set up some division of labor, so we can contribute to Numeric3 
>> without getting in each other's way. For example, I'd be interested in 
>> working on setup.py and putting different parts of Numeric3/scipy_base 
>> together.
>> 
>
> Michiel, you are free to work on setup.py all you want :-)
>
> Putting the parts of scipy_base together is a good idea.   Exactly how to 
> structure this is going to require some thought and need to be coordinated 
> with current scipy.
>
> I want a package that is as easy to install as current Numeric (so the 
> default will have something like lapack_lite). 
> But, this should not handicap nor ignore a speed-conscious user who wants to 
> install ATLAS or take advantage of vendor-supplied libraries.
>
> There should be a way to replace functionality that is clean and does not 
> require editing setup.py files.
>
> Anybody with good ideas about how to do this well is welcome to speak up. 
> Perhaps, the easiest thing to do is to keep the basic Numeric structure (with 
> C-based easy-to-install additions) and call it scipylite  (with backwards 
> compatibility provided for Numeric, LinearAlgebra, RandomArray, and MLab 
> names).  This also installs the namespace scipy which has a little 
> intelligence in it to determine if you have altas and fortran capabilities 
> installed or not.
>
> Then, provide a scipyatlas package that can be installed to take advantage of 
> atlas and vendor-supplied lapack/blas.
>
> Then, a scipyfortran package that can be installed if you have a fortran 
> compiler which provides the functionality provided by fortran libraries. 
> So, there are three divisions here.

Hmm, the idea of introducing scipylite, scipyatlas, scipyfortran packages 
does not sound like a good idea. The usage of atlas or fortran blas/lapack or 
vendor based blas/lapack libraries is an implementation detail and should 
not be reflected in scipy_base package structure. This is because such an 
approach is not suitable for writing portable Numeric3 based applications 
or packages. For example, if a developer uses scipyfortran package in a 
package, it immidiately reduces the number of potential users for this 
package.

I got an impression from earlier threads that scipy_distutils will be 
included to scipy_base. So, I am proposing to use scipy_distutils tools 
and our scipy experience for dealing with this issue, scipy.lib.lapack
would be a good working prototype here.

Ideally, scipy_base should provide a complete interface to LAPACK 
routines, but not immidiately, of course. Now, depending on the 
availability of compilers and resources in a particular computer, the 
following would happen:
1) No Fortran compiler, no lapack libraries in the system, only C compiler 
is available --- f2c generated lite-lapack C sources are used to build 
lapack extension module; wrappers to lapack routines, for which there are 
no f2c generated sources, are disabled by f2py `only:` feature.
lite-lapack C sources come with scipy_base sources.
2) No Fortran compiler, system has lapack libraries (atlas or Accelerate 
or vecLib), C compiler is available --- system lapack library will be used 
and a complete lapack extension module can be built.
3) Fortran and C compiler are available, no lapack libraries in the system 
--- Fortran lite-lapack sources are used to build lapack extension module;
lite-lapack Fortran sources come with scipy_base sources. Similar to 
the case (1), some wrappers are disabled.
4-..) other combinations are possible and users can choose their favorite 
approach.

The availability of system resources can be checked using 
scipy_distutils.system_info.get_info. Checking the availability of Fortran 
compiler should be done in a configuration step and only when an user 
specifically asks for it, by default we should assume that Fortran 
compiler is not available. The same should apply also to atlas/lapack/blas 
libraries, by default f2c generated lite-lapack C sources will be used.
In this way users that only need Numeric3 array capabilities will avoid 
all possible troubles that may show up when using all possible resources 
for speed on an arbitrary computer.

Btw, I would suggest using `scipy <dot> <package>` instead of
`scipy <package>` or `scipy <underscore> <package>` for naming packages.

Pearu