[Numpy-discussion] scipy.base

[Pearu Peterson]

Hi Travis,

I have committed scipy.{distutils,base} to Numeric3 CVS repository. 
scipy.distutils is a reviewed version of scipy_distutils and
as one of its new features there is Configuration class that allows
one to write much simpler setup.py files for subpackages. See setup.py
files under Numeric3/scipy directory for examples. scipy.base is a
very minimal copy of scipy_base plus ndarray modules.

When using setup_scipy.py for building, the ndarray package is installed
as scipy.base and

  from scipy.base import *

should work equivalently to

  from ndarray import *

for instance.

I have used information from Numeric3/setup.py to implement
Numeric3/scipy/base/setup.py and it should be updated whenever
Numeric3/setup.py is changed. 
However, I would recommend start using scipy.base instead of ndarray as
using both may cause unexpected behaviour when installed ndarray is older
than scipy.base installation (see [*]). In Numeric3 CVS repository that
would mean replacing setup.py with setup_scipy.py and any modification to
ndarray setup scripts should be done in scipy/base/setup.py. We can apply
this step whenever you feel confident with new setup.py files. Let me know
if you have any troubles with them.

To clean up Numeric3 CVS repository completely then Include, Src, Lib,
CodeGenerators directories should be moved under the scipy/base directory.
However, this step can be omitted if you would prefer working with files
at the top directory of Numeric3. Current setup.py scripts fully
support this approach as well.

There are also few open issues and questions. 

First, how to name Numeric3 project when it installs scipy.base,
scipy.distutils, Numeric packages, etc? This name will be used when
creating source distributions and also as part of the path where header
files will be installed. At the moment setup_scipy.py uses the name
'ndarray'. And so `setup_scipy.py sdist`, for example, produces
ndarray-30.0.tar.gz file; `setup_scipy.py install` installs header files
under <prefix>/include/ndarray/ directory. Though this is fine with me, I
am not sure that this is an ideal situation. I think we should choose the
name now and stick to it forever, especially since 3rd party extension
modules need to know where to look for ndarray header files. This name
cannot be 'numarray', obviously, but there are options like 'ndarray',
'numpy', and may be others. 
In fact, 'Numeric' (with version 3x.x) would be also an option but that
would be certainly cause some problems when one wants both Numeric 2x.x
and Numeric 3x.x to be installed in the system, the header files would end
up in the same directory, for instance. As a workaround, we could force
installing Numeric3 header files to <prefix>/include/Numeric/3/ or
something. I acctually like this idea but I wonder what other think about
this.

Second, is it already possible to use ndarray C/API as a replacement of
Numeric C/API, i.e. would simple replacement of 

  #include "Numeric/arrayobject.h"

with 

  #include "ndarray/arrayobject.h"

work? And if not, will it ever be? This would be interesting to know as an
extension writer.

[*] Due to keeping changes to Numeric3 sources minimal, scipy.base
multiarray and umath modules first try to import ndarray and then
scipy.base whenever ndarray is missing. One should remove ndarray
installation from the system before using scipy.base.

Regards,
Pearu