[MATRIX-SIG] NumPy in the Python 1.5 era

[L. Busby]

[ Konrad Hinsen said ]
KH> With Python 1.5 approaching slowly but steadily, it is time to discuss
KH> the future of NumPy as well. At IPC6 it was mentioned that LLNL might
KH> take over the responsibility for further NumPy development, but others
KH> are more qualified to comment on that - is anyone willing to do so?

Yes, LLNL did volunteer, and Jim Hugunin accepted.  We haven't
actually transferred any code yet - Jim is doing some cleanup first.
In a previous message to Jim, I said:

  Basically, here's what we (LLNL) propose to do:  Give the code a home
  in a CVS repository (with access to selected outside developers as
  needed).  Keep the release package current with Python's conventions,
  makefiles, and et cetera.  Keep the released code at python.org up to
  date with respect to our copy.  Implement changes as per discussion by
  the matrix-sig.  Protect existing and any future copyrights.
  
  In the near term, we'll plan to add support for the new floating point
  exception package in Python-1.5, and we'll take a look at how best to
  package NumPy given the new package support in 1.5.  I'll be doing some
  of this work myself, so I can't really commit to a lot more until I've
  had a chance to review and shuffle my priorities a bit.

We Python programmers in X-Division at LLNL absolutely intend to expose
NumPy to the widest possible circle of interested developers in the
matrix-sig.  We also have a lot of real big problems to solve with
NumPy, so we plan to be strong custodians of it with respect to
stability, performance, documentation, and so on.  Although we have a
lot of ideas for its future improvement, they generally need to be
worked out in the SIG.  We would especially appreciate ideas now about
how best to do source code management for an international community of
programmers.

KH> With the new package feature in 1.5, we could turn NumPy into a package
KH> again (veterans may remember that one alpha release used ni-based packages).
KH> This obviously raises the question of compatibility. My suggestion is
KH> to keep the current modules, but provide an alternate arrangement as
KH> a package, and add eventual new modules only to that package. In other
KH> words, we would still have module/package Numeric with all its functions,
KH> but the other modules (LinearAlgebra etc.) would be available as
KH> modules within Numeric, as well as stand-alone modules for compatibility.
KH> This would cost only minor modifications plus a few links (no duplicate
KH> code). Any comments?

My only comment is that I'm not very wedded to backward compatibility.
If NumPy can be presented as a Python-1.5 package in a neat and generic
fashion, I would be perfectly happy if that were the *only* (Unix)
distribution format.

KH> Even more important is the installation problem. It's still too difficult
KH> for many users, and the installation of other C modules using NumPy
KH> currently depends on the way NumPy was installed.
KH> 
KH> It is possible (and I even volunteer to do it!) to convert NumPy into
KH> a clean extension that can be installed in the same way on all Unix
KH> systems supporting dynamic libraries, and the C API would then also be
KH> accessible in the same way on all systems. Basically it involves
KH> making the C API accessible via a pointer array that other modules get
KH> by a standard import. This involves, however, a compulsory addition
KH> (of one line) to all C extension modules using the C API. In my
KH> opinion this is preferable to the current mess. Any other opinions?
KH> 
KH> Once NumPy loses its "special" status, one could think about providing
KH> an automatic installation program, such that users would essentially
KH> download the tar archive, unpack it, and type two commands, one for
KH> compilation and one (as root) for installation. It could hardly be
KH> easier than that.

I like that part about volunteering.  I haven't really studied the
new 1.5 package facility, so can't comment on it yet.  Your idea for
exposing the C API to other dynamically loaded modules sounds plausible.

KH> I can't say much about non-Unix systems, but I suppose that for
KH> Windows and Mac a binary distribution would be the best solution.

Ok by me.

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________