On Apr 6, 2010, at 9:08 AM, David Cournapeau wrote:

Hi Travis,

On Tue, Apr 6, 2010 at 7:43 AM, Travis Oliphant <oliphant@enthought.com> wrote:

I should have some time over the next couple of weeks, and I am very
interested in refactoring the NumPy code to separate out the Python
interface layer from the "library" layer as much as possible.   I had
some discussions with people at PyCon about making it easier for
Jython, IronPython, and perhaps even other high-level languages to
utilize NumPy.

Is there a willingness to consider as part of this reorganization
creating a clear boundary between the NumPy library code and the
Python-specific interface to it?   What other re-organization thoughts
are you having David?

This is mainly it, reorganizing the code for clearer boundaries
between boilerplate (python C API) and actual compuational code.
Besides helping other python implementations, I think this would
benefit NumPy itself in the long run (code maintainability), as well
as scipy (and other C extensions). I think the npymath effort is a
good example: albeit simple in nature (the API and boundaries are
obvious), it has already helped a lot to solve numerous platform
specific issues in numpy and scipy, and I think the overall code
quality is better.

My own goals were:
- exposing core computational parts through an exported C API, so
that other C extensions may use it (for example, exposing basic
blas/lapack operations)
- dynamic loading of the code (for example depending on the CPU
capabilities - I have a git branch somewhere where I started exposing
a simple C API to query cpu capabilities like cache size or SSE
dynamically to that intent)
- more amenable codebase: I think multiarray in particular is too
big. I don't know the code well enough to know what can be split and
how, but I would have hoped that the scalartypes, the type descriptor
could be put out of multiarray proper. Also, exposing an API for
things like fancy indexing would be very useful, but I don't know if
it even makes sense - I think a pure python implementation of fancy
indexing as a reference would be very useful for array-like classes (I
am thinking about scipy.sparse, for example).

Unfortunately, I won't be able to help much in the near future (except
maybe for the fancy indexing as this could be useful for my job),

I understand.   It just happens that there is some significant time for me to look at this over the next few weeks and I would really like to make progress on re-factoring.   I think it's O.K. if you don't have time right now to help as long as you have time to offer criticism and suggestions.   

We could even do that over Skype with whomever else wanted to join us (we could do a GotoMeeting discussion as well) if you think it would be faster to just talk in a group setting instead of email.     Of course, a summary of any off-line discussion should be sent to the list. 

Thanks for the input,