[Python-Dev] Expose the array interface in Python 2.5?

[Nick Coghlan]

Chris Barker wrote:
> I'm still a tiny bit confused about the proposed individual pieces 
> involved, but I'd like to see, in order of priority (and difficulty):
> 
> 1) A standard n-d array interface (numpy already defines this, but 
> outside of the numpy community, who knows about it?)

This is pretty much a matter of taking Travis's array interface and checking 
it in as a PEP. Checking it in is the easy part, but I'm not clear on what 
actually constitutes the array interface.

The array interface is apparently here:
http://numeric.scipy.org/array_interface.html

but it only talks about exposing this info to Python code - it doesn't discuss 
how to expose or use it at the C level (unless the intent is that it should be 
accessed via the PyObject abstract API, in which case a PEP would need to say 
that explicitly).

However, the top of that page references the genarray PEP:
http://svn.scipy.org/svn/PEP/PEP_genarray.txt

Here it gets even *more* confusing, because the dimarray objects in that PEP 
don't appear to expose the array interface described by the page above. If 
this PEP is out of date, then the reference needs to be removed until it is 
brought up to speed (a disclaimer on the draft PEP probably wouldn't hurt either).

A useful PEP for a C-level array interface would need to cover the following 
three things that an extension author would need to know in order to either 
produce or consume generic arrays:

   1. The C-level protocol for exposing and retrieving the array interface
      (potentially using the PyObject API to access Python-level attributes)
   2. How to use the array interface to access an extension type's data
   3. How to use the array interface to modify an extension type's data

This is doable in whatever time frame we like, since it is documenting a 
convention, rather than an implementation, and so isn't coupled directly to 
the release cycle.

However, if it's available soon (before the second alpha?), it may be possible 
to update arrayobject and ctypes to expose this interface for Python 2.5.

> 2) A standard, n-d array C-Object, inheritable for use by various 
> extension packages.
>
> 3) A standard n-d array object, with a python interface for indexing and 
> slicing for access and assignment (modeled after, or better yet taken 
> directly from, numpy).

For the sake of test writers' sanity, these would probably need to be done 
together so that at least some of the tests could be written in Python.

Due to the conflicting slice semantics between the standard library (slices 
are copies) and numpy (slices are mutable views), I'd actually suggest that 
the Python interface for this simple object *shouldn't* support slicing (at 
least, not in its first incarnation).

Regardless, I can't see either of these steps happening before Python 2.6. If 
we try to do step 2 without doing step 3, then we're stuck either writing all 
the test code in C, or else shipping an untested component. Neither seems like 
a good idea.

> 4) A standard n-d array object that supports array-wise arithmetic and 
> functions (ufuncs, in numpy parlance).

Definitely not before 2.6 :)

> There is no reason it has to have anything to do with the current array 
> module. For instance, having the arrays be of fixed size is just fine.
> It's really not that big a deal to make a new one when you need to 
> change the size, after all, we live with that for immutable objects, 
> like strings.

The array module is the easiest place to put the code - then you would have 
"array.array" as a 1 dimensional resizeable array, and "array.dimarray" as a 
multi-dimensional array.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org