[Python-Dev] Expose the array interface in Python 2.5?

[Travis Oliphant]

Edward C. Jones wrote:
> "Travis E. Oliphant" <oliphant.travis at ieee.org> wrote:
> 
>  > It is very important for many people to get access to memory with some
>  > description of how that memory should be interpreted as an array.
>  > Several Python packages could benefit if Python had some notion of an
>  > array interface that was at least supported in a duck-typing fashion.
> 
> Which packages? Which people? Which constituencies?
> 

I think I spell it out later.  Do you really need to argue about whether 
or not an array interface is a useful thing?  I thought we were beyond 
that and to the point of trying to figure out how to get the many groups 
to agree at least on a common interface.

> "Travis E. Oliphant" <oliphant.travis at ieee.org> also wrote:
> 
>  > My big quest is to get PIL, PyVox, WxPython, PyOpenGL, and so forth to
>  > be able to use the same interface.  Blessing the interface by
>  > including it in the Python core would help.  I'm also just wanting
>  > people in py-dev to get the concept of an array interface on their
>  > radar, as discussions of new bytes types emerges.
> 
> I use PIL and numarray a lot. It would be nice if they used a common 
> array format so I would not need to convert back and forth. But I 
> survive quite well with the current arrangement.
>

We all "survive",  but saying it is "quite well" is a bit optimistic as 
it means many very useful applications are harder to write than they 
really need to be.

> Many other packages besides PIL and Numeric / numarray / Numpy are 
> involved here: byte, struct, ctypes, SWIG, PyTables, Psyco, PyPy, Pyrex, 
> etc. There are some major issues that need to be dealt with which I will 

Sure they are involved, but I would argue the other ones you list care 
less about the multidimensional aspect of the array interface. 
(Actually PyTables just uses NumPy and so it should not be discussed as 
a "separate" package --- i.e. PyTables already tries to get along with 
NumPy as do many other packages...)

> 
> A data structure without an API and thorough documentation is useless. 
> Any proposal needs to include them from the very start.

Again, I restate.  The Numeric structure has been documented and has 
been around for a *long* time.  I'm just trying to get this basic 
interface into Python as a very simple object.  Let's not try to make it 
so complicated that no body can agree on what it should do.  To be 
specific, I want to see a type object with almost none of the Type 
structure filled in with specific behavior.

I'm mainly interested in an array structure that other packages can rely 
on (and inherit from if they so choose).

Because the C-structure of the Numeric PyArrayObject (which NumPy also 
uses) is so widely known and used and documented for over 10 years, I 
argue it ought to form the foundation for this simple Python object.

We can argue about explicit multidimensional indexing behavior, but to 
hold hostage the introduction of a simple inheritable object to 
disagreements about those more complicated issues seems to be missing 
the mark.

> 
> How should Python interact with low level data? By "low level data" I 
> mean data as seen by C, C++, and FORTRAN as well as linear arrays of bytes.

This is already known about in Numeric.  That's what I'm saying. 
Numeric handles this well, let's just bring over this basic memory model 
for an array over to Python itself and not worry about the other 
TypeObject function-pointer tables until later.

Everybody I talked to at SciPy was very enthused about this concept. 
There is a large number of people who don't read Python-dev that I'm 
speaking for here.

> 
> What changes in Python would make the packages listed above easier to 
> write and use? Easier enough to write that the package owners would be 
> willing to give up control of part of their packages.

They don't have to give up control if we just introduce a simple memory 
model for an array.


Thanks for your comments,


-Travis