[Python-Dev] Expose the array interface in Python 2.5?

Nick Coghlan ncoghlan at gmail.com
Sat Mar 18 05:09:13 CET 2006

Travis E. Oliphant wrote:
>> Given that even the bytes type has been deferred to 2.6 to allow further 
>> consideration of the appropriate API, my vote is to do the same for an 
>> array.dimarray type and allow more time to figure out the appropriate *Python* 
>> interface.
> I was afraid of that.  But, unless people in pydev actually care to 
> discuss these matters, I fear that yet again nothing will be done.  The 
> problem is that for most of us array users, it's only community outreach 
> and a desire to get people using Python talking the same array language 
> that makes us really care about these things.  The NumPy library works 
> fine for what we really need it to do, and it's hard to get motivated to 
>    convince people that haven't used an array-language like IDL or 
> MATLAB in the past to understand the reasons for NumPy's behavior.

Hmm, we could have the builtin type support only access to individual elements 
(raising an IndexError or ValueError for any slice operations, or failures to 
specify all dimensions). It wouldn't be that easy to *manipulate* data in that 
format, but it would meet the needs of a common memory format for the various 
array implementions that *do* make the data easy to manipulate.

Doing that would allow us to get a basic type while ducking most (all?) of the 
potentially controversial behaviour. Such a type would definitely belong in 
the array module rather than being a builtin, though.

> As the bytes type is developed please keep in mind it's uses as the 
> memory for an N-dimensional array.  Perhaps the bytes object could be a 
> default way (or built on a default way) to allocate memory.  A simple 
> reference-counted memory object would certainly belay the problems of 
> the buffer interface that the array object currently has problems with.

Even if array.dimarray was implemented as a fairly dumb implementation of the 
multi-dimensional array interface, then other types could either inherit from 
it or contain it and manipulate the memory directly.

> In other words, the array object should not malloc it's own memory but 
> create a memory object which is nothing more than a reference-counted 
> pointer to memory.  Surely this has been talked about. Is there a reason 
> it has not been implemented?  It would not be that hard.

A simple implementation of array.dimarray could certainly serve as such an object.

Note that we don't have to do everything at once. It would be possible to put 
a transition plan in a PEP whereby array.dimarray was added in Python 2.5 
(allowing external modules to start relying on it), while converting things 
like ctypes and array.array to use it could be deferred until 2.6.

If we defer the whole lot, then even if the standard library used a common 
bulk data format in 2.6, extension modules probably wouldn't be using it until 

Of course, there's a whole 'nother question of where Jython, IronPython and 
PyPy would fit into this. . .


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list