[Numpy-discussion] Performance of the array protocol

[Travis Oliphant]

Chris Barker wrote:

> Travis Oliphant wrote:
>
>> So, I guess the answer to your question is that for small arrays it 
>> is an intrinsic limitation of the use of Python attributes in the 
>> array protocol.
>
>
> IIRC, in the early discussion of the array protocol, we had talked 
> about defining a C struct, and a set of utilities to query that 
> struct. Now, I guess it uses Python attributes. Do I recall 
> incorrectly, or has there been a design change, or is this a prototype 
> implementation? I guess I'd like to see an array protocol that is as 
> fast as fromstring(), even for small arrays, though it's probably not 
> a big deal. Also, when writing C code to work with an array, it might 
> be easier, as well as faster, to not have to query Python attribute


You are correct that it would be good to have a C-protocol that did not 
require attribute lookups (the source of any speed difference with 
fromstring).  

Not much progress has been made on a C-version of the protocol, though.  
I don't know how to do it without adding something to Python itself.   
At SciPy 2005, I outlined my vision for how we could proceed in that 
direction.

There is a PEP in a subversion repository at

http://svn.scipy.org/svn/PEP

that explains my view.  Basically, I think we should push for a simple 
(C-based) Nd array object that is nothing more than the current 
C-structure of the N-d array.

Then, arrays would inherit from this base class but all of Python would 
be able to understand and query it's C-structure.

If we could also get an array interface into the typeobject table, it 
would be a simple thing to populate this structure even with objects 
that didn't inherit from the base object.

I am still interested in other ideas for how to implement the array 
interface in C, without adding something to the type-object table (we 
could push for that, but it might take more political effort).

-Travis