[Numpy-discussion] Latest Array-Interface PEP

Travis Oliphant oliphant at ee.byu.edu
Fri Jan 12 18:33:30 EST 2007

> I'm concerned about the direction that this PEP seems to be going. The 
> original proposal was borderline too complicated IMO, and now it seems 
> headed in the direction of more complexity.

Well at least people are talking about what they would like to see.   
But, I think we should reign in the discussion. 

>    1. Things that act like arrays, but have different storage methods.
>       This details of this still seem pretty vague, but to the extent
>       that I can figure them out, it doesn't seem useful or necessary
>       to tie this into the rest of the array interface PEP.  For
>       example,  "array_interface->get_block_from_slice()" has been
>       mentioned. Why that instead of
>       "PyObject_AsExtendedBuffer(PyObject_GetItem(index), ....)"[2].
>       I'll stop here, till I see some more details of what people have
>       in mind, but at this point, I think that alternative memory
>       models are a different problem that should be addressed separately.

I'm leaning this way too.

>  Sadly, I'm leaving town shortly and I'm running out of time, so I'll 
> have to leave my objections in this somewhat vague state.
> Oh, the way that F. Lundh plans to expose PIL's data a chunk at a time 
> is mentioned in this python-dev summary:
> http://www.python.org/dev/summary/2006-11-01_2006-11-15/
> It doesn't seem necessary to have special support for this; all that 
> is necessary is for the object returned by acquire_view to support the 
> extended array protocol.

Yes, I agree.

> [1] Remind me again why we can't simply use ctypes for this? It's 
> already in the core. I'm sure it's less efficient, but you shouldn't 
> need to parse the data structure information very often. I suspect 
> that something that leveraged ctypes would meet less resistance.

Two reasons:

1) ctypes wasn't designed for this purpose specifically and leaves out 
certain things
2) ctypes uses many Python types instead of just a single python type 
(the PyDataFormatOjbect).

> [2] Which reminds me. I never saw in the PEP what the actual call in 
> the buffer protocol was supposed to look like. Is it something like:
> PyObject_AsExtendedBuffer(PyObject * obj, void **buffer, Py_ssize_t 
> *buffer_len, funcptr *bf_getarrayview, funcptr *bf_relarrayview)

No, not like that.   The bf_getarrayview function pointers hang-off of 
the as_buffer method table which is pointed to by the typeobject.   You 
could always access the API using those function pointers, but it is 
more traditional to use an API call which adds some checking to make 
sure the function pointer is there and then calling the function pointer.

I don't know if I go into a lot of detail there, but I should probably 
add more.   PEP's are rather "expensive" for me in terms of how much 
immediate benefit the change to Python is to me personally versus that 
time spent writing them. 

The benefit here is much more long term in establishing a useful 
data-model that could be used  by a lot of applications in Python to 
exchange data (and help ameliorate the proliferation of objects in 
Python that are essentially and should be NumPy arrays).


More information about the NumPy-Discussion mailing list