[Numpy-discussion] Latest Array-Interface PEP
Chris.Barker at noaa.gov
Fri Jan 5 16:39:26 EST 2007
Travis Oliphant wrote:
> Christopher Barker wrote:
>> wxPython -- Robin Dunn
>> PIL -- Fredrik Lundh
>> PyOpenGL -- Who?
>> PyObjC -- would it be useful there? (Ronald Oussoren)
>> MatplotLib (but maybe it's already married to numpy...)
>> PyGtk ?
> It's a good start, but their is also
> PyMedia, PyVoxel, any video-library interface writers, any audo-library
> interface writers.
right -- I didn't intend that to be a comprehensive list.
> Anybody who wants to wrap/write code that does some kind of manipulation
> on a chunk of data of a specific data-format.
> There are so many people who would use it that I don't feel qualified to
> speak for them all.
I think this is key -- we all know that there are a lot of people that
*could* use it, and we might even say *should* use it. The question that
I think the core python devs want answered is *will* they use it. That's
why I suggest that rather than having a bunch of numpy users make
comments to python-dev, we really need authors of packages like the
above to make comments to python-dev, saying "I could use this, and I'll
*will* use this if it's been put into the standard lib".
I do think there is one issue that does need to be addressed. The
current buffer protocol already allows modules to share data without
copying -- but it doesn't provide any description of that data. This
proposal would provide more description of that data, but still not
describe it completely -- that's just not possible, so how helpful is
the additional description -- I think a lot, but others are not convinced.
Some examples, from my use. I use wxPython a fair bit, so I'll use that
as an example.
One can now currently pass a buffer into a wxImage constructor to create
an image from existing data. For instance, you can pass in an WxHx3
numpy array of unsigned bytes in to create a WxH RGB image. At the
moment, all the wx.Image constructor checks is if you've passed in the
correct amount of bytes. With this proposal, it could check to see if
you passed in a WxHx3 array of bytes. How helpful would that be? it's
still up to the programmer to make sure those bytes actually represent
what you want. You may catch a few errors, like accidentally passing in
a 3xWxH array. You also wouldn't have to pass in the size of the Image
you wanted - which would be kind of nice. One could also write the
Image constructor so that it could take a few different shapes of data,
and do the right thing with each of them.
How compelling are these advantages?
When you need to draw something like a whole lot of pixels or a long
polyline, you can now pass into wxPython either: a list of (x,y) tuples,
or any sequence of any (x,y) sequences. A Nx2 numpy array appears as the
latter, but it ends up being slower than the list of tuples, because
wxPython has some code to optimize accessing lists of tuples.
Internally, wxWidgets has drawing methods that accept a Nx2 c-array of
ints. With the proposed protocol, wxPython could recognize that such an
array was passed in, and save a LOT of sequence unpacking, type checking
and converting, etc. It could also take multiple data types -- floats,
ints, etc, and do the right thing with each of those.
This to me is more compelling than the Image example.
By the way, Robin Dunn has said that he doesn't want to include numpy
dependency in wxPython, but would probably accept code that did the
above if it didn't add any dependencies.
Francesc Altet wrote:
> In any case, I think that if the PEP has success, it would suppose an
> extraordinary leap towards efficient data interchange in applications that
> doesn't need (or are reluctant to include) NumPy for their normal operation.
That's the question -- is it an extraordinary leap over what you can now
do with the existing buffer protocol?
Matthew Brett wrote:
> Is there already, or could there be, some sort of consortium of these
> that agree on the features in the PEP?
There isn't now, and that's my question -- what is the best way to
involve the developers of some of the many packages that we envision
- random polling by the numpy devs and users?
- more organized polling by numpy devs (or Travis)
- just a note encouraging them to pipe in on the discussion here?
- a note encouraging them to pipe in on the discussion at python-dev.
I think the PEP has far more chances of success if it's seen as a
request from a variety of package developers, not just the numpy crowd
(which, after all, already has numpy)
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker at noaa.gov
More information about the NumPy-Discussion