On 1/9/07, Christopher Barker <Chris.Barker@noaa.gov> wrote:
Timothy Hochberg wrote:
> The reason that I ask is that the two projects that I use regularly are
> wxPython and PIL generally operate on relatively large data chunks and
> it's not clear that they would see much benefit over this mechanism
> versus the array protocol.

Let me preface my remarks by saying, that I was initially assuming that this was meant as a supplement to the earlier array protocol proposal, not a replacement as Travis subsequently explained.

But is this mechanism any harder?

It doesn't look like it to me. In
fact, as I have written a tiny bit of Numeric extension code, this looks
  familiar and pretty easy to work with.

I expect that the old proposal is easier to implement right now. We could implement the old array protocol in wxPython and have fairly seemless integration with numpy without any dependencies. To implement the new protocol, we'd need the C-API. Since that's not in Python at the moment, we'd have to include implementations of the various functions into wxPython. I suppose that wouldn't really be that bad assuming they are already implemented somewhere. It's a bit of a chicken and the egg problem though.
> I imagine that between us Chris Barker and I could hack together
> something for wxPython (not that I've asked him aout it).

I'm not sure when I'll find the time, but I do want to do this.

> And code would
> probably go a long way to convincing people what a great idea this is.
> However, all else being equal, it'd be a lot easier to do this for the
> array protocol since there's no extra infrastructure involved.

Is it that much infrastructure?

I'm not sure. It may not be that bad. I'd guess you'd need both an include file and a source file for the implementations of the functions.

It looks like this would, at the least, require an extra include file.
If this flies , then that will be delivered with python 2.8?  until then
(and for older pythons) would various extension writers all need to add
this extra file to their source? And might we get a mess with different
versions floating around out there trying to interact?

Possibly. It shouldn't be a big deal if the API is frozen. But I expect the best thing to get this to work would be to implement this for as many projects as possible as trial patches before trying to get this into those projects officially. That way we can get some experience, tweak the API if necessary, then freeze it and release it officially.

Like I said, I'll help with wxPython. I'm tempted to try with PIL as well, but I've never looked at the code there, not even tried to compile it, so I don't know how far I'd get.

> FWIW, the array protocol PEP seems more relevant to what I do since I'm
> not concerned so much with the overhead since I'm sending big chunks of
> data back and forth.

That's the biggest issue, but I think a lot of us use a lot of small
arrays as well -- and while I don't know if it's a performance hit worth
worrying about, it's always bugged me that is is faster to convert to a
python list, then pass it in to wxPython than it is to just pass in the
array directly.

You should have seen it in the old days before I sped it up. Actually I think you probably did.

Anyway, it seems like wxPython is low hanging fruit in the sense that we could probably get it done without too much trouble and it would be pretty easy. It's possible that Robin may not accept the patch until the relevant code goes into Python, but just having a patch available would be a useful template for other project and would show the performance gains this approach would lead to. At least I sure hope so.

Travis: does the code implementing the C API exist already, or is that something that still needs to be written?