Exposing buffer interface for non-extension types?

[Ken Watford]

On Tue, Jul 20, 2010 at 8:28 PM, Carl Banks <pavlovevidence at gmail.com> wrote:
> On Jul 20, 3:09 pm, Ken Watford <kwatford+pyt... at gmail.com> wrote:
>> Is there any way to expose the PEP 3118 buffer interface for objects
>> that aren't extension types?
>>
>> Currently, I can expose the NumPy array interface (using either
>> __array_interface__ or __array_struct__) for any class, extension or
>> otherwise. But I can't find any reference to python-side interfacing
>> for PEP 3118. SWIG makes an extension module for your wrapped code,
>> but not extension *types*, so the classes it produces are pure-python
>> with methods added in from the extension module.
>>
>> The NumPy array interface works fine for now (especially since NumPy
>> is the only thing I need to consume these objects), but the
>> documentation claims that it's being deprecated in favor of PEP 3118,
>> so I thought it might be relevant to bring this up.
>
> Can you tell let us know what you want to use it for?  We could offer
> better help.
>
> Numpy is generally how I get at buffers in Python 2.x.  For instance
> if I have an object m that supports buffer protocol (m could a string,
> mmap object, Python array, etc.), then the following will create an
> array view of the same buffer:
>
> numpy.ndarray((10,10),type=numpy.float32,buffer=m)
>
> As far as I know this procedure won't be too different under PEP 3118;
> if anything it's simplified in Python 3 since it can discover type and
> shape information itself.  (You'll have to check with the numpy people
> on that.)

I'm not having trouble using buffers, I'm having trouble providing them.

As a part of SWIG-wrapping a larger C++ project, I'm producing some
wrappings for Blitz++ arrays. I can extract the shape and stride
information from the array object to fill either NumPy's or PEP 3118's
appropriate structure. In the case of NumPy, I can easily arrange for
the necessary interface on the proxy object to be fulfilled, because
NumPy doesn't care what kind of object it's attached to. But the PEP
3118 interface can only be provided by C extension types.

One possibility I've considered is injecting a small extension type
into the wrapper that provides PEP 3118 by reading the NumPy array
interface info off of the object, and then inject it into all
appropriate SWIG-generated proxy classes as an additional base class.

This isn't a big deal for me - the array interface works just fine,
and probably will for longer than I'll be working on this project -
but it just struck me as strange that some of my existing
array-interface-enabled classes can't be trivially ported to PEP 3118
because they're defined in pure Python modules rather than extension
modules.