[Python-3000] PEP Draft: Enhancing the buffer protcol

[Travis E. Oliphant]

Greg Ewing wrote:
> Travis E. Oliphant wrote:
> 
>>     typedef char *(*formatbufferproc)(PyObject *view, int *itemsize)
>>
>>       Get the format-string of the memory using the struct-module
>>       string syntax
> 
> I'm not sure whether a struct-format string would be
> the most convenient form for use by C-level code, as
> it could require some tedious parsing to extract
> useful information from it.

Yes, this was the reason for my dtype object.  But, I think that folks 
felt it was too much, especially since the struct-style syntax is 
already there in Python.

Do you have any other suggestions?

> 
>>     typedef PyObject *(*shapebufferproc)(PyObject *view)
>>
>>       Return a 2-tuple of lists containing shape information: (shape,
>>       strides).
> 
> I'm also not sure about using Python data structures
> to represent this, as it will force C-level code to
> use Python API calls to pull it apart. What would be
> wrong with C array of structs containing two integers
> each?

Nothing except memory management.  Now, you have to worry about 
allocating and deallocating memory.

> 
> The buffer API is for the use of C code, and it should
> be designed with the convenience of C code in mind.

I agree.  I would like to use something besides Python objects, but 
handling the memory allocation is non-trivial.

On the other hand, Python tuples are pretty simple wrappers around 
integers.

> Using Python data structures unnecessarily seems like
> the wrong way to go about that.
> 
> The following alternative would seem to provide most of
> the things that Travis's proposal does without involving
> Python objects:
> 
>      struct pybuffer_shape {
>         Py_ssize_t length;
>         Py_ssize_t stride;
>      };
> 
>      typedef int (*getbufferproc)(PyObject *obj,
>         void **buf, Py_ssize_t *len,
>         char **format,
>         struct pybuffer_shape **shape, int *ndim);
> 
>         /* Any of buf, format and shape may be NULL if you're
>            not interested in them. */
>

Besides not allowing for the request of a "contiguous" buffer from the 
object or a writeable one you are also not describing how allocation for 
this array of structs will be handled.

I'm not opposed in principle.  In fact, I would like to get rid of the 
Python objects in the protocol (in the array_struct interface for NumPy 
we have the shape and strides in an array of integers).

The memory management is the only issue.

-Travis