[Python-Dev] PEP 3118: Extended buffer protocol (new version)

[Travis Oliphant]

Lisandro Dalcin wrote:
> On 4/9/07, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> Travis, all this is far better and simpler than previous approaches...
> Just a few comments

Thanks for your wonderful suggestions.  I've incorporated many of them.
>
> 1) I assume that 'bufferinfo' structure will be part of public Python
> API. In such a case, I think it should be renamed and prefixed.
> Something like 'PyBufferInfo' sounds good?

I prefer that as well. 

>
> 2) I also think 'bufferinfo' could also have an 'itemsize' field
> filled if Py_BUF_ITEMSIZE flag is passed. What do you think? Exporters
> can possibly fill this field more efficiently than next parsing
> 'format' string, it can also save consumers from an API call.
I think the itemsize member is a good idea.   I'm re-visiting what the 
flags should be after suggestions by Carl.
>
> 3) It does make sense to make 'format' be 'const char *' ?
Yes,
>
> 4) I am not sure about this, but perhaps 'buferingo' should save the
> flags passed to 'getbuffer' in a 'flags' field. This can be possibly
> needed at 'releasebuffer' call.
>
I think this is un-necessary.
>
>>   typedef struct {
>>       PyObject_HEAD
>>       PyObject *base;
>>       struct bufferinfo view;
>>       int itemsize;
>>       int flags;
>>   } PyMemoryViewObject;
>
> 5) If my previous comments go in, so 'PyMemoryViewObject' will not
> need 'itemsize' and 'flags' fields (they are in 'bufferinfo'
> structure).
>
After suggestions by Greg, I like the idea of the PyMemoryViewObject 
holding a pointer to another object (from which it gets memory on 
request) as well as information about a slice of that memory. 

Thus, the memory view object is something like:

typedef struct {
      PyObject_HEAD
      PyObject *base;     
      int ndims;
      Py_ssize_t *offsets;    /* slice starts */
      Py_ssize_t *lengths;   /* slice stops */
      Py_ssize_t *skips;       /* slice steps */
} PyMemoryViewObject;

It is more convenient to store any slicing information (so a memory view 
object could store an arbitrary slice of another object) as offsets, 
lengths, and skips which can be used to adjust the memory buffer 
returned by base.

>>     int PyObject_GetContiguous(PyObject *obj, void **buf, Py_ssize_t 
>> *len,
>>                                int fortran)
>>
>> Return a contiguous chunk of memory representing the buffer.  If a
>> copy is made then return 1.  If no copy was needed return 0.
>
> 8) If a copy was made, What should consumers call to free memory?

You are right.  We need a free function.

> 9) What about using a char, like 'c' or 'C', and 'f' or 'F', and 0 or
> 'a' or 'A' (any) ?

I'm happy with that too. 
>
>>     int PyObject_CopyToObject(PyObject *obj, void *buf, Py_ssize_t len,
>>                               int fortran)
>
> 10) Better name? Perhaps PyObject_CopyBuffer or PyObject_CopyMemory?
I'm not sure why those are better names.  The current name reflects the 
idea of copying the data into the object.

>
>>     int PyObject_SizeFromFormat(char *)
>>
>>     int PyObject_IsContiguous(struct bufferinfo *view, int fortran);
>>
>>     void PyObject_FillContiguousStrides(int *ndims, Py_ssize_t *shape,
>>                                         int itemsize,
>>                                         Py_ssize_t *strides, int 
>> fortran)
>>
>>     int PyObject_FillBufferInfo(struct bufferinfo *view, void *buf, 
>> Py_ssize_t len,
>>                                  int readonly, int infoflags)
>>
>
> 11) Perhaps the 'PyObject_' prefix is wrong, as those functions does
> not operate with Python objects.

Agreed.

-Travis