[Python-Dev] memoryview: "B", "c", "b" format specifiers

[Stefan Krah]

Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 18 Aug 2011 18:57:00 +0200
> Stefan Krah <stefan at bytereef.org> wrote:
> > 
> > Oh no, the name isn't quite right then. It should be a replacement
> > for the combination PyBuffer_FillInfo()/PyMemoryView_FromBuffer()
> > and it should temporarily wrap a C-string.
> 
> Ah, nice.
> 
> > PyObject * PyMemoryView_FromCString(char *s, Py_ssize_t size, int flags);
> 
> It's not really a C string, since it's not null-terminated.
> PyMemoryView_FromMemory?
> 
> (that would mirror PyUnicode_FromUnicode, for example)

I see, yes. PyMemoryView_FromStringAndSize()? No, too much typing. I prefer
PyMemoryView_FromMemory().


> > 'flags' is just PyBUF_READ or PyBUF_WRITE.
>
> Why do we have these in addition to PyBUF_WRITABLE already?

That's a bit involved, this is how I see it:

There are four buffer *request* flags that can be sent to a buffer provider
and that indicate the amount of complexity that a consumer can handle (in
decreasing order):

PyBUF_INDIRECT  -> suboffsets   (PIL-style)
PyBUF_STRIDES   -> strides      (Numpy-style)
PyBUF_ND        -> C-contiguous, but possibly multi-dimensional
PyBUF_SIMPLE    -> contiguous, one-dimensional, unsigned bytes

Each of those flags can be mixed freely with two additional flags:

PyBUF_WRITABLE
PyBUF_FORMAT

All other buffer request flags are simply combinations of those.
For example, if you use PyBUF_WRITABLE as the only flag, logically
it should be seen as PyBUF_WRITABLE|PyBUF_SIMPLE (this works since
PyBUF_SIMPLE is defined as 0).


PyBUF_READ and PyBUF_WRITE are so far only used for PyMemoryView_GetContiguous().
The PEP still has a flag named PyBUF_UPDATEIFCOPY, but that didn't make it
into object.h.

I thought it might be appropriate to use PyBUF_READ and PyBUF_WRITE
to underline the fact that you cannot send a fine grained buffer
request to PyMemoryView_FromMemory()[1]. Also, PyBUF_READ is easier
to understand than PyBUF_SIMPLE.


But I'd be equally happy with PyBUF_SIMPLE/PyBUF_WRITABLE.


Stefan Krah

[1] The terminology might sound funny, but there is a function that
can act a micro buffer provider:

int PyBuffer_FillInfo(Py_buffer *view, PyObject *obj, void *buf, Py_ssize_t len,
                      int readonly, int infoflags)

An exporter can use this function as a building block for a getbuffer()
method for unsigned bytes, since it reacts correctly to *all* possible
buffer requests in 'infoflags'.