[Python-Dev] Allocation of shape and strides fields in Py_buffer

[Antoine Pitrou]

Hello,

The Py_buffer struct has two pointers named `shape` and `strides`. Each points
to an array of Py_ssize_t values whose length is equal to the number of
dimensions of the buffer object. Unfortunately, the buffer protocol spec doesn't
explain how allocation of these arrays should be handled.

Right now this is circumvented by either pointing them to an externally-managed
piece of memory (e.g. a Py_ssize_t field in the original PyObject), or by
pointing them to another field in the Py_buffer (because in the case of a
one-dimensional buffer with itemsize == 1, shape[0] is simply equal to the
length of the buffer in bytes).

Of course this is not flexible, and it makes fixing the situation with buffers
of itemsize larger than 1 difficult (indeed, for those buffers, we can't simply
point the shape array to the byte length, and if we are taking a slice of the
memoryview, we can't either point it to the size of the original object (for
example an array.array)). Therefore, arises the problem of allocation of the
shape array.

For the one-dimensional case, I had in mind a simple scheme where the Py_buffer
struct has an additional two-member Py_ssize_t array. Then `shape` and `strides`
can point to the first and second member of this array, respectively. This
wouldn't solve the multi-dimensional case, however.

Thanks for any ideas on how to solve this.

Regards

Antoine.