[Python-Dev] Allocation of shape and strides fields in Py_buffer

[Travis Oliphant]

Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> For the slicing problem in particular, memoryview is currently trying to
>> get away with only one Py_buffer object when it needs TWO.
> 
> Why should it need two? Why couldn't the embedded Py_buffer fullfill all the
> needs of the memoryview object? If the memoryview can't be a relatively thin
> object-oriented wrapper around a Py_buffer, then this all screams failure to me.
>

The advice to look at NumPy is good because memoryview is modeled after 
NumPy -- and never completed.

When a slice view is made, a new memoryview object is created with a 
Py_buffer  structure that needs to allocate it's own shape and strides 
(or something that will allow correct shape and strides to be reported 
to any consumer).  In this way, there are two Py_buffer structures.

I do not remember implementing slicing for memoryview objects and it 
looks like the problem is there.


> ----
> 
> In all honesty, I admit I am annoyed by all the problems with the buffer API /
> memoryview object, many of which are caused by its utterly bizarre design (and
> the fact that the design team went missing in action after imposing such a
> bizarre and complex design on us), and I'm reluctant to add yet another level of
> byzantine complexity in order to solve those problems. It explains I may sound a
> bit angry at times :-)

I understand your frustration, but I've been here (just not able to 
follow python-dev), and I've tried to respond to issues that came to my 
attention.   I did not have time to complete the memoryview 
implementation, but that does not meen the buffer API is "bizarre".

Yes, the cobbled together memoryview object itself may be "bizarre", but 
that is sometimes the reality of volunteer work.  Just ignore the 
memoryview object if it does not meet your needs.

Please let me know what other problems exist.

> 
> If we really need to change things a lot to make them work, we should re-work
> the buffer API from the ground up, make the Py_buffer struct a true PyObject
> (that is, a true variable-length object so as to solve the shape and strides
> allocation issue) and merge it with the current memoryview implementation. It
> would make things both more simpler and more flexible.
> 

The only place there is a shape/strides allocation issue is with the 
memoryview object itself.   There is not an issue as far as I can see 
with the buffer protocol itself.

I'm glad you are trying to help clean up the memoryview implementation. 
         I welcome the eyes and the keystrokes.  Are you familiar at all 
with NumPy?  That may help you understand what you currently consider to 
be "utterly bizarre"

Best regards,

-Travis