[issue4580] slicing of memoryviews when itemsize != 1 is wrong
report at bugs.python.org
Wed Dec 10 21:48:39 CET 2008
Nick Coghlan <ncoghlan at gmail.com> added the comment:
It's quite possible that I misread the array.array implementation since
I only glanced at the code where it fills in the Py_buffer details - I
saw the point where it set shape to NULL, but there may have been code
later on to fill it in correctly that I missed.
Regardless, a coupe of sanity checks on the Py_buffer contents before
returning from PyObject_GetBuffer may still be useful to avoid the need
for buffer protocol consumers to repeat them all the time (I see this as
similar to the type-correctness and range checks we do for things like
the index protocol).
Antoine, regarding the shapes and strides info for subviews: don't think
of it as reallocating or altering the shape of the underlying buffer.
Think of it as having separate shape and stride information for the
contents of the underlying buffer (i.e. what memoryview gets back from
the PyObject_GetBuffer call) and for the view it exposes through
indexing and the like (i.e. taking into account the start and stop
offsets from the slicing operation). This is what I mean when I say that
the current memoryview relies too heavily on the Py_buffer that
describes the underlying object - the memoryview needs to maintain its
own state *outside* that description. It just so happens that fully
describing an arbitrary fragment of the underlying buffer will take a
number of the fields that would exist in a separate Py_buffer struct
(specifically len, ndim, shape, strides and offsets if we want to cover
the non-contiguous and ndim > 1 cases - for the contiguous 1-D cases, we
don't needs strides or offsets).
Once we consider that aspect, I think it makes the most sense to just
maintain 2 Py_buffer instances inside the memoryview - one that
describes the underlying buffer, and one that describes the memoryview
itself after slicing is taken into account.
I also think it is worth considering changing the memoryview to also
take start/stop/step arguments in addition to the object to be viewed
(initially only supporting step=1, just like slicing, but we should be
able to lift that limitation as the implementation matures). The
FromBuffer C-level constructor could probably go away at that point.
Python tracker <report at bugs.python.org>
More information about the Python-bugs-list