[Numpy-discussion] "expected a single-segment buffer object"

[Anne Archibald]

2008/7/9 Robert Kern <robert.kern at gmail.com>:

> Yes, the buffer interface, at least the subset that ndarray()
> consumes, requires that all of the data be contiguous in memory.
> array_as_buffer() checks for that using PyArray_ISONE_SEGMENT(), which
> looks like this:
>
> #define PyArray_ISONESEGMENT(m) (PyArray_NDIM(m) == 0 ||                      \
>                                 PyArray_CHKFLAGS(m, NPY_CONTIGUOUS) ||       \
>                                 PyArray_CHKFLAGS(m, NPY_FORTRAN))
>
> Trying to get a buffer object from anything that is neither C- or
> Fortran-contiguous will fail. E.g.
>
> In [1]: from numpy import *
>
> In [2]: A = arange(10)
>
> In [3]: B = A[::2]
>
> In [4]: ndarray(strides=B.strides, shape=B.shape, buffer=B, dtype=B.dtype)
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
>
> /Users/rkern/today/<ipython console> in <module>()
>
> TypeError: expected a single-segment buffer object

Is this really necessary? What does making this restriction gain? It
certainly means that many arrays whose storage is a contiguous block
of memory can still not be used (just permute the axes of a 3d array,
say; it may even be possible for an array to be in C contiguous order
but for the flag not to be set), but how is one to construct exotic
slices of an array that is strided in memory? (The real part of a
complex array, say.)

I suppose one could follow the linked list of .bases up to the
original ndarray, which should normally be C- or Fortran-contiguous,
then work out the offset, but even this may not always work: what if
the original array was constructed with non-C-contiguous strides from
some preexisting buffer?

If the concern is that this allows users to shoot themselves in the
foot, it's worth noting that even with the current setup you can
easily fabricate strides and shapes that go outside the allocated part
of memory.

> What is the use case, here? One rarely has to use the ndarray
> constructor by itself. For example, the result you seem to want from
> the call you make above can be done just fine with .view().

I was presenting a simple example. I was actually trying to use
zero-strided arrays to implement broadcasting.  The code was rather
long, but essentially what it was meant to do was generate a view of
an array in which an axis of length one had been replaced by an axis
of length m with stride zero. (The point of all this was to create a
class like vectorize that was suitable for use on, for example,
np.linalg.inv().) But I also ran into this problem while writing
segmentaxis.py, the code to produce a "matrix" of sliding windows.
(See http://www.scipy.org/Cookbook/SegmentAxis .) There I caught the
exception and copied the array (unnecessarily) if this came up.

Anne