problem with view() and strided arrays?
Hello all, I'm writing up a general function to allocate aligned numpy arrays (I'll post it shortly, as Anne suggested that such a function would be useful). However, I've run into trouble with using ndarray.view() in odd corner- cases: In : numpy.__version__ Out: '1.1.0.dev5077' In : a = numpy.ones((3,8),dtype=numpy.uint8) In : a.view(numpy.uint16) Out: array([[257, 257, 257, 257], [257, 257, 257, 257], [257, 257, 257, 257]], dtype=uint16) In : a = numpy.ones((3,9),dtype=numpy.uint8) In : a[:,1:].view(numpy.uint16) ValueError: new type not compatible with array. In : a[:,:-1].view(numpy.uint16) ValueError: new type not compatible with array. In : numpy.array(a[:,:-1]).view(numpy.uint16) Out: array([[257, 257, 257, 257], [257, 257, 257, 257], [257, 257, 257, 257]], dtype=uint16) It seems like 'view' won't create a view where the stride length is not a whole multiple of the dtype's itemsize. However, since strides are measured in bytes (right?), this shouldn't be a problem. Is this a minor bug? Or am I woefully misunderstanding the issue involved here? Thanks, Zach
It seems like 'view' won't create a view where the stride length is not a whole multiple of the dtype's itemsize. However, since strides are measured in bytes (right?), this shouldn't be a problem.
Actually -- it seems like view() doesn't work with strided arrays at all. (?) In : a = numpy.ones((4,32), dtype=numpy.uint8) In : a.view(numpy.uint16).shape Out: (4, 16) In : a[:,:16].view(numpy.uint16) ValueError: new type not compatible with array. I think this might be a recent regression, because before I updated my numpy installation to the latest SVN version (to check if the bug was fixed!), I'm pretty sure this kind of operation worked. Zach
On Thu, Apr 24, 2008 at 10:00 AM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
It seems like 'view' won't create a view where the stride length is not a whole multiple of the dtype's itemsize. However, since strides are measured in bytes (right?), this shouldn't be a problem.
Actually -- it seems like view() doesn't work with strided arrays at all. (?)
In : a = numpy.ones((4,32), dtype=numpy.uint8)
In : a.view(numpy.uint16).shape Out: (4, 16)
In : a[:,:16].view(numpy.uint16) ValueError: new type not compatible with array.
I think this might be a recent regression, because before I updated my numpy installation to the latest SVN version (to check if the bug was fixed!), I'm pretty sure this kind of operation worked.
Views only work for items of the same size. You are trying to view bytes as two byte values. View is like strict pointer aliasing in C. Chuck
On Thu, Apr 24, 2008 at 10:53 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
Views only work for items of the same size. You are trying to view bytes as two byte values. View is like strict pointer aliasing in C.
No, it works just fine if you have a contiguous array. In [5]: from numpy import * In [6]: a = ones(10, dtype=uint8) In [7]: a Out[7]: array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=uint8) In [8]: b = a.view(uint16) In [9]: b Out[9]: array([257, 257, 257, 257, 257], dtype=uint16) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi all,
Actually -- it seems like view() doesn't work with strided arrays at all. (?)
In : a = numpy.ones((4,32), dtype=numpy.uint8)
In : a.view(numpy.uint16).shape Out: (4, 16)
In : a[:,:16].view(numpy.uint16) ValueError: new type not compatible with array.
I think this might be a recent regression, because before I updated my numpy installation to the latest SVN version (to check if the bug was fixed!), I'm pretty sure this kind of operation worked.
The problem starts out in arrayobject.c:6392, in array_descr_set(), where the following test is performed: if ((newtype->elsize != self->descr->elsize) && \ (self->nd == 0 || !PyArray_ISONESEGMENT(self) || \ newtype->subarray)) goto fail; I *think* I could fix it by relaxing the restrictions to only require that the array have at least one dimension where self->strides[dim] == self->descr->elsize, and then adjust the size of that dimension. Here's my perhaps-fix. The old code is: if ((newtype->elsize != self->descr->elsize) && \ (self->nd == 0 || !PyArray_ISONESEGMENT(self) || \ newtype->subarray)) goto fail; if (PyArray_ISCONTIGUOUS(self)) index = self->nd - 1; else index = 0; My suggested fix is: if ((newtype->elsize != self->descr->elsize) && \ (self->nd == 0 || newtype->subarray)) goto fail; if (PyArray_ISCONTIGUOUS(self)) index = self->nd - 1; else if (PyArray_ISFORTRAN(self)) index = 0; else { int index_found = FALSE; for (index = 0; index < self->nd; index++) { if (self->strides[index] == self->descr->elsize) { index_found = TRUE; break; } } if (!index_found) goto fail; } Could someone look this over? If it looks basically right, I'll make this a proper patch and post it to trac. Thanks, Zach
participants (3)
-
Charles R Harris -
Robert Kern -
Zachary Pincus