[Numpy-discussion] Bug in as_strided/reshape

[Sebastian Berg]

Hello,

looking at the code, when only adding/removing dimensions with size 1,
numpy takes a small shortcut, however it uses 0 stride lengths as value
for the new one element dimensions temporarily, then replacing it again
to ensure the new array is contiguous.
This replacing does not check if the dimension has more then size 1.
Likely there is a better way to fix it, but the attached diff should do
it.

Regards,

Sebastian

On Do, 2012-08-09 at 13:06 +0000, Dave Hirschfeld wrote:
> Dave Hirschfeld <dave.hirschfeld <at> gmail.com> writes:
> 
> > 
> > It seems that reshape doesn't work correctly on an array which has been
> > resized using the 0-stride trick e.g.
> > 
> > In [73]: x = array([5])
> > 
> > In [74]: y = as_strided(x, shape=(10,), strides=(0,))
> > 
> > In [75]: y
> > Out[75]: array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])
> > 
> > In [76]: y.reshape([10,1])
> > Out[76]: 
> > array([[          5],
> >        [          8],
> >        [  762933412],
> >        [-2013265919],
> >        [         26],
> >        [         64],
> >        [  762933414],
> >        [-2013244356],
> >        [         26],
> >        [         64]]) <================ Should all be 5????????
> > 
> > In [77]: y.copy().reshape([10,1])
> > Out[77]: 
> > array([[5],
> >        [5],
> >        [5],
> >        [5],
> >        [5],
> >        [5],
> >        [5],
> >        [5],
> >        [5],
> >        [5]])--- a/numpy/core/src/multiarray/shape.c
+++ b/numpy/core/src/multiarray/shape.c
@@ -273,21 +273,21 @@ PyArray_Newshape(PyArrayObject *self, PyArray_Dims
*newdims,
          * appropriate value to preserve contiguousness
          */
         if (order == NPY_FORTRANORDER) {
-            if (strides[0] == 0) {
+            if ((strides[0] == 0) && (dimensions[0] == 1)) {
                 strides[0] = PyArray_DESCR(self)->elsize;
             }
             for (i = 1; i < ndim; i++) {
-                if (strides[i] == 0) {
+                if ((strides[i] == 0) && (dimensions[i] == 1)) {
                     strides[i] = strides[i-1] * dimensions[i-1];
                 }
             }
         }
         else {
-            if (strides[ndim-1] == 0) {
+            if ((strides[ndim-1] == 0) && (dimensions[ndim-1] == 1)) {
                 strides[ndim-1] = PyArray_DESCR(self)->elsize;
             }
             for (i = ndim - 2; i > -1; i--) {
-                if (strides[i] == 0) {
+                if ((strides[i] == 0) && (dimensions[i] == 1)) {
                     strides[i] = strides[i+1] * dimensions[i+1];
                 }
             }
> > 
> > In [78]: np.__version__
> > Out[78]: '1.6.2'
> > 
> > Perhaps a clause such as below is required in reshape?
> > 
> > if any(stride == 0 for stride in y.strides):
> >     return y.copy().reshape(shape)
> > else:
> >     return y.reshape(shape)
> > 
> > Regards,
> > Dave
> > 
> 
> Though it would be good to avoid the copy which you should be able to do in this 
> case. Investigating further:
> 
> In [15]: y.strides
> Out[15]: (0,)
> 
> In [16]: z = y.reshape([10,1])
> 
> In [17]: z.strides
> Out[17]: (4, 4)
> 
> In [18]: z.strides = (0, 4)
> 
> In [19]: z
> Out[19]: 
> array([[5],
>        [5],
>        [5],
>        [5],
>        [5],
>        [5],
>        [5],
>        [5],
>        [5],
>        [5]])
> 
> In [32]: y.reshape([5, 2])
> Out[32]: 
> array([[5, 5],
>        [5, 5],
>        [5, 5],
>        [5, 5],
>        [5, 5]])
> 
> In [33]: y.reshape([5, 2]).strides
> Out[33]: (0, 0)
> 
> So it seems that reshape is incorrectly setting the stride of axis0 to 4, but 
> only when the appended axis is of size 1.
> 
> -Dave
> 
> 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Fix-reshaping-of-arrays-with-stride-0-in-a-dimension.patch
Type: text/x-patch
Size: 1655 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120809/7b57e449/attachment.bin>