It seems that reshape doesn't work correctly on an array which has been resized using the 0-stride trick e.g. In [73]: x = array([5]) In [74]: y = as_strided(x, shape=(10,), strides=(0,)) In [75]: y Out[75]: array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5]) In [76]: y.reshape([10,1]) Out[76]: array([[ 5], [ 8], [ 762933412], [-2013265919], [ 26], [ 64], [ 762933414], [-2013244356], [ 26], [ 64]]) <================ Should all be 5???????? In [77]: y.copy().reshape([10,1]) Out[77]: array([[5], [5], [5], [5], [5], [5], [5], [5], [5], [5]]) In [78]: np.__version__ Out[78]: '1.6.2' Perhaps a clause such as below is required in reshape? if any(stride == 0 for stride in y.strides): return y.copy().reshape(shape) else: return y.reshape(shape) Regards, Dave
Dave Hirschfeld
It seems that reshape doesn't work correctly on an array which has been resized using the 0-stride trick e.g.
In [73]: x = array([5])
In [74]: y = as_strided(x, shape=(10,), strides=(0,))
In [75]: y Out[75]: array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])
In [76]: y.reshape([10,1]) Out[76]: array([[ 5], [ 8], [ 762933412], [-2013265919], [ 26], [ 64], [ 762933414], [-2013244356], [ 26], [ 64]]) <================ Should all be 5????????
In [77]: y.copy().reshape([10,1]) Out[77]: array([[5], [5], [5], [5], [5], [5], [5], [5], [5], [5]])
In [78]: np.__version__ Out[78]: '1.6.2'
Perhaps a clause such as below is required in reshape?
if any(stride == 0 for stride in y.strides): return y.copy().reshape(shape) else: return y.reshape(shape)
Regards, Dave
Though it would be good to avoid the copy which you should be able to do in this case. Investigating further: In [15]: y.strides Out[15]: (0,) In [16]: z = y.reshape([10,1]) In [17]: z.strides Out[17]: (4, 4) In [18]: z.strides = (0, 4) In [19]: z Out[19]: array([[5], [5], [5], [5], [5], [5], [5], [5], [5], [5]]) In [32]: y.reshape([5, 2]) Out[32]: array([[5, 5], [5, 5], [5, 5], [5, 5], [5, 5]]) In [33]: y.reshape([5, 2]).strides Out[33]: (0, 0) So it seems that reshape is incorrectly setting the stride of axis0 to 4, but only when the appended axis is of size 1. -Dave
Hello, looking at the code, when only adding/removing dimensions with size 1, numpy takes a small shortcut, however it uses 0 stride lengths as value for the new one element dimensions temporarily, then replacing it again to ensure the new array is contiguous. This replacing does not check if the dimension has more then size 1. Likely there is a better way to fix it, but the attached diff should do it. Regards, Sebastian On Do, 2012-08-09 at 13:06 +0000, Dave Hirschfeld wrote: > Dave Hirschfeldgmail.com> writes: > > > > > It seems that reshape doesn't work correctly on an array which has been > > resized using the 0-stride trick e.g. > > > > In [73]: x = array([5]) > > > > In [74]: y = as_strided(x, shape=(10,), strides=(0,)) > > > > In [75]: y > > Out[75]: array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5]) > > > > In [76]: y.reshape([10,1]) > > Out[76]: > > array([[ 5], > > [ 8], > > [ 762933412], > > [-2013265919], > > [ 26], > > [ 64], > > [ 762933414], > > [-2013244356], > > [ 26], > > [ 64]]) <================ Should all be 5???????? > > > > In [77]: y.copy().reshape([10,1]) > > Out[77]: > > array([[5], > > [5], > > [5], > > [5], > > [5], > > [5], > > [5], > > [5], > > [5], > > [5]])--- a/numpy/core/src/multiarray/shape.c +++ b/numpy/core/src/multiarray/shape.c @@ -273,21 +273,21 @@ PyArray_Newshape(PyArrayObject *self, PyArray_Dims *newdims, * appropriate value to preserve contiguousness */ if (order == NPY_FORTRANORDER) { - if (strides[0] == 0) { + if ((strides[0] == 0) && (dimensions[0] == 1)) { strides[0] = PyArray_DESCR(self)->elsize; } for (i = 1; i < ndim; i++) { - if (strides[i] == 0) { + if ((strides[i] == 0) && (dimensions[i] == 1)) { strides[i] = strides[i-1] * dimensions[i-1]; } } } else { - if (strides[ndim-1] == 0) { + if ((strides[ndim-1] == 0) && (dimensions[ndim-1] == 1)) { strides[ndim-1] = PyArray_DESCR(self)->elsize; } for (i = ndim - 2; i > -1; i--) { - if (strides[i] == 0) { + if ((strides[i] == 0) && (dimensions[i] == 1)) { strides[i] = strides[i+1] * dimensions[i+1]; } } > > > > In [78]: np.__version__ > > Out[78]: '1.6.2' > > > > Perhaps a clause such as below is required in reshape? > > > > if any(stride == 0 for stride in y.strides): > > return y.copy().reshape(shape) > > else: > > return y.reshape(shape) > > > > Regards, > > Dave > > > > Though it would be good to avoid the copy which you should be able to do in this > case. Investigating further: > > In [15]: y.strides > Out[15]: (0,) > > In [16]: z = y.reshape([10,1]) > > In [17]: z.strides > Out[17]: (4, 4) > > In [18]: z.strides = (0, 4) > > In [19]: z > Out[19]: > array([[5], > [5], > [5], > [5], > [5], > [5], > [5], > [5], > [5], > [5]]) > > In [32]: y.reshape([5, 2]) > Out[32]: > array([[5, 5], > [5, 5], > [5, 5], > [5, 5], > [5, 5]]) > > In [33]: y.reshape([5, 2]).strides > Out[33]: (0, 0) > > So it seems that reshape is incorrectly setting the stride of axis0 to 4, but > only when the appended axis is of size 1. > > -Dave > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
Sebastian Berg
Hello,
looking at the code, when only adding/removing dimensions with size 1, numpy takes a small shortcut, however it uses 0 stride lengths as value for the new one element dimensions temporarily, then replacing it again to ensure the new array is contiguous. This replacing does not check if the dimension has more then size 1. Likely there is a better way to fix it, but the attached diff should do it.
Regards,
Sebastian
Thanks for the confirmation. So this doesn't get lost I've opened issue #380 on GitHub https://github.com/numpy/numpy/issues/380 -Dave
participants (2)
-
Dave Hirschfeld
-
Sebastian Berg