[Numpy-discussion] Rolling window (moving average, moving std, and more)
kwgoodman at gmail.com
Mon Jan 3 10:52:28 EST 2011
On Mon, Jan 3, 2011 at 7:41 AM, Erik Rigtorp <erik at rigtorp.com> wrote:
> On Mon, Jan 3, 2011 at 10:36, Keith Goodman <kwgoodman at gmail.com> wrote:
>> On Mon, Jan 3, 2011 at 5:37 AM, Erik Rigtorp <erik at rigtorp.com> wrote:
>>> It's only a view of the array, no copying is done. Though some
>>> operations like np.std() will copy the array, but that's more of a
>>> bug. In general It's hard to imagine any speedup gains by copying a
>>> 10GB array.
>> I don't think that np.std makes a copy of the input data if the input
>> is an array. If the input is, for example, a list, then an array is
> When I tried it on a big array, it tried to allocate a huge amount of
> memory. As I said it's probably a bug.
Yes, that would be a big bug.
np.std does have to initialize the output array. If the window size is
small compared to arr.shape[axis] then the memory taken by the output
array is of the same order as that of the input array. Could that be
what you are seeing?
>> a = np.arange(10)
Small window, output array shape (8,):
>> rolling_window(a, 2)
Big window, output array shape (2,):
>> rolling_window(a, 9)
array([[0, 1, 2, 3, 4, 5, 6, 7, 8],
[1, 2, 3, 4, 5, 6, 7, 8, 9]])
More information about the NumPy-Discussion