[Numpy-discussion] Rolling window (moving average, moving std, and more)

[Eric Firing]

On 12/31/2010 06:29 PM, Erik Rigtorp wrote:
> Hi,
>
> Implementing moving average, moving std and other functions working
> over rolling windows using python for loops are slow. This is a
> effective stride trick I learned from Keith Goodman's
> <kwgoodman at gmail.com>  Bottleneck code but generalized into arrays of
> any dimension. This trick allows the loop to be performed in C code
> and in the future hopefully using multiple cores.
>

An alternative is to go straight to C, with a cython interface.  If you 
look in the num/src subdirectory of 
http://currents.soest.hawaii.edu/hgstage/hgwebdir.cgi/pycurrents/ you 
will find this approach, labeled "ringbuf" and "runstats".  See 
pycurrents/setup.py, and its driver, pycurrents/runsetup.py, to see how 
runstats is presently being built.

Instead of calculating statistics independently each time the window is 
advanced one data point, the statistics are updated.  I have not done 
any benchmarking, but I expect this approach to be quick.

The code is old; I have not tried to update it to take advantage of 
cython's advances over pyrex.  If I were writing it now, I might not 
bother with the C level at all; it could all be done in cython, probably 
with no speed penalty, and maybe even with reduced overhead.

Eric