On Fri, Dec 19, 2008 at 6:53 AM, John Hunter jdh2358@gmail.com wrote:

On Thu, Dec 18, 2008 at 8:27 PM, Bradford Cross bradford.n.cross@gmail.com wrote:

This is a new project I just released.

I know it is C#, but some of the design and idioms would be nice in numpy/scipy for working with discrete event simulators, time series, and event stream processing.

I think an incremental stats module would be a boon to numpy or scipy. Eric Firing has a nice module wrtten in C with a pyrex wrapper (ringbuf) that does trailing incremental mean, median, std, min, max, and percentile. It maintains a sorted queue to do the last three efficiently, and handles NaN inputs. I would like to see this extended to include exponential or other weightings to do things like incremental trailing exponential moving averages and variances. I don't know what the licensing terms are of this module, but it might be a good starting point for an incremental numpy stats module, at least if you were thinking about supporting a finite lookback window. We have a copy of this in the py4science examples dir if you want to take a look:

svn co https://matplotlib.svn.sourceforge.net/svnroot/matplotlib/trunk/py4science/e... cd trailstats/ make python movavg_ringbuf.py

Other things that would be very useful are incremental covariance and regression.

Some sort of Kalman filter?

Chuck