[SciPy-User] Determining if statistics are converged
Paul Hobson
pmhobson at gmail.com
Wed Feb 19 14:21:07 EST 2014
On Wed, Feb 19, 2014 at 10:47 AM, Kathleen Tacina <
kathleen.m.tacina at nasa.gov> wrote:
> Hi,
>
> This is slightly off-topic, but I'm not sure where a better place to ask
> this would be ...
>
> I have a time series, and I'd like to check that we've taken enough points
> so that the mean and rms are converged. I hoping to get help with 2 things:
>
> (1) Good references on how to do this.
>
> I've been using ad hoc methods. For example, comparing the mean of the 1st
> n samples to the overall mean. This works well if, for example, the mean of
> the 1st 50 samples is the same as the mean of all 5,000 samples. But when
> the mean of the 1st 300 samples hasn't yet converged to the mean of all 400
> samples, it isn't as helpful.
>
> (2) Tools to help with this in the scipy ecosystem.
>
> The application is highly turbulent flow<http://en.wikipedia.org/wiki/Turbulence>where we expect the rms (or, equivalently, the standard deviation) to be on
> the same order of magnitude as the mean.
>
> I'd also appreciate suggestions for better places to ask this question.
>
> Thanks!
>
> Best regards,
> Kathleen
>
Hey Kathleen,
It seems to me that a reasonable approach would be to to simply compute the
expanding mean. You could then do some rolling inspection of how much the
expanding mean has changed with time.
The pandas library has great built-in support for doing this to time series
data.
http://pandas.pydata.org/pandas-docs/stable/computation.html#expanding-window-moment-functions
Maybe something like this:
http://nbviewer.ipython.org/gist/phobson/9099530
I wish pandas had be around when I was writing my hydraulic engineering
master's thesis :)
-paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140219/f10cbf04/attachment.html>
More information about the SciPy-User
mailing list