[SciPy-User] Determining if statistics are converged

[Kathleen Tacina]

On 2/19/14 2:21 PM, Paul Hobson wrote:
>
>
>
> On Wed, Feb 19, 2014 at 10:47 AM, Kathleen Tacina 
> <kathleen.m.tacina at nasa.gov <mailto:kathleen.m.tacina at nasa.gov>> wrote:
>
>     Hi,
>
>     [snip]
>
>     I have a time series, and I'd like to check that we've taken
>     enough points so that the mean and rms are converged.  I hoping to
>     get help with 2 things:
>
>     (1) Good references on how to do this.
>
>     [snip]
>     (2) Tools to help with this in the scipy ecosystem.
>
>     The application is highly turbulent flow
>     <http://en.wikipedia.org/wiki/Turbulence> where we expect the rms
>     (or, equivalently, the standard deviation) to be on the same order
>     of magnitude as the mean.
>
>     I'd also appreciate suggestions for better places to ask this
>     question.
>
>     Thanks!
>
>     Best regards,
>     Kathleen
>
>
> Hey Kathleen,
>
> It seems to me that a reasonable approach would be to to simply 
> compute the expanding mean. You could then do some rolling inspection 
> of how much the expanding mean has changed with time.
>
> The pandas library has great built-in support for doing this to time 
> series data.
> http://pandas.pydata.org/pandas-docs/stable/computation.html#expanding-window-moment-functions
>
> Maybe something like this:
> http://nbviewer.ipython.org/gist/phobson/9099530
>
> I wish pandas had be around when I was writing my hydraulic 
> engineering master's thesis :)
>
> -paul
>

Paul,

Thanks for the link to the pandas page.  The expanding (and rolling) 
statistics will be very helpful -- much better than having to rewrite 
them myself.

I've looked at the expanding mean.  Unfortunately, for some cases, it 
looks like I'm not converged.  Referring to your (very helpful) ipython 
gist, it would be like we stopped collecting data after 5-15 sec, before 
statistics converge.

Best regards,
Kathleen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140219/37d8de0a/attachment.html>