[CentralOH] OT: Statistician needed

[Bim Walker]

I just read your other reply and I think I better understand what you are asking now. If you use RMS by dividing by N-1 instead of N you have an unbiased standard deviation. Or you could not take the square root and you would have MS, which is a variance. You can compare two variances with an F-test and get a p-value for a significance test. Confidence intervals are trickier for variances, but this might help:http://www.milefoot.com/math/stat/ci-variances.htm

Does that make any sense?

Bim

------ Original message------From: Eric FloehrDate: Mon, May 16, 2016 6:56 PMTo: Mailing list for Central Ohio Python User Group (COhPy);Cc: Subject:Re: [CentralOH] OT: Statistician needed
Thanks Bim. I have no idea :-). I'm at the limits of my statistical knowledge...
On Mon, May 16, 2016 at 4:11 PM, Bim Walker <bim at digitalbim.com> wrote:

Can’t you just think of the set of errors as the raw data, and do an independent t-test comparing provider A to provider B?  (or an anova if you have more than two providers.)

Bim


On May 16, 2016, at 2:59 PM, Eric Floehr <eric at intellovations.com> wrote:
Hey all,
I'm in need of some help with statistics, and if anyone has any thoughts on this, or know someone who could do this, I would appreciate it greatly.
I have a set of errors, normally distributed around 0 error (it's temperature forecast error). You can assume that the sample of forecasts is representative of the entire population (for example, taking 50 strategic locations around the U.S. to represent all U.S. locations).
I then calculate the mean absolute error, and the RMSE. These have some positive value.
What I would like to calculate on the MAE and RMSE is a confidence interval that the population error is within given the sample MAE or RMSE and it's related statistics (sample size, mean error, MAE, RMSE, standard deviation, etc.).
For example, let's say that one provider's RMSE is 3.18 (A) and another's is 3.5 (B). I'd like to know with some confidence that there is (or isn't) a difference between providers (i.e. that provider A confidently has lower error than B).
Currently, the way I'm doing it is using the normative inverse function in Excel:
Lower bound: NORMINV(0.005,RMSE,STDDEV_RMSE/SQRT(NUMBER_OF_SAMPLES))

Upper bound: NORMINV(0.995,RMSE,STDDEV_RMSE/SQRT(NUMBER_OF_SAMPLES))

as in section 9.18 of: http://www.saylor.org/site/wp-content/uploads/2012/10/BUS204-Ling-6.2.pdf
But I'm not at all convinced that I'm doing that right, or that it applies in this situation.
Thanks so much!Eric
_______________________________________________
CentralOH mailing list
CentralOH at python.org
https://mail.python.org/mailman/listinfo/centraloh


_______________________________________________
CentralOH mailing list
CentralOH at python.org
https://mail.python.org/mailman/listinfo/centraloh


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/centraloh/attachments/20160516/cd9b8ac3/attachment.html>