[SciPy-dev] Problem with F distribution, or with me? - error in stats.fatiguelife.rvs

[Robert Kern]

On Sun, Aug 24, 2008 at 13:19,  <josef.pktd at gmail.com> wrote:
>>On Wed, Aug 13, 2008 at 06:42,  <josef.pktd at gmail.com> wrote:
>>> It looks like that there is an error in stats.fatiguelife.rvs
>>
>>Correct. I have a fix. When I run the scipy test suite tomorrow, I'll check it in.
>>
>>--
>>Robert Kern
>
> Hi,
> Running the scipy test suite looks pretty useless for verifying the
> actual distribution except for serious mistakes.

True. I just don't want to break anything else.

I don't really trust the automated K-S tests anyways. They don't have
good parameter coverage. At the sprint yesterday, I wrote a little GUI
to help me go through all of the distributions with Q-Q plots and a
comparison of the histogram to the theoretical PDF with interactive
control over the parameters. The remaining problems are mostly
failures of the machinery rather than problems with the formulae of
the distributions themselves.

> It didn't detect
> before anything wrong with fatiguelife or loggamma (which I think also
> gives incorrect random numbers)
>
> The Kolmogorov test in
> http://projects.scipy.org/scipy/scipy/browser/trunk/scipy/stats/tests/test_distributions.py
> is pretty powerless to detect mistakes in the actual distribution.
> N=30 is too small and the fail threshold for the pval for fatiguelife
> is set to alpha = 0.01, while for the other distributions it is at
> alpha = 0.1.

No, they are 0.001 and 0.01, but your point is taken.

> The pvalue for N=100 or N=1000  should be a much better indicator
> whether the random variable really follows the theoretical
> distribution.

True. After bumping those up to 1000, the tests still pass with after
my fixes. They will remain bumped up to 1000.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco