[SciPy-dev] Sad sad sad... Was: Warning about remaining issues in stats.distributions ?

[josef.pktd at gmail.com]

On Sat, Mar 14, 2009 at 8:56 PM, Thouis (Ray) Jones
<thouis at broad.mit.edu> wrote:
> On Thu, Mar 12, 2009 at 23:51,  <josef.pktd at gmail.com> wrote:
>> Fixing numerical integration over the distance of a machine epsilon of
>> a function that has a singularity at the boundary was not very high on
>> my priority list.
>
> I've been working with some code for integration using the
> double-exponential transform, which is useful for exactly this case.
> I'm not familiar with all the issues involved here, but if you would
> like to explore that option, let me know and I'll send you the code.
>
> I was planning on cleaning it up a bit and contributing it to scipy, anyway.

Thank you for the offer, it would be good to have some more tools
available to handle these special cases. There are several
distributions that have a point in the interior or the boundary of the
support where the density function goes to infinity. In general
scipy.integrate.quad seems to be handling them quite well except for
corner cases. But I don't know what the precision loss in these cases
is.

For the specific case of the r-distribution, rdist, the main source of
the problem is that I had to disable the explicit formula for the
cumulative distribution function because scipy.special.hyp2f1 produces
incorrect numbers over part of the parameter range used in rdist, and
I had to fall back to numerical integration.

But for me there is no hurry for these corner case problems, since, in
my available time, I still have other problems or improvements in
stats to chase, (and for me it's not so much fun to figure out what
epsilon multiplied by (going to infinity) is, especially if it's not
clear what the use case for it is.)

Josef