[Numpy-discussion] BigInteger equivalent in numpy

[wierob]

Hi,

int64 and float seem to work for the stderr calculation. Now, the 
calculation of the p-value causes an underflow.

File "C:\Python26\lib\site-packages\scipy\stats\distributions.py", line 2829,in _cdf
    return special.stdtr(df, x)
FloatingPointError: underflow encountered in stdtr

It seems that the error occurs in scipy.special.stdtr(df, x) if df = 
array([13412]) and x = array([61.88071696]).

>>> from scipy import special
>>> import numpy
>>> df = numpy.array([13412])
>>> x = numpy,array([61.88071696])
>>> special.stdtr(df,x)
array([ 1.])
>>> numpy.seterr(all="raise")
>>> special.stdtr(df,x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FloatingPointError: underflow encountered in stdtr

So, is there another function or datatype that can handle this?

Besides, the overlow in stderr calculation caused nan as result value 
whereas the underflow in p-value calculation eventually leads to 0.0 
(returned by linregress) which is somewhat inconsistent. And in the 
latter case, tricky to identify as an error.

kind regards
robert

josef.pktd at gmail.com schrieb:
> On Thu, Jun 4, 2009 at 8:19 AM, wierob <wierob83 at googlemail.com> wrote:
>   
>> Hi,
>>
>> is there a BigInteger equivalent in numpy? The largest integer type I
>> wound was dtype int64.
>>
>> I'm using stats.linregress to perform a regression analysis. The return
>> stderr was nan because stas.ss(...) returned a negative number due to an
>> overflow. Setting dtype to int64 for my input data seems to fix this.
>> But what if my data does not fit in int64?
>>
>> Since Python's long type can hold large data I tried to convert my input
>> to long but it gets converted to int64 in numpy.
>>
>>     
>
> you could try to use floats. stats.ss does the calculation in the same
> type as the input.
> If you convert your input data to floating point you will not get an
> overflow, but floating point precision instead.
>
> Note during the last bugfix, I also changed the implementation of
> stats.linregress and now (0.7.1 and later) it doesn't use stats.ss
> anymore, instead it uses np.cov which always uses floats.
> Also, if you are using an older version there was a mistake in the
> stderr calculations, http://projects.scipy.org/scipy/ticket/874
>
> Josef
>
>   
>> kind regards
>> robert
>>     
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>