[Numpy-discussion] BigInteger equivalent in numpy
wierob
wierob83 at googlemail.com
Sun Jun 7 05:40:43 EDT 2009
Hi,
int64 and float seem to work for the stderr calculation. Now, the
calculation of the p-value causes an underflow.
File "C:\Python26\lib\site-packages\scipy\stats\distributions.py", line 2829,in _cdf
return special.stdtr(df, x)
FloatingPointError: underflow encountered in stdtr
It seems that the error occurs in scipy.special.stdtr(df, x) if df =
array([13412]) and x = array([61.88071696]).
>>> from scipy import special
>>> import numpy
>>> df = numpy.array([13412])
>>> x = numpy,array([61.88071696])
>>> special.stdtr(df,x)
array([ 1.])
>>> numpy.seterr(all="raise")
>>> special.stdtr(df,x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FloatingPointError: underflow encountered in stdtr
So, is there another function or datatype that can handle this?
Besides, the overlow in stderr calculation caused nan as result value
whereas the underflow in p-value calculation eventually leads to 0.0
(returned by linregress) which is somewhat inconsistent. And in the
latter case, tricky to identify as an error.
kind regards
robert
josef.pktd at gmail.com schrieb:
> On Thu, Jun 4, 2009 at 8:19 AM, wierob <wierob83 at googlemail.com> wrote:
>
>> Hi,
>>
>> is there a BigInteger equivalent in numpy? The largest integer type I
>> wound was dtype int64.
>>
>> I'm using stats.linregress to perform a regression analysis. The return
>> stderr was nan because stas.ss(...) returned a negative number due to an
>> overflow. Setting dtype to int64 for my input data seems to fix this.
>> But what if my data does not fit in int64?
>>
>> Since Python's long type can hold large data I tried to convert my input
>> to long but it gets converted to int64 in numpy.
>>
>>
>
> you could try to use floats. stats.ss does the calculation in the same
> type as the input.
> If you convert your input data to floating point you will not get an
> overflow, but floating point precision instead.
>
> Note during the last bugfix, I also changed the implementation of
> stats.linregress and now (0.7.1 and later) it doesn't use stats.ss
> anymore, instead it uses np.cov which always uses floats.
> Also, if you are using an older version there was a mistake in the
> stderr calculations, http://projects.scipy.org/scipy/ticket/874
>
> Josef
>
>
>> kind regards
>> robert
>>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
More information about the NumPy-Discussion
mailing list