[Numpy-discussion] Different results from repeated calculation
kwgoodman at gmail.com
Mon Jan 29 14:12:19 EST 2007
On 1/29/07, Charles R Harris <charlesr.harris at gmail.com> wrote:
> That's odd, the LSB bit of the double precision mantissa is only about
> 2.2e-16, so you can't *get* differences as small as 8.4e-22 without about
> 70 bit mantissa's. Hmmm, and extended double precision only has 63 bit
> mantissa's. Are you sure you are computing the error correctly?
That is odd.
8.4e-22 is just the output of the test script: abs(z - z0).max(). That
abs is from python.
More information about the NumPy-Discussion