Comparing float and decimal
timr at probo.com
Sat Sep 27 05:34:43 CEST 2008
Mark Dickinson <dickinsm at gmail.com> wrote:
>On Sep 25, 8:55 am, Tim Roberts <t... at probo.com> wrote:
>> Marc 'BlackJack' Rintsch <bj_... at gmx.net> wrote:
>> >0.1 actually is
>> >In : '%.50f' % 0.1
>> >Out: '0.10000000000000000555111512312578270211815834045410'
>> Actually, it's not. Your C run-time library is generating random digits
>> after it runs out of useful information (which is the first 16 or 17
>> digits). 0.1 in an IEEE 784 double is this:
>I get (using Python 2.6):
>>>> n, d = 0.1.as_integer_ratio()
>>>> from decimal import Decimal, getcontext
>>>> getcontext().prec = 100
>which is a lot closer to Marc's answer. Looks like your float
>approximation to 0.1 is 6 ulps out. :-)
Hmmph, that makes the vote 3 to 1 against me. I need to go re-examine my
"extreme float converter".
Tim Roberts, timr at probo.com
Providenza & Boekelheide, Inc.
More information about the Python-list