Problem whit float precision, 1.6 Windows 98

Robin Becker robin at jessikat.fsnet.co.uk
Wed Sep 13 14:39:29 CEST 2000


In article <slrn8rur4i.to7.scarblac-spamtrap at flits104-37.flits.rug.nl>,
Remco Gerlich <scarblac-spamtrap at pino.selwerd.nl> writes
>
...
>If you need a precision of, say, 1/10th of a cent, just multiply the
>amount by 1000 and use integers. Integers are exact.
>
...
you better use long integers if you want to do real world dollar calcs
as you'll only be able to calc up to $2.14M with 32bit ints and .1cent
accuracy. On the other hand floats can hold integers that are a good bit 
in my win32 python
>>> a=2.0**52
>>> (a+1) - a
1.0
>>> a
4.50359962737e+015

so you should be able to do 1/10 cent accurate calcs with floats up to
around 1000 trillion dollars. This won't give you total freedom from
precision errors. If you want 1/3 etc you'll need rationals.
-- 
Robin Becker



More information about the Python-list mailing list