Hrounding error

Jerry Hill malaclypse2 at gmail.com
Wed Jun 18 17:02:32 CEST 2008


On Wed, Jun 18, 2008 at 1:47 AM,  <cooperq at gmail.com> wrote:
>>>> 234 - 23234.2345
> -23000.234499999999
>
> This is not correct by my calculations.

Python floating point operations use the underlying C floating point
libraries which, in turn, usually rely on the hardware's floating
point implementations.  This Wikipedia article talks about how those
values are usually stored and manipulated:
http://en.wikipedia.org/wiki/IEEE_floating-point_standard

So, which IEEE double precision floating point value would you like
instead?  As far as I understand it, these are your two choices:

'ba490c020f76d6c0' = -23000.234499999999
'bb490c020f76d6c0' = -23000.234500000002

Alternatively, investigate the Decimal module, but keep in mind that
it can have the same sorts of issues, just on different numbers.
Compare, for instance:

>>> (1.0/3.0) * 3.0
1.0

>>> from decimal import Decimal
>>> ( Decimal('1.0')/Decimal('3.0') ) * Decimal('3.0')
Decimal("0.9999999999999999999999999999")
>>>

-- 
Jerry



More information about the Python-list mailing list