Hrounding error
Jerry Hill
malaclypse2 at gmail.com
Wed Jun 18 11:02:32 EDT 2008
On Wed, Jun 18, 2008 at 1:47 AM, <cooperq at gmail.com> wrote:
>>>> 234 - 23234.2345
> -23000.234499999999
>
> This is not correct by my calculations.
Python floating point operations use the underlying C floating point
libraries which, in turn, usually rely on the hardware's floating
point implementations. This Wikipedia article talks about how those
values are usually stored and manipulated:
http://en.wikipedia.org/wiki/IEEE_floating-point_standard
So, which IEEE double precision floating point value would you like
instead? As far as I understand it, these are your two choices:
'ba490c020f76d6c0' = -23000.234499999999
'bb490c020f76d6c0' = -23000.234500000002
Alternatively, investigate the Decimal module, but keep in mind that
it can have the same sorts of issues, just on different numbers.
Compare, for instance:
>>> (1.0/3.0) * 3.0
1.0
>>> from decimal import Decimal
>>> ( Decimal('1.0')/Decimal('3.0') ) * Decimal('3.0')
Decimal("0.9999999999999999999999999999")
>>>
--
Jerry
More information about the Python-list
mailing list