[Tutor] Question on rounding
Wed, 10 Jan 2001 01:14:12 -0500
> I know this must have been discussed before,
Actually, it's been discussed a lot since binary computers first came into
use by "real people" <wink> in the middle of the last century!
> but I'm having a problem with rounding. My hand calculator gives
> me the following: 25350 * .0055 = 139.425
Yup. Your calculator uses decimal arithmetic. Almost all computers use
binary arithmetic instead, and most decimal real numbers can't be
represented exactly in binary format. There's a detailed example here:
but you may find it hard to understand. In part that's because it simply
doesn't have an easy answer. The odd truth is that 139.425 cannot be
exactly represented in binary at all.
This is similar to that your calculator can't represent 1/3 exactly. That's
necessarily loses some information if you cut it off after any finite number
of digits. For example, on an 8-digit calculator the closest you can get is
and while that's *close* to 1/3, it's not 1/3 exactly.
When you're expecting 139.425 but getting things like 139.42499999999998 or
139.42500000000001 instead, the same thing is happening except in a
different base: they can't be represented exactly in base 2, but you can
get *close* in base 2.
This isn't unique to Python, by the way. It's the same story in any
language that uses all the "floating point" hardware you paid for <wink>.