# Decimal arithmatic, was Re: Python GUI app to impress the boss?

Dennis Lee Bieber wlfraed at ix.netcom.com
Sun Sep 29 23:17:02 EDT 2002

```Chris Gonnerman fed this fish to the penguins on Sunday 29 September
2002 03:33 pm:

> You ask how it is a different problem.  Simple.  A human,
> training in decimal math in grade school, knows that
>
>     1/3 = 0.3333333...
>
> and would *expect* it to round to 0.33 (still in two decimal

So, in 2 decimal place arithmetic

a = 1.00 / 3.00
b = a * 3.00

gives
b = 0.99

which is NOT equal to 1... But this is okay because it's done is
"decimal arithmetic"? <G>

The telling phrase is "training in DECIMAL math"... Most computers
don't work in decimal math, so the user /should/ be ready to learn the
ins&outs of binary floating point...

For every example of "what is wrong with binary floating point"
someone else can find a similar case when using decimal math. Even VB's
"currency" type (which looks to be a scaled 64-bit integer) carries 4
decimal places -- which allows for "proper" results when displaying
final results in two decimal places with rounding.

1/3 => 0.3333   internally
3*0.3333 => 0.9999
1.00 for display after rounding.

--
> ============================================================== <
>   wlfraed at ix.netcom.com  | Wulfraed  Dennis Lee Bieber  KD6MOG <
>      wulfraed at dm.net     |       Bestiaria Support Staff       <
> ============================================================== <