# Decimal arithmatic, was Re: Python GUI app to impress the boss?

Chris Gonnerman chris.gonnerman at newcenturycomputers.net
Mon Sep 30 01:13:33 EDT 2002

```----- Original Message -----
From: "Dennis Lee Bieber" <wlfraed at ix.netcom.com>

> Chris Gonnerman fed this fish to the penguins on Sunday 29 September
> 2002 03:33 pm:
>
>
> > You ask how it is a different problem.  Simple.  A human,
> > training in decimal math in grade school, knows that
> >
> >     1/3 = 0.3333333...
> >
> > and would *expect* it to round to 0.33 (still in two
> > decimal
>
>         So, in 2 decimal place arithmetic
>
>         a = 1.00 / 3.00
>         b = a * 3.00
>
> gives
>         b = 0.99
>
> which is NOT equal to 1... But this is okay because it's done
> is "decimal arithmetic"? <G>

It's not "OK"... it's EXPECTED.

>         The telling phrase is "training in DECIMAL math"...
> Most computers don't work in decimal math, so the
> user /should/ be ready to learn the ins&outs of binary
> floating point...

Incorrect.  The *user* expects the dang computer to produce
the same results as for a manual calculation.  In other words,
truly serious businesspeople expect to be able to hand-check
the numbers.  The occasional (but far too common) binary float
anomalies are unacceptable in that environment.

>
>         For every example of "what is wrong with binary
> floating point" someone else can find a similar case when
> using decimal math.

True, but the decimal anomalies are *expected* and the binary
anomalies are *surprising*...  I've been a programmer for
a long time, working in everything from a scientific lab to
a retail store.  I still don't understand the ins and outs,
as you put it, of binary floating point.

> Even VB's "currency" type (which looks to be a scaled 64-bit
> integer) carries 4 decimal places -- which allows for
> "proper" results when displaying final results in two decimal
> places with rounding.
>
>         1/3 => 0.3333   internally
>         3*0.3333 => 0.9999
>         1.00 for display after rounding.

I hate to give anything to Microsoft, but they seem to have a
good, workable, not-often-surprising idea here.

I still prefer the FixedPoint.py module's programmer-controlled
scaling.

Chris Gonnerman -- chris.gonnerman at newcenturycomputers.net
http://newcenturycomputers.net

```