Decimal arithmatic, was Re: Python GUI app to impress the boss?

Chris Gonnerman chris.gonnerman at newcenturycomputers.net
Mon Sep 30 09:10:41 EDT 2002


----- Original Message ----- 
From: "Paul Rubin" <phr-n2002b at NOSPAMnightsong.com>


> "Chris Gonnerman" <chris.gonnerman at newcenturycomputers.net> writes:
> > > With decimal arithmetic, you get $2.97 the first way and
> > > $3.00 the second way.
> > 
> > Excellent example, wrong focus.  The business user (in this
> > case, the retail store owner/manager) would define which
> > method, and which results, are right.  The second mode is
> > the "canonical" way in the manual system, but lately I've
> > realized that many (most?) retail POS systems can do it 
> > "both" ways, handling single items by a variation of method 
> > 1 which gives answers consistent with method 2.
> > 
> > I'm glad I didn't have to write and debug that code.
> 
> I'm really skeptical that any "business user" can specify a 
> set of rules that gets all this stuff straight for a complex 
> business involving cents-off coupons, percentage advertising 
> discounts, etc.

Usually, in the "canned" systems I've seen, those 3 for a 
buck deals are supported by allowing three to five decimal
places rather than just two for amounts.  Results come out
much as you are expecting there.

It seems that what you are finding fault with (IMHO) is not
decimal arithmetic, it's *fixed* decimal arithmetic.  What
I am finding fault with isn't floating point arithmetic, it's
floating *binary* point arithmetic.  Later today (tonight 
probably) I plan to post a test module comparing the very 
simple "tax" calculation I've been using as an example.  My
code isn't quite clean enough to show yet, and given the 
nature of this discussion I want it real clean.

However, here are some interesting results so far:

    5% of $0.50 = 0.02 decimal, 0.03 float
    5% of $0.70 = 0.04 decimal, 0.03 float
    6% of $0.75 = 0.04 decimal, 0.05 float
    9% of $0.50 = 0.04 decimal, 0.05 float

Sell a lot of fifty-cent items in a 5% tax district, and you'll
wind up with a large discrepancy in your sales tax.  Of course,
it'll be long rather than short...  just watch out for those
seventy-five cent items.

If you sell exactly as many $0.75 as $0.50, you're OK...

In other words, it's just not good enough.

Now, if you can point me at a *floating decimal* math library,
THAT would be COOL.


Chris Gonnerman -- chris.gonnerman at newcenturycomputers.net
http://newcenturycomputers.net





More information about the Python-list mailing list