Decimal arithmatic, was Re: Python GUI app to impress the boss?

[Dennis Lee Bieber]

Chris Gonnerman fed this fish to the penguins on Tuesday 01 October 
2002 06:21 pm:


> In other words, 0.7 * 0.05 should always be 0.035, and
> never 0.034999...
> 

        When I took science classes, I'd have been dinged a few points if I 
turned in 0.035 -- the rule was to report results to the same 
significance as the inputs.

        0.7 * 0.05 => 0.0 in those science classes. The assumption here being 
that the accuracy of measurements is such that the last supplied 
decimal place is already an estimate -- one does not add places which 
imply increased precision.

>>> print "%5f" % (0.7 * 0.05)
0.035000
>>> print "%5.3f" % (0.7 * 0.05)
0.035
>>> print "%5.2f" % (0.7 * 0.05)
 0.03
>>> print "%5.1f" % (0.7 * 0.05)
  0.0


        Any "formal" application likely has formatted output, and under those 
constraints this example passes as "right"

--
 > ============================================================== <
 >   wlfraed at ix.netcom.com  | Wulfraed  Dennis Lee Bieber  KD6MOG <
 >      wulfraed at dm.net     |       Bestiaria Support Staff       <
 > ============================================================== <
 >        Bestiaria Home Page: http://www.beastie.dm.net/         <
 >            Home Page: http://www.dm.net/~wulfraed/             <