Decimal arithmatic, was Re: Python GUI app to impress the boss?

Aahz aahz at pythoncraft.com
Wed Oct 2 12:49:00 EDT 2002


In article <77udna.6n5.ln at ix.netcom.com>,
Dennis Lee Bieber  <wlfraed at ix.netcom.com> wrote:
>Chris Gonnerman fed this fish to the penguins on Tuesday 01 October 
>2002 06:21 pm:
>>
>> In other words, 0.7 * 0.05 should always be 0.035, and
>> never 0.034999...
>
>When I took science classes, I'd have been dinged a few points if
>I turned in 0.035 -- the rule was to report results to the same
>significance as the inputs.
>
>0.7 * 0.05 => 0.0 in those science classes. The assumption here being
>that the accuracy of measurements is such that the last supplied
>decimal place is already an estimate -- one does not add places which
>imply increased precision.

Nope.  Here's the correct calculation:

7x10^-1 * 5x10^-2 = 35x10^-3 -> 4x10^-2 -> 0.04
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/



More information about the Python-list mailing list