Decimal arithmatic, was Re: Python GUI app to impress the boss?
James J. Besemer
jb at cascade-sys.com
Wed Oct 2 09:38:40 CEST 2002
Dennis Lee Bieber wrote:
> When I took science classes, I'd have been dinged a few points if I
>turned in 0.035 -- the rule was to report results to the same
>significance as the inputs.
> 0.7 * 0.05 => 0.0 in those science classes. The assumption here being
>that the accuracy of measurements is such that the last supplied
>decimal place is already an estimate -- one does not add places which
>imply increased precision.
Wow. That's pretty Wierd Science, if you ask me. ;o)
You must have gotten a lot of ZEROS whenever using universal constants,
like Planck's Constant, Avogadro's number, Amperes, Joules, Ergs or
Electron Volts vs. any practical quantity in chemistry or physics class.
I think you're confusing some other message about precision. E.g., I
suspect you were supposed to learn that
0.7 * 0.05 => 0.04 ( instead of 0.035)
Precision -- the number of significant digits is independent of the
scale factor, or magnitude, of the number.
James J. Besemer 503-280-0838 voice
2727 NE Skidmore St. 503-280-0375 fax
Portland, Oregon 97211-6557 mailto:jb at cascade-sys.com
More information about the Python-list