[Python-ideas] Floating point "closeness" Proposal Outline
abarnert at yahoo.com
Mon Jan 19 22:05:51 CET 2015
On Jan 19, 2015, at 12:28, Chris Barker <chris.barker at noaa.gov> wrote:
> Has anyone in this thread, or in any test code you've seen or written, provided a tolerance with more than one significant figure? i.e, I always use 1e-12 or 1e-10, I've never used a tolerance anything like 0.000153427213
If you're working with measurement error or mechanical tolerance or something similar, you often have something with a non-unit mantissa. For example, the first resistor in the examples for the standard color code chart is 4.7KOhm +/- 10%, so you've got an error of 4.7e2.
Also, keep in mind that if you do an error propagation analysis, you're unlikely to get round numbers. Even for a dead-simple case, where you start off with two non-covariant values that are +/- 1.00e-5 and add them, the result is +/- 1.41e-5.
The reasons you often only have one significant figure are (a) you're hand-waving the error analysis, or (b) your error ends up a couple orders of magnitude smaller than your values and you've only got a couple significant digits. (1.234e5 +/- 6.789e0 is no more meaningful than 1.234e5 +/- 1e1, after all.)
More information about the Python-ideas