[Python-ideas] Floating point "closeness" Proposal Outline

[Andrew Barnert]

On Jan 19, 2015, at 12:28, Chris Barker <chris.barker at noaa.gov> wrote:

> Has anyone in this thread, or in any test code you've seen or written, provided a tolerance with more than one significant figure? i.e, I always use 1e-12 or 1e-10, I've never used a tolerance anything like 0.000153427213

If you're working with measurement error or mechanical tolerance or something similar, you often have something with a non-unit mantissa. For example, the first resistor in the examples for the standard color code chart is 4.7KOhm +/- 10%, so you've got an error of 4.7e2.

Also, keep in mind that if you do an error propagation analysis, you're unlikely to get round numbers. Even for a dead-simple case, where you start off with two non-covariant values that are +/- 1.00e-5 and add them, the result is +/- 1.41e-5.

The reasons you often only have one significant figure are (a) you're hand-waving the error analysis, or (b) your error ends up a couple orders of magnitude smaller than your values and you've only got a couple significant digits. (1.234e5 +/- 6.789e0 is no more meaningful than 1.234e5 +/- 1e1, after all.)