Significant digits in a float?
Ned Batchelder
ned at nedbatchelder.com
Tue Apr 29 12:47:22 EDT 2014
On 4/29/14 12:30 PM, Chris Angelico wrote:
> On Tue, Apr 29, 2014 at 11:38 PM, Roy Smith <roy at panix.com> wrote:
>> I'm trying to intuit, from the values I've been given, which coordinates
>> are likely to be accurate to within a few miles. I'm willing to accept
>> a few false negatives. If the number is float("38"), I'm willing to
>> accept that it might actually be float("38.0000"), and I might be
>> throwing out a good data point that I don't need to.
>
> You have one chance in ten, repeatably, of losing a digit. That is,
> roughly 10% of your four-decimal figures will appear to be
> three-decimal, and 1% of them will appear to be two-decimal, and so
> on. Is that "a few" false negatives? It feels like a lot IMO. But
> then, there's no alternative - the information's already gone.
>
Reminds me of the story that the first survey of Mt. Everest resulted in
a height of exactly 29,000 feet, but to avoid the appearance of an
estimate, they reported it as 29,002: http://www.jstor.org/stable/2684102
--
Ned Batchelder, http://nedbatchelder.com
More information about the Python-list
mailing list