[Tutor] the binary math "wall"
Steven D'Aprano
steve at pearwood.info
Wed Apr 21 01:39:30 CEST 2010
On Wed, 21 Apr 2010 02:58:06 am Lowell Tackett wrote:
> I'm running headlong into the dilemma of binary math representation,
with game-ending consequences, e.g.:
> >>> 0.15
>
> 0.14999999999999999
>
> Obviously, any attempts to manipulate this value, under the misguided
> assumption that it is truly "0.15" are ill-advised, with inevitable
> bad results.
That really depends on what sort of manipulation you are doing.
>>> x = 0.15
>>> x
0.14999999999999999
>>> x*100 == 15
True
Seems pretty accurate to me.
However:
>>> 18.15*100 == 1815
False
The simplest, roughest way to fix these sorts of problems (at the risk
of creating *other* problems!) is to hit them with a hammer:
>>> round(18.15*100) == 1815
True
[...]
> What I'm shooting for, by the way, is an algorithm that converts a
> deg/min/sec formatted number to decimal degrees. It [mostly] worked,
> until I stumbled upon the peculiar cases of 15 minutes and/or 45
> minutes, which exposed the flaw.
I'm afraid that due to the nature of floating point, this is a hard
problem. Even the professionals at Hewlett-Packard's scientific
calculator division don't always get it right, and they are *extremely*
careful:
http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv018.cgi?read=132690
The best result I can suggest is, change the problem! Don't pass
degrees-minutes-seconds around using a floating point value, but as a
tuple with distinct (DEG, MIN, SEC) integer values. Or create a custom
class.
But if you really need D.MMSS floats, then something like this should be
a good start.:
def dms2deg(f):
"""Convert a floating point number formatted as D.MMSS
into degrees.
"""
mmss, d = math.modf(f)
assert d == int(f)
if mmss >= 0.60:
raise ValueError(
'bad fractional part, expected < .60 but got %f' % mmss)
mmss *= 100
m = round(mmss)
if m >= 60:
raise ValueError('bad minutes, expected < 60 but got %d' % m)
s = round((mmss - m)*100, 8)
if not 0 <= s < 60.0:
raise ValueError('bad seconds, expected < 60.0 but got %f' % s)
return d + m/60.0 + s/3600.0
>>> dms2deg(18.15)
18.25
>>> dms2deg(18.1515)
18.254166666666666
which compares well to my HP-48GX:
18.15 HMS->
gives 18.25, and:
18.1515 HMS->
gives 18.2541666667.
Note though that this still fails with some valid input. I will leave
fixing it as an exercise (or I might work on it later, time
permitting).
--
Steven D'Aprano
More information about the Tutor
mailing list