Decimal() instead of float?
carsten at uniqsys.com
Fri Nov 17 23:20:49 CET 2006
On Fri, 2006-11-17 at 16:51 -0500, Michael B. Trausch wrote:
> On Fri, 2006-11-17 at 21:25 +0100, Fredrik Lundh wrote:
> > > Some of the lat/long pairs that I have used seem to come out fine, but
> > > some do not. Because the mathmatics used with them involve complex
> > > equations when determining distance and the like, any error gets
> > > massively compounded before a final result is evident.
> > sorry, but I don't think you have the slightest idea what you're doing,
> > really.
> Sure, I do. Let's say that I want to work with the latitude
> 33.6907570. In Python, that number can not be stored exactly without
> the aid of decimal.Decimal().
> >>> 33.6907570
> As you can see, it loses accuracy after the 6th decimal place. That's
> not good enough: I have 8 numbers that need to be exact, and I am
> only getting six.
>>> x = 33.6907570
>>> for i in range(7,16): print "%.*f" % (i,x)
Looks to me like you're getting, uh, 14 decimal digits, for a total of
16 significant digits. What kinds of operations are you performing that
compound this error by seven orders of magnitude?
More information about the Python-list