[Tutor] floating point accuracy [working with Fractions]
bob gailer
bgailer at gmail.com
Fri Nov 14 17:01:04 CET 2008
Karen Bester wrote:
> Hi
>
> Is there a method to determine the accuracy that you can get given a
> double precision number. I'm looking for a formula or table that can
> tell me what accuracy I can get depending on "where" the decimal point
> lies.
What do you mean by "accuracy"? How do you measure it?
In my vocabulary accuracy means how close to a real world measurement
the number is.
There are 2 factors affecting floating point accuracy: # of bits for the
magnitude and whether the real world value can be expressed exactly as a
binary fraction of that many bits.
http://en.wikipedia.org/wiki/Floating_point asserts that "any integer
less than or equal to 2^53 can be exactly represented in the double
precision format".
2^53 = 18014398509481984. math.pi shows up in Python as
3.1415926535897931. This is an approximation, as pi can't be expressed
in a finite # of digits.
You can say that math.pi is within 1/18014398509481984 of the value of pi.
I don't think it matters where the decimal point lies.
--
Bob Gailer
Chapel Hill NC
919-636-4239
When we take the time to be aware of our feelings and
needs we have more satisfying interatctions with others.
Nonviolent Communication provides tools for this awareness.
As a coach and trainer I can assist you in learning this process.
What is YOUR biggest relationship challenge?
More information about the Tutor
mailing list