[Tutor] floating point accuracy [working with Fractions]

bob gailer bgailer at gmail.com
Fri Nov 14 17:01:04 CET 2008


Karen Bester wrote:
> Hi
>  
> Is there a method to determine the accuracy that you can get given a 
> double precision number.  I'm looking for a formula or table that can 
> tell me what accuracy I can get depending on "where" the decimal point 
> lies.

What do you mean by "accuracy"? How do you measure it?

In my vocabulary accuracy means how close to a real world measurement 
the number is.

There are 2 factors affecting floating point accuracy: # of bits for the 
magnitude and whether the real world value can be expressed exactly as a 
binary fraction of that many bits.

http://en.wikipedia.org/wiki/Floating_point asserts that "any integer 
less than or equal to 2^53 can be exactly represented in the double 
precision format".
2^53 = 18014398509481984. math.pi shows up in Python as 
3.1415926535897931. This is an approximation, as pi can't be expressed 
in a finite # of digits.
You can say that math.pi is within 1/18014398509481984 of the value of pi.

I don't think it matters where the decimal point lies.
 

-- 
Bob Gailer
Chapel Hill NC 
919-636-4239

When we take the time to be aware of our feelings and 
needs we have more satisfying interatctions with others.

Nonviolent Communication provides tools for this awareness.

As a coach and trainer I can assist you in learning this process.

What is YOUR biggest relationship challenge?



More information about the Tutor mailing list