[Pythonmac-SIG] Re: Number format in IDE

[Chris Barker]

Jon Bradley wrote:

> system where a number that should be really zero (by reality) but isn't
> because of the bit representation would be rounded accordingly.

There is no way for the computer to know what your application's
definition of "close enough to zero" would be: I know I don't want the
machine to throw away ANY bits unless I ask it to.

> But, in most 3d applications they
> do use decimal floating point.

decimal floating point has all the same limitations, EXCEPT that human's
are used to it! We expect this:

>>> 1.0/3
0.33333333333333331

but dont expect:

>>> 1.0/10
0.10000000000000001

then there is :
>>> math.sqrt(2)
1.4142135623730951

which is not representable in finite form in any system!

> But, pointing the err out to your students is also a good idea. 

Absolutely!

"Louis M. Pecora" wrote:

> I think your solution is similar to what is done on some calculators.  More
decimals are calculated than will be displayed, then some algorithm
checks out the
extra ones and decides how to round off.

Actually, many calculators use decimal floating point, just to stay
compatable with what people used to base ten expect.

-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker@home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------