Float precision and float equality
Raymond Hettinger
python at rcn.com
Thu Dec 10 20:23:10 EST 2009
[Carl Banks]
> > You're talking about machine epsilon? I think everyone else here is
> > talking about a number that is small relative to the expected smallest
> > scale of the calculation.
That was also my reading of the OP's question.
The suggestion to use round() was along the
lines of performing a quantize or snap-to-grid
operation after each step in the calculation.
That approach parallels the recommendation for how
to use the decimal module for fixed point calculations:
http://docs.python.org/library/decimal.html#decimal-faq
Raymond
More information about the Python-list
mailing list