[issue11128] decimal.py: to_integral() corner cases

Stefan Krah report at bugs.python.org
Mon Feb 7 12:02:17 CET 2011


Stefan Krah <stefan-usenet at bytereef.org> added the comment:

For the record, I prefer Python's behavior. The quantize() definition
does not work well for arbitrary precision input and leads to situations
like:


Precision: 1
Maxexponent: 1
Minexponent: -1

tointegral  101  ->  101
tointegral  101.0  ->  NaN  Invalid_operation


A comment in tointegral.decTest suggests that the to-integral definition 
was modeled after IEEE 854 and 754r:

-- This set of tests tests the extended specification 'round-to-integral
-- value' operation (from IEEE 854, later modified in 754r).
-- All non-zero results are defined as being those from either copy or
-- quantize, so those are assumed to have been tested.
-- Note that 754r requires that Inexact not be set, and we similarly
-- assume Rounded is not set.


This definition of course works fine as long as the input does not
have more than 'precision' digits and Emax is sufficiently large. I
think that for arbitrary precision input the definition should read:


"Otherwise (the operand has a negative exponent) the result is the
 same as using the quantize operation using the given operand as the
 left-hand-operand and 1E+0 as the right-hand-operand. For the purpose
 of quantizing a temporary context is used with the precision of the
 operand as the precision setting, Emax >= prec and Emin <= -Emax. The
 rounding mode is taken from the context, as usual."

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11128>
_______________________________________


More information about the Python-bugs-list mailing list