Decimal arithmetic, with example code

Tim Peters tim.one at comcast.net
Tue Oct 1 20:56:56 CEST 2002


[Paul Boddie]
> "Chris Gonnerman" <chris.gonnerman at newcenturycomputers.net> wrote
> in message news:<mailman.1033438201.27954.python-list at python.org>...
> > """fp-test.py -- fixedpoint versus float comparison"""
>
> It's interesting to compare the results from FixedPoint with
> ScaledDecimal and with FixedPoint objects that have increased
> precision. Many issues in the supplied program can explain the
> apparent difference between rounded floating point results and the
> FixedPoint results. Here are some example results from an extended
> version of the original program:
>
> PCT    AMT    Fixed  Fixed  Fixed  Scaled Scaled Float
>               2dp    4dp    -> 2dp 2dp    4dp
> 0.01   0.50   0.00   0.0050 0.00   0.01   0.0050 0.01
> 0.02   0.25   0.00   0.0050 0.00   0.01   0.0050 0.01
> 0.05   0.10   0.00   0.0050 0.00   0.01   0.0050 0.01
> 0.05   0.50   0.02   0.0250 0.02   0.03   0.0250 0.03
> 0.05   0.70   0.04   0.0350 0.04   0.04   0.0350 0.03
> 0.05   0.90   0.04   0.0450 0.04   0.05   0.0450 0.05
> 0.06   0.75   0.04   0.0450 0.04   0.05   0.0450 0.05
> 0.09   0.50   0.04   0.0450 0.04   0.05   0.0450 0.05
> 0.10   0.05   0.00   0.0050 0.00   0.01   0.0050 0.01
> 0.10   0.25   0.02   0.0250 0.02   0.03   0.0250 0.03
> 0.10   0.35   0.04   0.0350 0.04   0.04   0.0350 0.03
> 0.10   0.45   0.04   0.0450 0.04   0.05   0.0450 0.05
> 0.10   0.65   0.06   0.0650 0.06   0.07   0.0650 0.07
> 0.10   0.85   0.08   0.0850 0.08   0.09   0.0850 0.09

I don't see what this shows beyond that FixedPoint implements "banker's
rounding" (round the infinitely precise result to the nearest retained
digit, but in case of tie round to the nearest even number), while you seem
to prefer "add a half and chop" rounding.  No single rounding discipline is
suitable for all commercial applications, and I hope the FixedPoint project
finds a way to let users specify the rounding *their* app needs.
Unfortunately, I was never able to find a FixedPoint user who was able to
articulate the rounding rules they were trying to emulate <wink/sigh>.

BTW, note that the result of expressions like

    %0.2f" % float_res

is platform-dependent!  The underlying C sprintf function differs across
platforms.  Windows seems to use "add a half and chop" for halfway cases,
while *most* other platforms use IEEE-754 to-nearest/even rounding in
halfway cases (same as "banker's rounding" in this context).  FixedPoint's
results are independent of platform.





More information about the Python-list mailing list