[Python-Dev] Re: WYSIWYG decimal fractions)

[Tim Peters]

[M.-A. Lemburg, on FixedPoint.py]
> ...
> Very impressive ! The code really show just how difficult it is
> to get this done right (w/r to some definition of that term ;).

Yes and no.  Here's the "no" part:  I can do code like this in my sleep, due
to decades of experience.  So code like that isn't difficult at all for the
right person (yes, it *is* difficult if you don't already have the background
for it!  it's learnable, though <wink>).

Here's the "yes" part:  I have no experience with database or commercial
non-scientific applications, while people who do seem to have no clue about
how to *specify* what they need.  When I was writing FixedPoint.py, I asked
and asked what kind of rounding rules people needed, and what kind of
precision-propagation rules.  I got a grand total of 0 *useful* replies.  In
that sense it seems a lot like getting Python threads to work under HP-UX:
lots of people can complain, but no two HP-UX users agree on what's needed to
fix it.

In the end (for me), it *appeared* that there simply weren't any explicable
rules:  that among users of 10 different commerical apps, there were 20
different undocumented and proprietary legacy schemes for doing decimal fixed
and floats.  I'm certain I could implement any of them via trivial variations
of the FixedPoint.py code, but I couldn't get a handle on what exactly they
were.

> BTW, is the implementation ANSI/IEEE standards conform ?

Sure, the source code strictly conforms to the ANSI character set <wink>.

Which standards specifically do you have in mind?  The decimal portions of
the COBOL and REXX standards are concerned with how decimal arithmetic
interacts with language-specific features, while the 854 standard is
concerned with decimal *floating* point (which the astute reader may have
guessed FixedPoint.py does not address).  So it doesn't conform to any of
those.  Rounding, when needed, is done in conformance with the *default*
"when rounding is needed, round via nearest-or-even as if the intermediate
result were known to infinite precision" 854 rules.  But I doubt that many
commercial implementations of decimal arithmetic use that rule.

My much fancier Rational package (which I never got around to making
available) supports 9 rounding modes directly, and can be user-extended to
any number of others.  I doubt any of the builtin ones are in much use either
(for example, the builtin "round away from 0" and "round to nearest, or
towards minus infinity in case of tie" aren't even useful to me <wink>).

Today I like Cowlishaw's "Standard Decimal Arithmetic Specification" at

    http://www2.hursley.ibm.com/decimal/decspec.html

but have no idea how close that is to commerical practice (OTOH, it's
compatible w/ REXX, and lots of database-heads love REXX).

> ...
> Note that I will have to interface to database using the string
> representation, so I might get away with adding scale and precision
> parameters to a (new) asString() method.

As some of the module comments hint, FixedPoint.py started life with more
string gimmicks.  I ripped them out, though, for the same reason we *should*
drop thread support on HP-UX <0.6 wink>:  no two emails I got agreed on what
was needed, and the requests were mutually incompatible.  So I left a clean
base class for people to subclass as desired.

On 23 Dec 1999, Jim Fulton again raised "Fixed-decimal types" on Python-Dev.
I was on vacation & out of touch at the time.  Guido has surely forgotten
that he replied

    I like the idea of using the dd.ddL notation for this.

and will deny it if he reads this <wink>.

There's a long discussion after that -- look it up!  I see that I got around
to replying on 30 Dec 1999-- a repetition of this thread, really! --and
posted (Python) kernels for more flexible precision-control and rounding
policies than FixedPoint.py provided.

As is customary in the Python world, the first post that presented actual
code killed the discussion <wink/sigh> -- 'twas never mentioned again.

>> FixedPoint.py is better suited to computation than I/O, though,
>> since it uses Python longs internally, and conversion between
>> BCD-like formats and Python longs is expensive.

> See above: if string representations can be computed fast,

They cannot.  That was the point.  String representations *are* "BCD-like" to
me, in that they separate out each decimal digit.  To suck the individual
decimal digits out of a Python long requires a division by 10 for each digit.
Since people in COBOL routinely work with 32-digit decimal numbers, that's 32
*multi-precision* divisions by 10.  S-l-o-w.  You can play tricks like
dividing by 1000 instead, then use table lookup to get three digits at a
crack, but the overall process remains quadratic-time in the number of
digits.

Converting from a string of decimal digits to a Python long is also quadratic
time, so using longs as an internal representation is expensive in both
directions.

It is by far the cheapest way to do *computations*, though.  So I meant what
I said in all respects.

> ...
> Hmm, ideal would be an Open Source C lib which could be used as
> backend for the implementation... haven't found such a beast yet
> and the IBM BigDecimal Java class doesn't really look attractive as
> basis for a C++ reimplementation.

It's easy to find GPL'ed code for decimal arithmetic (for example, pick up
the Regina REXX implementation linked to from the Cowlishaw page).  For that
matter, you could just clone Python's longint code and fiddle the base to a
power of 10 (mutatis mutandis), and stick an exponent ("scale factor") on it.
This is harder than it sounds, but quite doable.

then-again-if-god-had-wanted-us-to-use-base-10-he-wouldn't-have-
    given-us-2-fingers-ly y'rs  - tim