[Python-Dev] decimal API

Tim Peters tim.peters at gmail.com
Sat Jul 3 01:45:55 CEST 2004

[Raymond Hettinger]
> Currently, calling the Decimal constructor with an invalid literal (such
> as Decimal("Fred")) returns a quiet NaN.  This was done because the spec
> appeared to require it (in fact, there are IBM test cases to confirm
> that behavior).
> I've discussed this with Mike Cowlishaw (author of the spec and test
> cases) and he has just clarified that, "... the intent here was not to
> disallow an exception.   The analogy, perhaps, is to a divide-by-zero:
> the latter raises Invalid Operation and returns a qNaN.

Don't be confused by his odd wording here.  There's a concept of
"raising" in Python, but not in his spec!  Exceptional conditions in
the spec are "signaled", not raised.  Whether they go on to raise (in
the Python sense) a user-visible exception is to be controlled by the
specific signaled exceptional condition's trap-enable flag.

> The string conversion is similar.   (Again, in some implementations/languages,
> the result after such an exception is not available.)   I'll see if I can
> clarify that, at least making it clear that Invalid Operation is OK at
> that point."

Sorry, but I'm really confused now.  The current version of the spec
defines an exceptional condition specifically for this purpose:

Conversion syntax 

This occurs and signals invalid-operation if an string is being
converted to a number and it does not conform to the numeric string
syntax. The result is [0,qNaN].

In other words, the current spec *requires* that trying to convert a
nonsense string signal invalid-operation, not merely allows it. 
AFAICT, "our" code already does that, too(!).

Note that I've suggested that some trap-enable flags should be set by
default for Python:  those for invalid operation, divide by 0, and
overflow.  So, in my mind (unsullied by reality <wink>):  (a)
conversion of a nonsense string already signals invalid-operation;
and, (b) because the invalid-operation trap-enable flag is set by
default in Python, it also already raises an exception.

> So, my question for the group is whether to:
> * leave it as-is
> * raise a ValueError just like float('abc') or int('abc')
> * raise an Invalid Operation and return a quiet NaN.

That last choice is impossible:  you can *signal* invalid operation
and return a quiet NaN, but if you *raise* something then it's
impossible to return anything.  We could make InvalidOperation a
subclass of ValueError, BTW.  That would make sense (InvalidOperation
always has to do with an insane input value).

> Either of the last two involves editing the third-party test cases which
> I am loathe to do.

Or, until Mike fixes the test vectors, special-case the snot out of
these specific tests in the test driver.  That's assuming the test
vectors don't actually match the spec now.  But I suspect that they
do, and that there's a different confusion at work here.

> The second is the most Pythonic but does not match Mike's clarification.
>  The third keeps within context of the spec but doesn't bode well for Decimal
> interacting well with the rest of python.

That's one reason I want Python to enable the invalid-operation trap
by default.  That choice isn't suitable for all apps all the time, but
I have no hesitation Pronouncing that it will be suitable for most
Python apps most of the time.

> The latter issue is unavoidable to some degree because no other python
> numeric type has context sensitive operations, settable traps, and
> result flags.

That's because Python has blissfully ignored IEEE-754 for its current
floating-point operations.  754 requires all of those for binary fp,
and a small part of my hope for Decimal is that it nudges Python into
coming to grips with that 20-year-old standard.  Continuing to ignore
it creates problems for scientific programmers too, both those who
hate 754 and those who love it.  Python's binary fp support will end
up looking more like Decimal over time; the fact that Python's binary
fp is 20 years out of date isn't a good reason to cripple Decimal too.

> A separate question is determining the default precision.  Currently, it
> is set at 9 which conveniently matches the test cases, the docstring
> examples, and examples in the spec.  It is also friendly to running
> time.

I expect that's a red herring.  Financial apps are usually dominated
by the speed of I/O conversions and of addition, and those all take
time proportional to the number of significant decimal digits in the
operands, not proportional to precision.  Division certainly takes
time proportional to precision, but that's a rarer operation.  Even
after division or multiplication, financial apps will usually round
the result back, again reducing the number of significant decimal
digits (e.g., your tax forms generally make you round on every line,
not accumulate everything to 100 digits and round once at the end).

> Tim had suggested that 20 or so would handle many user requirements
> without needing a context change.

It would be pathetic to ship with a default precision smaller than
mid-range hand calculators.

> Mike had suggested default single and double precision matching those
> proposed in 754R.  The rationale behind those sizes has nothing to do
> with use cases; rather, they were chosen so that certain representations
> (not the ones we use) fit neatly into byte/word sized multiples (once
> again showing the hardware orientation of the spec).

I don't care about that either.

> No matter what the default, the precision is easy to change:
>    >>> getcontext().prec = 42
>    >>> Decimal(1) / Decimal(7)
>    Decimal("0.142857142857142857142857142857142857142857")

It should be at least 12 (copying HP hand calculators).   I believe
current COBOL requires 36 decimal digits in its decimal type.  VB's
Currency type has 19 decimal digits, and its Decimal type has 28
decimal digits.  Those are all defaults decided by people who care
about applications more than word boundaries <wink>.

More information about the Python-Dev mailing list