[Python-Dev] RE: Adding decimal (aka FixedPoint) numbers to Python

[Michael McLay]

On Tuesday 17 December 2002 12:34 am, Tim Peters wrote:
> [Michael McLay]
>
> > Fixing the decimal place would make sense for applications having
> > to do with money. That is quite different than how numbers are
> > treated in scientific calculations.
>
> Of course!  FixedPoint was written for people with fixed-point
> applications, and especially for those of such who think a computer should
> be as good at decimal math as their 12-year old daughter driving a $1 hand
> calculator <0.500021 wink>.
>
> > That is were I borrowed the idea of significant digits.  Accountants
> > tend to think that all digits are significant, down to the last
> > penny.
>
> "The last penny" is an inherently fixed-point concept:  the last penny is
> measured in units of 0.01 regardless of how many digits appear before the
> decimal point.  That's what FixedPoint models directly.  The total number
> of digits is unbounded, much like the national debt, but the position of
> the decimal point is fixed.

It's always good to plan ahead for the government's budget. Thanks. 

> > My goal is to have a new numerical type added to Python that
> > satisfies the requirements for accountants and newbies.  Your
> > description of fixed point seems to meet this goal.
>
> That was its intent, but it is an expensive flavor of arithmetic, and not
> to everyone's tastes.

Yes, but since it does what most people expect, it will make Python more 
appealing to the average Joe. Now that I have a better understanding of 
Cowlishaw's decimal numbers I can see how they would be a better solution, 
but for the short term having numbers that behave like most calculators would 
be a reasonable start. I am concerned about one thing. Will the fixed point 
implementation you have implemented be backwards compatible with an 
implementation of Cowlishaw's decimal numbers?

> > My confusion was raised by the 'e' notation seeming to add precision
> > to numbers (e.g. 1.0e5 expands to 100000.0.)
>
> Well, that didn't come from the FixedPoint module -- it always gives a
> number of digits after the decimal point equal to the number the user tells
> it to use; it doesn't try to infer anything from the form of a literal.

Most of the time this is probably the right thing to do. Bit I think it might 
be a good idea to infer the precision from string literals if no precision is 
explicitly defined. As it is currently implemented the behavior is:

> fixed("12.3")
12.30

The addition of '0' is unexpected. I can see the need to make specify 
precision when the value being converted is a floating point number, but if 
it is a string then the number is a literal value specified by the user. They 
should get exactly what they write. That would be:

> fixed("12.3")
fixed(12.3,1)
> fixed("12.305")
fixed(12.305,3)

Maybe this is too subtle of a rule, but without this rule it will be difficult 
to implement a literal that doesn't require the explicit definition of a 
level of precision. So, looking forward the rule for adding precision should 
be a default to 2 decimal places for floats and integers, but it should be a 
literal interpretation of the value passed to the constructor as a string. 
For conversions for floats the results would be:

> fixed(12.3)
fixed(12.30,2)
> fixed(12.305)
fixed(12.30,2)
> fixed(12.3,3)
fixed(12.300,3)
> fixed(12.305,1)
fixed(12.3,1)
> fixed(12)
fixed(12.00,2)

(Perhaps it should be a default to a precision of 0 if the value passed is an 
integer.)

Ideally the proposal for adding literals would be added to 2.3, but Guido 
vetoed that, and the literals would derive the precision from the number of 
trailing zeros that were written:

> 12.3d
12.3
> 12.300d
12.300

If the rule for defining the default precision doesn't make the exception for 
a literal string then it will be difficult to add a syntactic literal that 
doesn't require the explicit definition of the level of precision. This would 
end up with an awkward syntax, such as:

> 12.3:1d
12.3
> 12.30:3d
12.300

> > This would be incorrect for reporting significant digits in scientific
> > papers, but the rules for accounting math are different. This was
> > simply my confusion.  FixedPoint appears to be sufficient for bankers
> > and accountants.
>
> I believe so, but IANAB and IANAA.

And neither am I. But considering the inconsistencies in their current 
expectations I think you have defined a class that will have few objections.
> > ...
> > This is a very good long term goal.  So for Python3k the standard
> > numerical type will be Cowlishaw's decimal numbers.
>
> Don't know.  It brings its own kinds of problems, not least of which is
> sloth.

Has someone accused you of sloth? Hardly. I'm sure I'll irritate Guido by 
suggesting the addition of a second parser to Python, but through this 
mechanism I think it will be possible to make a natural and relatively 
painless transition to using Cowlishaw's numbers as the default number type 
in Python 3k 

> > To bad the decNumber library from IBM is not open source.
>
> A Java implementation is open:
>
>     http://www2.hursley.ibm.com/decimalj/

That's great for jython, but it will require more work for the C Python 
implementation.

> > In the short term what do you advise be done with the patch to
> > add a fixed point literal and a builtin fixed point conversion
> > function?
>
> In the short term it's most important to get FixedPoint into 2.3 (under
> whatever name people like).  One thing at a time is more than I can handle
> these days.

Typing "FixedPoint" instead of "fixed" will be quite irritating, but I'll be 
happy with either name if the class is included in the standard distribution.

> > I could change the name from decimal() to FixedPoint(), but the
> > capital letters in the name scream "ugly".
>
> I don't care about the name.

I'd rather have a literal added to the language and a builtin "fixed" as the 
name, but Guido has vetoed the literal. Again. I'll be happy with anything 
that adds the capabilities to the standard distribution.

> > This isn't consistent with the short names used by
> > the other numerical conversion functions (int, float, hex, oct...).
> > Would fixed() be an acceptable name?
>
> Make everyone unhappy and call it decimal_fixed_point() <0.9 wink>.  I'm
> not sure why you might prefer fixed() to decimal(), BTW.  One reason I
> would is that decimal() reminds of oct() and hex(), but would do something
> very different.

Good idea, calling it decimal_fixed_point should maximize the irritation of 
end users. This will lead to more pressure to add a literal to the language.

I thought it might be best to use "fixed" for fixed point in case there was 
some incompatibility with the eventual inclusion of Cowlishaw's decimal 
numbers. The intention was to reserve "decimal" for the more generalized form 
of decimal numbers. If your FixedPoint implementation is 100% compatible then 
using "decimal" would be best. Being able to extend FixedPoint semantics to 
include floating point decimals in the future would be the best option. 

> > This also suggests the suffix should be changed to 'f' and
> > 'F' instead of 'd' and 'D'.
>
> -1.  'f' is a legit suffix for a float literal in C, and so much of
> Python's literal syntax is identical to C's that reusing part of it to mean
> something else seems a Bad Idea.

I didn't like the change for the same reason, but I made the change so that 
'd' and 'D' could be reseved for Cowlishaw's decimal numbers. If they are 
compatible then I withdraw the change from my original proposal.

> >>     http://www2.hursley.ibm.com/decimal/decifaq.html
> >
> > According to this reference the most frequent numerical column
> > type used in commercial databases is decimal (55%). The float type
> > is used just 1.4% of the time with the rest being split between
> > Integer and SmallInt.
> > ...
> > Is it safe to conclude that Python will be be much more appealing
> > to a huge segment of the IT market if grows a native decimal number
> > type?
>
> I've never worked in the IT market and don't know what drives them.  From
> email received in response to FixedPoint, all I've gleaned is that (1) they
> want arithmetic that exactly mimics whatever idiosyncratic schemes are used
> by a huge variety of software systems I never heard of; and, (2) they're
> unable to explain what those schemes are well enough to mimic them.  Most
> people in this business have no idea what "the rules" are, they just know
> that their butt is on the line if they deviate from what The Company has
> been seeing for the last 20 years.  So, e.g., you get requests for flatly
> insane rounding rules, like "if the fraction is over 0.4, round away from
> 0, else round toward it".  Most of the time it turns out that isn't really
> the rule they need, either, it's just the first one they generalized from
> one example, after failing to find docs.
>
> FixedPoint is extremely careful to give the best possible numeric result in
> all cases.  I do know that isn't really what the IT market cares about --
> compatability with legacy systems-- themselves incompatible with each
> other --is much more important to them.  OTOH, giving the best possible
> result is, as the IEEE-754 fp standard demonstrated with force in practice,
> explainable, reproducible, and ultimately compelling.  That it's also
> expensive is just icing on the cake <wink>.

I think you have done a fine job of providing what they need, regardless of 
what the end users think they want. It will be easy enough to provide 
conversion capabilities to the broken semantics of the sundry implementation 
of decimal numbers that exist.

> [Michael McLay]
>
> >> I could change the name from decimal() to FixedPoint(), but the
> >> capital letters in the name scream "ugly".
>
> [Greg Ewing]
>
> > Worse, it would completely lose the connotation that it is a *decimal*
> > number type, which surely is the main point of it in the first place!
>
> Only to a compsci type.  The intended audience never heard of binary
> fixed-point -- fixed-point means decimal to them as surely as New Zealand
> means anarchy to an American <wink>.

Let's go easy on the kiwis. They don't always get their due. It looks like 
"Mad" Pearse [1] may have flown before the Wright bothers, but since he was a 
perfectionist he did not try to claim the title. <wink>
 
[1] http://slashdot.org/article.pl?sid=02/12/16/0338254&mode=thread&tid=126