[Python-Dev] Adding decimal (aka FixedPoint) numbers to Python

[Michael McLay]

sf/653938I is a patch to integrate the fixedpoint module, that was created by 
Tim Peters, into Python as a first class object. That is, the patch adds this 
new number type so that it has direct syntax support like float, int, long, 
str., etc. within the interpreter. 

I use Tim's module to implement the type. This limits the patch to a small bit 
of code that adds the a syntax interface to this module. Syntax recognition 
for the new number format required a change to Parser/tokenizer.c and to 
Python/compile.c.  This patch allows the new decimal type (I renamed the 
FixedPoint type to decimal in the fixedpoint.py file because the name is 
shorter and it is sufficient to distinguish the type from a binary float.) to 
be entered as a literal that is similar to a long, int, or float. The new 
syntax works as follows:

>>> 12.00d
12.00d
>>> .012d
0.012d
>>> 1d
1.d
>>> str(1.003d)
'1.003'

As you can see from the example, the default precision for decimal literals 
are determined by the precision entered expressed in the literal.

The patch also adds glue to Python/bltinmodule.c to create a builtin decimal() 
function. This builtin decimal (Formally FixedPoint) function invokes the 
constructor of the Python implementation of the decimal class defined in the 
fixedpoint module.  The implementation follows the familiar pattern for 
adding special builtin functions, much like the implementation of  apply and 
abs. It does not follow the typical method for adding built in types. That 
would require a more invasive patch. The builtin decimal function works as 
follows:

>>> decimal(1.333)
1.33d
>>> decimal(1.333,4)
1.3330d
>>> decimal(400,2)
400.00d

The semantics of the precision parameter may be incorrect because it is only 
addressing the precision of the fraction part of the number. Shouldn't it 
reflect the number of significant digits? If that were the case then the 
following would have been the result:

>>> decimal(400,2)
4.0e2d

This problem is more noticable when exponents are large relative to the number 
of significant digits. For instance:

>>> 40.e3d
40000.d

The result should have been 40.e3d, not 40000. This implies more precision 
than is declared by the constant.

As it currently is implemented the type pads with zeros on the integer side of 
the decimal point, whic implies more accuracy in the number than is true. I 
ran into this problem when I tried to implement the automatic conversion of 
string number values to decimal representation for numbers in 'e' format.

>>> 3.03e5d
303000.00d

The representation is incorrect. It should have returned 3.03e5d but it padded 
the result with bogus zeros.

For this first cut at a new decimal type I am primarily interested in 
investigating the semantics of the new native decimal number type in Python. 
This type will be useful to bankers, accountants, and newbies. The backend 
implementation could be replaced with a C implementation without affecting 
the Python language semantics. 

The approach used in this patch makes it very easy to experiment with changes 
to the semantics because all of the experimentation can be done by changing 
the code in Lib/fixedpoint.py. The compiled wrapper doesn't have to be 
modified and recompiled.

Unit testing so far just reuses the _test function in the fixedpoint.py 
module. Since I am not sure which way to go with the semantic interpretation 
of precision I decided to post the patch before making a significant change 
to fixedpoint.py. Some feedback on the interpretation of precision would be 
appreciated.

Documentation: 

The following needs to be added to section 2.1 of the library reference 
manual:

decimal(value, precision)
    Convert a string or a number to fixed point decimal number. If the 
argument is a string, it must contain a possibly signed decimal or floating 
point number, possibly embedded in whitespace. The precision parameter will 
be ignored on string values. The precision will be set based on the number of 
significant digits. Otherwise, the argument may be a plain or long integer or 
a floating point number, and a decimal number with the same value ( the 
precision attribute will set the precision of the fraction roundoff) is 
returned.


Section 3.1.1 of the tutorial will add a brief description of decimal number 
usage after the description of floating point: 

There is full support for floating point; operators with mixed type operands 
convert the integer operand to floating point:

>>> 3 * 3.75 / 1.5
7.5
>>> 7.0 / 2
3.5

Python also supports fixed point decimal numbers. These numbers do not suffer 
from the somewhat random roundoff errors that can occur with binary floating 
point numbers. A decimal number is created by adding a 'd' or 'D' suffix to a 
number:

>>> 3.3d + 3d
6.3d
>>> 3.3d + 3.03
6.3d
>>> 3.3d + decimal(3.03,3)
6.330d
>>> decimal(1.1, 16)
1.1000000000000001d
>>>
The builtin decimal constructor provides a means for converting float and int 
types to decimal types. Since floats are approximations of decimal floating 
point numbers there are often roundoff errors introduced by using floats.  
(The 1 in the last place of the conversion of 1.1 with 16 digits is a binary 
round off error.)  For this reason the decimal function requires the 
specification of precision when converting a float to a decimal. This allows 
the significant digits of the number to be specified. (Accountants and 
bankers love this because it allows them to balance the books without pennies 
being lost due to the use of binary numbers.)

>>> 3.3d/2
1.6d

 Note that in the example above the  expression 3.3d/2 returned 1.6d. The 
rounding scheme for Python decimals uses "banking" rounding rules. With 
floating point numbers the result would have been as follows:

>>> 3.3/2
1.6499999999999999

So as you can see the banking rules round off the .04999999999 portion of the 
number and calls it a an even 1.6.