[Tutor] Unexpected result from decimal

Tim Peters tim.peters at gmail.com
Thu Jan 20 04:07:35 CET 2005


[Jacob S.]
>    I'm having a problem that is ticking me off. (to put it lightly)
> Why does decimal do this --  I thought that getcontext().prec
> was number of decimal places?

It's unclear what you mean by "decimal places".  From context, you
_appear_ to mean "number of decimal digits after the radix point".  In
that case, no, that's not what precision means.  decimal is a
floating-point type, not a fixed-point type, and precision is the
total number of significant digits; the location of the radix point is
irrelevant.  The rules are spelled out in great detail here:

    http://www2.hursley.ibm.com/decimal/

> >>> import decimal
> >>> decimal.getcontext().prec = 2
> >>> a = decimal.Decimal(2)
> >>> b = decimal.Decimal(3)
> >>> 100*a/b
> Decimal("67")
> >>> print 100*a/b
> 67
> >>>
>
> Why does it do this?

It's doing what you told it to do.  It would have helped if you had
been specific about what you wanted it to do.  For example, did you
want 66.67, or what?

> It's really, really, messing things up for me because results are
> not interpreted in my programs correctly.

Change your code <wink>.  Perhaps this is what you wanted?

>>> import decimal
>>> pennies = decimal.Decimal("0.01")
>>> a = decimal.Decimal(2)
>>> b = decimal.Decimal(3)
>>> print 100*a/b
66.66666666666666666666666667
>>> print (100*a/b).quantize(pennies)
66.67
>>>


More information about the Tutor mailing list