help needed on decimal formatting issue
Robin Becker
robin at reportlab.com
Mon Sep 19 08:42:30 EDT 2011
I'm not really very used to the decimal module so I'm asking here if any one can
help me with a problem in a well known third party web framework
The code in question is
def format_number(value, max_digits, decimal_places):
"""
Formats a number into a string with the requisite number of digits and
decimal places.
"""
if isinstance(value, decimal.Decimal):
context = decimal.getcontext().copy()
context.prec = max_digits
return u'%s' % str(
value.quantize(decimal.Decimal(".1") ** decimal_places,
context=context))
else:
return u"%.*f" % (decimal_places, value)
we have set up decimal fields with max_digits=7 decimal_places=2 and
max_digits=10, decimal_places=4. We are getting issues with quantize failing
with the message 'quantize result has too many digits for current context'.
Anyhow, I believe that we need to adjust the above code so that the precision is
actually set to
context.prec = max_digits+decimal_places
but I'm not certain that is right. Presumably the author thought that the fields
could have large values of both max_digits and decimal_places. Initially I
thought precision must be to do with decimal_places only, but changing
context.prec = decimal_places
did not fix the issue. Can someone enlighten me? The failing case for the
original code was
format_number(Decimal('914146.80'),7,2)
with context.prec = decimal_places we failed differently with
format_number(Decimal('42.7571'),10,4)
so I adopted context.prec = max_digits+decimal_places and that seems to work. If
this is indeed a proper fix I will send a bug report to the well known jazz
based framework.
--
Robin Becker
More information about the Python-list
mailing list