[Tutor] Need help with rewriting script to use Decimal module
Kent Johnson
kent37 at tds.net
Mon Jan 1 14:37:06 CET 2007
Dick Moores wrote:
> bestFracForMinimumError() is only a small part of a program I wrote
> long ago, called frac.py
> (<http://www.rcblue.com/Python/fracForWeb.py>). I'm trying to rewrite
> it so as to get more precision by using the Decimal module, but am
> getting nowhere. Errors all over the place. Can some kind and
> knowledgeable soul show me how?
I don't see where you are actually using the decimal module here, am I
missing something?
It doesn't help that you cal the argument to your function 'decimal'.
> dec = .345765988765560057657654654
Your Python interpreter probably doesn't have enough precision to
represent this number directly. Here is what I get:
In [1]: dec = .345765988765560057657654654
In [2]: dec
Out[2]: 0.34576598876556008
Kent
More information about the Tutor
mailing list