[Tutor] Need help with rewriting script to use Decimal module
rdm at rcblue.com
Wed Jan 3 20:51:11 CET 2007
At 10:39 AM 1/2/2007, Kent Johnson wrote:
>Dick Moores wrote:
>>from decimal import Decimal as D
>>def bestFracForMinimumError(decimal, minimumError):
>> denom = 0
>> while True:
>> denom += 1
>> num = round(D(str(decimal)) * D(str(denom)))
>> error = abs(str((str(D(num) / D(str(denom))) -
>This looks backwards ^^^^^^^^^^
Thanks for catching that.
>Don't you need D(str(num)) ? Then converting it back to a str before
>you call abs will not work.
>Your old approach of
> return Decimal(str(num))
>would probably make for more readable code and fewer errors.
Yes, I went back to it.
>I'm not sure this approach will work, though, if you are trying to
>get higher precision, I would think you would have to do all the
>calculations in Decimal, not going to floats for num. I admit I
>haven't thought it through, though. I think you can use
>Decimal.quantize() instead of round().
Thanks very much, Kent. You got me back on track.
More information about the Tutor