Precision Tail-off?
Greg Ewing
greg.ewing at canterbury.ac.nz
Fri Feb 17 20:37:44 EST 2023
On 18/02/23 7:42 am, Richard Damon wrote:
> On 2/17/23 5:27 AM, Stephen Tucker wrote:
>> None of the digits in RootNZZZ's string should be different from the
>> corresponding digits in RootN.
>
> Only if the storage format was DECIMAL.
Note that using decimal wouldn't eliminate this particular problem,
since 1/3 isn't exactly representable in decimal either.
To avoid it you would need to use an algorithm that computes nth
roots directly rather than raising to the power 1/n.
--
Greg
More information about the Python-list
mailing list