String representations of numbers and approximate equality

Dave Angel davea at davea.name
Thu Sep 25 14:40:28 CEST 2014


Chris Angelico <rosuav at gmail.com> Wrote in message:
> On Thu, Sep 25, 2014 at 5:56 AM, Dave Angel <davea at davea.name> wrote:
>> Your definition is not nearly as concrete as you think. Is the
>>  first number considered to be exact, and we'll only check the
>>  second? Will the factor always be an int, and thus
>>  exact?
> 
> Apologies: the definition is concrete, just under-specified in the post.
> 
> No, both numbers are inexact; most notably, one third of the factor
> might be involved - both of these should be correct:
> 
> ["1.333", "40"]
> ["1.33", "40"]
> 
> But if it's easier to assume that one or t'other must be exact, that's
> fine too - I can just run the check twice, once each way.
> 

You could assume that the one with more significant digits is the
 exact one.

Another approach is to see if twiddling the last digit gets you
 from too high to too low. I used this approach to check a trig
 package I wrote in '75. An answer was close enough if twiddling
 the last digit changed the result value from too high to too low,
 or if twiddling the last digit of the argument did. I needed to
 go both ways because the curve sometimes had a slope greater than
 one.

Oh, and don't forget to set the precision of Decimal to a couple
 more than the sum of the digits. 


-- 
DaveA




More information about the Python-list mailing list