[Tutor] Controlling the number of decimal places "Representation Error"

Carnell, James E jecarnell at saintfrancis.com
Fri Oct 5 17:43:08 CEST 2007


Question:
Is it a bad practice to avoid 0.1 representation errors
(0.100000000000001) by just doing the following?

#NOTE: all the myVariableName stuff is because Outlook keeps changing
everything I type.

#I need 2 decimal places (my input number shouldn't be over 255)

myNum = 1
myDiv = 3

#10000/3 = 3333 and I remember that the last 3 digits are decimals the
very last can have int() problems

myResult = (myNum * 1000)/myDiv 

myArray = []

For I in range( wookie loads ):
     myArray.append(myResult)     #this thing can get big, but I don't
need that much precision
                                  #but I do need to keep it small since
it might be running on
                                  #an embedded system


Sincerely,

James Carnell



More information about the Tutor mailing list