[Tutor] Controlling the number of decimal places "Representation Error"
Carnell, James E
jecarnell at saintfrancis.com
Fri Oct 5 17:43:08 CEST 2007
Is it a bad practice to avoid 0.1 representation errors
(0.100000000000001) by just doing the following?
#NOTE: all the myVariableName stuff is because Outlook keeps changing
everything I type.
#I need 2 decimal places (my input number shouldn't be over 255)
myNum = 1
myDiv = 3
#10000/3 = 3333 and I remember that the last 3 digits are decimals the
very last can have int() problems
myResult = (myNum * 1000)/myDiv
myArray = 
For I in range( wookie loads ):
myArray.append(myResult) #this thing can get big, but I don't
need that much precision
#but I do need to keep it small since
it might be running on
#an embedded system
More information about the Tutor