[Tutor] Controlling the number of decimal places "Representation Error"
kent37 at tds.net
Fri Oct 5 18:49:44 CEST 2007
Carnell, James E wrote:
> Is it a bad practice to avoid 0.1 representation errors
> (0.100000000000001) by just doing the following?
I'm afraid I don't understand either the problem you are trying to solve
or your proposed solution. Can you say more about it?
> #NOTE: all the myVariableName stuff is because Outlook keeps changing
> everything I type.
> #I need 2 decimal places (my input number shouldn't be over 255)
> myNum = 1
> myDiv = 3
> #10000/3 = 3333 and I remember that the last 3 digits are decimals the
> very last can have int() problems
> myResult = (myNum * 1000)/myDiv
> myArray = 
> For I in range( wookie loads ):
> myArray.append(myResult) #this thing can get big, but I don't
> need that much precision
> #but I do need to keep it small since
> it might be running on
> #an embedded system
> James Carnell
> Tutor maillist - Tutor at python.org
More information about the Tutor