[Tutor] the binary math "wall"
waynejwerner at gmail.com
Tue Apr 20 19:23:48 CEST 2010
On Tue, Apr 20, 2010 at 11:58 AM, Lowell Tackett <lowelltackett at yahoo.com>wrote:
> I'm running headlong into the dilemma of binary math representation, with
> game-ending consequences, e.g.:
> >>> 0.15
> Obviously, any attempts to manipulate this value, under the misguided
> assumption that it is truly "0.15" are ill-advised, with inevitable bad
> the particular problem I'm attempting to corral is thus:
> >>> math.modf(18.15)
> (0.14999999999999858, 18.0)
> with some intermediate scrunching, the above snippet morphs to:
> >>> (math.modf(math.modf(18.15)*100))/.6
> The last line should be zero, and needs to be for me to continue this
> Any of Python's help-aids that I apply to sort things out, such as
> formatting (%), or modules like "decimal" do nothing more than "powder up"
> the display for visual consumption (turning it into a string). The
> underlying float value remains "corrupted", and any attempt to continue with
> the math adapts and re-incorporates the corruption.
That is not precisely correct - modf first converts decimal to a float and
then applies the calculation. Try this instead:
num = decimal.Decimal('1.0')
return (mydecimal%num, mydecimal//num)
On my machine this returns (with
In : modf(d)
Out: (Decimal('0.15'), Decimal('18'))
Which are easily converted.
In : (modf(modf(d)*100))/decimal.Decimal('.6')
So your problem with decimal isn't that it lacks the precision you're
seeking - you're simply converting it to a float /before/ performing the
calculations, which makes turning it into a decimal in the first place
pretty much useless.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Tutor