prePEP: Decimal data type

[Bengt Richter]

On Mon, 10 Nov 2003 01:56:54 +0100, "Samuel Kilchenmann" <skilchen at bluewin.ch> wrote:

>"Bengt Richter" <bokr at oz.net> schrieb im Newsbeitrag
>news:bojrg5$vim$0 at 216.39.172.122...
>>
>> Following is an exact decimal/<somewhat>rational module hack that
>> implements rounding when the .round() method is called.
>> This is not too elegant, but it might make
>> an interesting toy to cross-check results with.
>
>Thanks a lot for this interesting toy!
You're welcome ;-)

>
>some results seem a little bit strange to me, eg.:
>
>Python 2.3.2 (#49, Oct  2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on
>win32
>Type "help", "copyright", "credits" or "license" for more information.
>>>> from exactdec import ED
>>>> ED('0.95',1)
>ED('9..0')
>>>> ED('0.995',2)
>ED('99..0')
>>>> ED('0.9995',3)
>ED('99.9.0')
>>>> ED('0.99995',4)
>ED('99.99.0')
>>>> ED('0.999995',5)
>ED('99.999.0')
>>>> ED('0.9999995',6)
>ED('1.0')
>>>>
Gak. Yes, I had seen that weirdness before, and thought I fixed it, but not all of it ;-/

>
>its probably due to the line:
>        ret.num //= (ret.den*10**psh)
>in your rounding code and the fact that:
>>>> 1 // 0.1
>9.0
>>>> 10 // 0.01
>999.0
>>>> 100 // 0.001
>99999.0
>>>> 1000 // 0.0001
>9999999.0
>>>> 10000 // 0.00001
>999999999.0
>>>> 100000 // 0.000001
>100000000000.0
>

Thanks, you are are right. That line (and the companion exponent adjustment)
should not be executed unless psh>0. If psh<=0 then there are no less significant
digits to get rid of with the //.  I hope this is a fix, no time for extensive test
(I have to disappear for a few days):

--- exactdec.r1.py      Sat Nov 08 14:12:29 2003
+++ exactdec.r1a.py     Sun Nov 09 19:18:21 2003
@@ -2,6 +2,7 @@
 # 20031107 14:54:21 alpha -- bokr
 # 20031108 09:42:18 added contruction from float with all info or specified rounding -- bokr
 # 20031108 13:58:10 added more docs and interactive calculator thing -- bokr
+# 20031109 19:14:10 fix to round method -- thanks to Samuel Kilchenmann for bug report -- bokr
 #
 # WARNING: Not signigicantly tested. Not optimized. No warranty. Use at your own risk. And,
 #          if you run this program interactively, it passes your raw_input to eval unvalidated!

@@ -219,9 +220,10 @@
         ret.num = abs(ret.num)
         ret = ret + type(self)((5,1,-ndec-1))
         psh = -(ret.p10+ndec)
-        ret.num //= (ret.den*10**psh)
+        if psh>0:
+            ret.num //= (ret.den*10**psh)
+            ret.p10 += psh
         ret.den = 1
-        ret.p10 += psh
         if sign: ret.num = -ret.num
         ret.normalize()
         return ret

Regards,
Bengt Richter