Re: [Python-Dev] Adventures with Decimal
[Tim and Raymond are slugging it out about whether Decimal constructors should respect context precision] Tim, I find Raymond's arguments to be much more persuasive. (And that's even BEFORE I read his 11-point missive.) I understood the concept that *operations* are contex-dependent, but decimal *objects* are not, and thus it made sense to me that *constructors* were not context-dependent. On the other hand, I am NOT a floating-point expert. Can you educate me some? What is an example of a case where users would get "wrong" results because constructors failed to respect context precision? (By the way... even if other constructors begin to respect context precision, the constructor from tuple should NOT -- it exists to provide low-level access to the implementation. I'll express no opinion on the constructor from Decimal, because I don't understand the issues.) -- Michael Chermside
[Michael Chermside]
Tim, I find Raymond's arguments to be much more persuasive. (And that's even BEFORE I read his 11-point missive.) I understood the concept that *operations* are context- dependent, but decimal *objects* are not, and thus it made sense to me that *constructors* were not context-dependent.
On the other hand, I am NOT a floating-point expert. Can you educate me some?
Sorry, I can't make more time for this now. The short course is that a module purporting to implement an external standard should not deviate from that standard without very good reasons, and should make an effort to "hide" whatever deviations it thinks it needs to indulge (e.g., make them harder to spell). This standard provides 100% portable (across HW, across OSes, across programming languages) decimal arithmetic, but of course that's only across standard-conforming implementations. That the decimal constructor here deviates from the standard appears to be just an historical accident (despite Raymond's current indefatigable rationalizations <wink>). Other important implementations of the standard didn't make this mistake; for example, Java's BigDecimal|(java.lang.String) constructor follows the rules here: http://www2.hursley.ibm.com/decimalj/deccons.html Does it really need to be argued interminably that deviating from a standard is a Big Deal? Users pay for that eventually, not implementors. Even if a standard "is wrong" (and leaving aside that I believe this standard asks for the right behavior here), users benefit from cross-implementation predictability a lot more than they can benefit from a specific implementation's non-standard idiosyncracies.
[Tim Peters]
... Other important implementations of the standard didn't make this mistake; for example, Java's BigDecimal (java.lang.String) constructor follows the rules here:
Hmm -- or maybe it doesn't! The text says: The BigDecimal constructed from the String is in a standard form, as though the add method had been used to add zero to the number with unlimited precision.[1] and I read "add zero" as "applies context". But then it says "unlmited precision". I'm not at all sure what it means now.
Maybe they just meant it as an explanation of "standard form", clarifying that -0 is turned into +0? (That's what adding 0 does, right?) On 5/20/05, Tim Peters <tim.peters@gmail.com> wrote:
[Tim Peters]
... Other important implementations of the standard didn't make this mistake; for example, Java's BigDecimal (java.lang.String) constructor follows the rules here:
Hmm -- or maybe it doesn't! The text says:
The BigDecimal constructed from the String is in a standard form, as though the add method had been used to add zero to the number with unlimited precision.[1]
and I read "add zero" as "applies context". But then it says "unlmited precision". I'm not at all sure what it means now. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Does it really need to be argued interminably that deviating from a standard is a Big Deal?
The word deviate inaccurately suggests that we do not have a compliant method which, of course, we do. There are two methods, one context aware and the other context free. The proposal is to change the behavior of the context free version, treat it as a bug, and alter it in the middle of a major release. The sole argument resembles bible thumping. Now for a tale. Once upon a time, one typed the literal 1.1 but ended up with the nearest representable value, 1.1000000000000001. The representation error monster terrorized the land and there was much sadness.
From the mists of Argentina, a Palidan set things right. The literal 1.1 became representable and throughout the land the monster was believed to have been slain. With their guard down, no one thought twice when a Zope sorcerer had the bright idea that long literals like 1.1000000000000001 should no longer be representable and should implicitly jump to the nearest representable value, 1.1. Thus the monster arose like a Phoenix. Because it was done in a bugfix release, without a PEP, and no public comment, the citizens were caught unprepared and faced an eternity dealing with the monster so valiantly assailed by the Argentine.
Bible thumping notwithstanding, this change is both unnecessary and undesirable. Implicit rounding in the face of explicit user input to the contrary is a bad idea. Internally, the implementation relies on the existing behavior so it is not easily changed. Don't do it. Raymond
[Raymond Hettinger]
The word deviate inaccurately suggests that we do not have a compliant method which, of course, we do. There are two methods, one context aware and the other context free. The proposal is to change the behavior of the context free version, treat it as a bug, and alter it in the middle of a major release.
I didn't suggest changing this for 2.4.2. Although, now that you mention it ... <wink>.
The sole argument resembles bible thumping.
I'm sorry, but if you mentally reduced everything I've written about this to "the sole argument", rational discussion has become impossible here. In the meantime, I've asked Mike Cowlishaw what his intent was, and what the standard may eventually say. I didn't express a preference to him. He said he'll think about it and try to get back to me by Sunday.
[Tim]
I'm sorry, but if you mentally reduced everything I've written about this to "the sole argument", rational discussion has become impossible here.
Forgive me one melodramatic email. I've laid out my reasoning and understand yours. Crossing light sabers with one such as yourself is of course a foolhardy undertaking. A root difference is that I believe we have both a compliant implementation (using Context.create_decimal) and a practical context free extension in the form of the regular Decimal constructor. A second difference is that you see harm in allowing any context free construction while I see greater harm from re-introducing representation error when that is what we were trying to fix in the first place. The rest is just practicalities and engineering (altering decimal's internals may be a non-trivial undertaking). May the force be with you, Raymond
On 5/21/05, Raymond Hettinger <raymond.hettinger@verizon.net> wrote:
A root difference is that I believe we have both a compliant implementation (using Context.create_decimal) and a practical context free extension in the form of the regular Decimal constructor.
Please forgive an intrusion by someone who has very little knowledge of floating point pitfalls. My mental model of Decimal is "pocket calculator arithmetic" (I believe this was originally prompted by Tim, as I had previously been unaware that calculators used decimal hardware). In that model, fixed precision is the norm - it's the physical number of digits the box displays. And setting the context is an extremely rare operation - it models swapping to a different device (something I do do in real life, when I have an 8-digit box and am working with numbers bigger than that - but with Decimal, the model is a 28-digit box by default, and that's big enough for me!) Construction models typing a number in, and this is where the model breaks down. On a calculator, you physically cannot enter a number with more digits than the precision, so converting a string with excess precision doesn't come into it. And yet, Decimal('...') is the "obvious" constructor, and should do what people "expect". In many ways, I could happily argue for an exception if the string has too many digits. I could also argue for truncation (as that's what many calculators actually do - ignore any excess typing). No calculator rounds excess input, but I can accept it as what they might well do if was physically possible. And of course, in a practical sense, I'll be working with 28-digit precision, so I'll never hit the situation in any case, and I don't care :-)
A second difference is that you see harm in allowing any context free construction while I see greater harm from re-introducing representation error when that is what we were trying to fix in the first place.
The types of rounding errors (to use the naive term deliberately) decimal suffer from are far more familiar to people because they use calculators. With a calculator, I'm *used* to (1/3) * 3 not coming out as exactly 1. And indeed we have
(Decimal(1)/Decimal(3))*Decimal(3) Decimal("0.9999999999999999999999999999")
Now try that with strings:
(Decimal("1")/Decimal("3"))*Decimal("3") Decimal("0.9999999999999999999999999999") (Decimal("1.0")/Decimal("3.0"))*Decimal("3.0") Decimal("0.9999999999999999999999999999")
# Remember, my argument is that I'd never do the following in
Nope, I don't see anything surprising. After a bit more experimentation, I'm unable to make *anything* surprise me, using either Decimal() or getcontext().create_decimal(). Of course, I've never bothered typing enough digits that I care about (trailing zeroes don't count!) to trigger the rounding behaviour of the constructor that matters here, but I don't ever epect to in real life. Apologies for the rambling discussion - it helped me as a non-expert to understand what the issue is here. Having done so, I find that I am unable to care. (Which is good, because I'm not the target audience for the distinction :-)) So, to summarise, I can't see that a change would affect me at all. I mildly favour Tim's position - because Raymond's seems to be based on practicality for end users (where Tim's is based on convenience for experts), and I can't see any practical effect on me to Tim's change. OTOH, if end user impact were the driving force, I'd rather see Decimal(string) raise an Inexact exception if the string would be rounded: practice, so this is
# solely for a highly unusual edge case! decimal.getcontext().prec=5
# This confuses me - it silently gives "the wrong" answer in my mental model. Decimal("1.23456789") * 2 Decimal("2.4691")
c = decimal.getcontext().copy() c.traps[decimal.Inexact] = True
# This does what I expect - it tells me that I've done something wrong! c.create_decimal("1.23456789") * 2 Traceback (most recent call last): File "<stdin>", line 1, in ? File "C:\Apps\Python24\lib\decimal.py", line 2291, in create_decimal return d._fix(self) File "C:\Apps\Python24\lib\decimal.py", line 1445, in _fix ans = ans._round(prec, context=context) File "C:\Apps\Python24\lib\decimal.py", line 1567, in _round context._raise_error(Inexact, 'Changed in rounding') File "C:\Apps\Python24\lib\decimal.py", line 2215, in _raise_error raise error, explanation decimal.Inexact: Changed in rounding
I hope this helps, Paul.
Raymond Hettinger wrote:
From the mists of Argentina, a Palidan set things right. The literal 1.1 became representable and throughout the land the monster was believed to have been slain.
I don't understand. Isn't the monster going to pop right back up again as soon as anyone does any arithmetic with the number? I don't see how you can regard what Decimal does as "schoolbook arithmetic" unless the teacher is reaching over your shoulder and blacking out any excess digits after everything you do. And if that's acceptable, I don't see how it helps significantly to have just the very first step -- turning the input into numbers -- be exempt from this behaviour. If anything, people are going to be even more confused. "But it can obviously cope with 1.1000000000000000001, so why does it give the wrong answer when I add something to it?" Greg
participants (7)
-
Greg Ewing -
Guido van Rossum -
Michael Chermside -
Paul Moore -
Raymond Hettinger -
Raymond Hettinger -
Tim Peters