Thanks Oscar, that's a very well-reasoned post.
On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin
On 8 March 2014 16:54, Guido van Rossum email@example.com wrote:
I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman.
On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson firstname.lastname@example.org wrote:
On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson email@example.com wrote:
I see three sane options for float to Decimal conversion:
I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself
insight the experience of non-expert users.
Standards compliance is important though. Mark is essentially restating something that is repeated hundreds of times throughout various floating point standards documents: a result may be exact or it must be correctly rounded according to the current context (with appropriate flags set and traps fired).
I have mixed feelings about such standards. I can see its importance. But like the Unicode standard, it seems to want to grab authority over areas that I think belong to the language design. Also at this point claiming "compliance" with some standard is usually a morass of weasel-words rather than clearly implementing a spec.
It is not mandated that results requiring more precision than the current context be exact. It is mandated that if a result is to be inexact then the implementation must use a very specific type of inexactness. I don't believe that a standards-compliant decimal module has any wiggle room to invent a new kind of rounding (which I think would be required to achieve what you suggest).
That would be unfortunate.
When a non-expert writes Decimal(1.1), each of the three above outcomes surprises. We know that (1) was unpopular, that's why we changed it. We now know that (3) is unpopular at least in some circles (Mark Harrison can't be the only one who doesn't like it). Changing to (2) wouldn't do much to address this, because the default context has way more precision than float, so it still shows a lot of extraneous digits.
Yes, it's trivial to get rid of those extra digits, just use quotes. But if my proposal were adopted, it would be trivial for numerical experts to get the extra digits -- just use from_float(). At this point, my claim is that we're talking essentially about what is the better experience for most users, and while I am not much of a user of Decimal myself, I believe that my proposal has benefits more people and situations than it has downsides.
If you write Decimal(1.1) and are surprised by the result then you have misunderstood something. It may be that you have little understanding of the difference between binary and decimal floating point (but then why are you using Decimal?). Perhaps you don't fully understand the literal->float->Decimal pathway that occurs when the expression is evaluated because you're new to Python or just haven't really thought about it before.
Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the result, but it's still not what I want, and it's a pain to train myself to do the extra thing that gives me what I want.
In any case if the result of Decimal(1.1) surprises you then it's because you're expecting it do something that should be done in a different way. Hiding the extra digits does not help a user to understand how to use Decimal.
But does showing the extra digits do anything to help? It's just as likely to teach them a trick (add quotes or a str() call) without any new understanding.
I actually use this in teaching to demonstrate how binary floating point works. I think it's important when teaching my students for them to understand that the following is a lie:
>>> a = 1.1 >>> a 1.1
The helpful digit-hiding repr is lying to you. There is no float with the value 1.1. I can sort-of demonstrate this with some arithmetic:
>>> 0.1 + 0.11 - 0.11 - 0.1 1.3877787807814457e-17
But that gives the misleading impression that inexact arithmetic is the source of the error. It's important to understand that the error occurs straight away in the literal "1.1". I demonstrate this by showing that
>>> from decimal import Decimal >>> a = 1.1 >>> Decimal(a) Decimal('1.100000000000000088817841970012523233890533447265625')
which also shows the true value stored in the float.
The fact that I use this in teaching is not supposed to serve as an argument for keeping it (I could just as easily use from_float). My point is that showing the digits helps someone to understand what is going on. If a user is converting float to Decimal without knowing what they're doing then the extra digits are a clue that they don't fully understand what's happening and haven't necessarily used the best approach.
I don't think every user of Decimal necessarily needs to be an expert capable of explaining what's going on. Certainly that's not needed to be an effective user of float -- the anomalies are explained to most people's satisfaction by some hand-waving about imprecise results.
There is a good solution to the problem of non-experts wanting to write 1.1 and get the exact value 1.1: decimal literals. With that they can just write 1.1d and not need to learn any more about it. Earlier in this thread you reject that idea saying that you can't teach it to "newbies": ''' Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.) ''' I disagree. If you're at the point of wanting to use the Decimal module then you're at the point where it's reasonable to learn about Decimal literals.
You're right that I dismissed it too quickly. 3.14d is clearly even better than Decimal(3.14) doing the right thing. It is also still a lot more work (touches many parts of the code rather than just the Decimal class).
Also I didn't realize that the C-implemented decimal module was already used in CPython (so I thought it would be even more work).
Also I've written code using the decimal module for high precision calculation and it has lots of decimal literals. That is I do (and I see other's doing)
>>> from decimal import Decimal as D >>> d = D('1e-20')
even though this is never used in the decimal documentation. This is the closest you can get to decimal literals right now.
But I still have this nagging feeling that the precision Decimal(<float>) currently gives you is, in a sense, fake, given that the input has much less precision.
-- --Guido van Rossum (python.org/~guido)