[Python-ideas] Python Numbers as Human Concept Decimal System

[Guido van Rossum]

Thanks Oscar, that's a very well-reasoned post.

On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote:
> > I'll try to respond to Mark Dickinson's second message (and nothing else
> > that happens in the thread since last night), because (a) it concisely
> > summarizes his position and (b) brings up a new strawman.
> >
> > On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com>
> wrote:
> >> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com>
> wrote:
> >>> I see three sane options for float to Decimal conversion:
> >>>
> >>> 1. Raise an exception.
> >>> 2. Round to the nearest Decimal value using the current context for
> that
> >>> round operation.
> >>> 3. Do what we're currently doing, and do an exact conversion.
> >
> > I think you're writing this entirely from the POV of an expert in
> floating
> > point. And I'm glad we have experts like you around! I don't consider
> myself
> > an expert at all, but I do think I have something to bring to the table
> --
> > insight the experience of non-expert users.
>
> Standards compliance is important though. Mark is essentially
> restating something that is repeated hundreds of times throughout
> various floating point standards documents: a result may be exact or
> it must be correctly rounded according to the current context (with
> appropriate flags set and traps fired).
>

I have mixed feelings about such standards. I can see its importance. But
like the Unicode standard, it seems to want to grab authority over areas
that I think belong to the language design. Also at this point claiming
"compliance" with some standard is usually a morass of weasel-words rather
than clearly implementing a spec.


> It is not mandated that results requiring more precision than the
> current context be exact. It is mandated that if a result is to be
> inexact then the implementation must use a very specific type of
> inexactness. I don't believe that a standards-compliant decimal module
> has any wiggle room to invent a new kind of rounding (which I think
> would be required to achieve what you suggest).
>

That would be unfortunate.


>  > When a non-expert writes Decimal(1.1), each of the three above outcomes
> > surprises. We know that (1) was unpopular, that's why we changed it. We
> now
> > know that (3) is unpopular at least in some circles (Mark Harrison can't
> be
> > the only one who doesn't like it). Changing to (2) wouldn't do much to
> > address this, because the default context has way more precision than
> float,
> > so it still shows a lot of extraneous digits.
> >
> > Yes, it's trivial to get rid of those extra digits, just use quotes. But
> if
> > *my* proposal were adopted, it would be trivial for numerical experts to
> get
> > the extra digits -- just use from_float(). At this point, my claim is
> that
> > we're talking essentially about what is the better experience for most
> > users, and while I am not much of a user of Decimal myself, I believe
> that
> > my proposal has benefits more people and situations than it has
> downsides.
>
> If you write Decimal(1.1) and are surprised by the result then you
> have misunderstood something. It may be that you have little
> understanding of the difference between binary and decimal floating
> point (but then why are you using Decimal?). Perhaps you don't fully
> understand the literal->float->Decimal pathway that occurs when the
> expression is evaluated because you're new to Python or just haven't
> really thought about it before.
>

Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the
result, but it's still not what I want, and it's a pain to train myself to
do the extra thing that gives me what I want.


> In any case if the result of Decimal(1.1) surprises you then it's
> because you're expecting it do something that should be done in a
> different way. Hiding the extra digits does not help a user to
> understand how to use Decimal.
>

But does showing the extra digits do anything to help? It's just as likely
to teach them a trick (add quotes or a str() call) without any new
understanding.


> I actually use this in teaching to demonstrate how binary floating
> point works. I think it's important when teaching my students for them
> to understand that the following is a lie:
>
>     >>> a = 1.1
>     >>> a
>     1.1
>
> The helpful digit-hiding repr is lying to you. There is no float with
> the value 1.1. I can sort-of demonstrate this with some arithmetic:
>
>     >>> 0.1 + 0.11 - 0.11 - 0.1
>     1.3877787807814457e-17
>
> But that gives the misleading impression that inexact arithmetic is
> the source of the error. It's important to understand that the error
> occurs straight away in the literal "1.1". I demonstrate this by
> showing that
>
>     >>> from decimal import Decimal
>     >>> a = 1.1
>     >>> Decimal(a)
>     Decimal('1.100000000000000088817841970012523233890533447265625')
>
> which also shows the true value stored in the float.
>
> The fact that I use this in teaching is not supposed to serve as an
> argument for keeping it (I could just as easily use from_float). My
> point is that showing the digits helps someone to understand what is
> going on. If a user is converting float to Decimal without knowing
> what they're doing then the extra digits are a clue that they don't
> fully understand what's happening and haven't necessarily used the
> best approach.
>

I don't think every user of Decimal necessarily needs to be an expert
capable of explaining what's going on. Certainly that's not needed to be an
effective user of float -- the anomalies are explained to most people's
satisfaction by some hand-waving about imprecise results.


> There is a good solution to the problem of non-experts wanting to
> write 1.1 and get the exact value 1.1: decimal literals. With that
> they can just write 1.1d and not need to learn any more about it.
> Earlier in this thread you reject that idea saying that you can't
> teach it to "newbies":
> '''
> Maybe we can add a new literal notation meaning 'decimal'; that would
> be a relatively straightforward change to the parser (once the new
> decimal extension module is incorporated), but it would not do much to
> avoid surprising newbies (certainly you can't go teaching them to
> always write 3.14d instead of 3.14). However, it would probably help
> out frequent users of Decimal. (Then again, they might not be using
> literals that much -- I imagine the source of most such programs is
> user or file input.)
> '''
> I disagree. If you're at the point of wanting to use the Decimal
> module then you're at the point where it's reasonable to learn about
> Decimal literals.
>

You're right that I dismissed it too quickly. 3.14d is clearly even better
than Decimal(3.14) doing the right thing. It is also still a lot more work
(touches many parts of the code rather than just the Decimal class).

Also I didn't realize that the C-implemented decimal module was already
used in CPython (so I thought it would be even more work).


> Also I've written code using the decimal module for high precision
> calculation and it has lots of decimal literals. That is I do (and I
> see other's doing)
>
>     >>> from decimal import Decimal as D
>     >>> d = D('1e-20')
>
> even though this is never used in the decimal documentation. This is
> the closest you can get to decimal literals right now.


Right.

But I still have this nagging feeling that the precision Decimal(<float>)
currently gives you is, in a sense, *fake*, given that the input has much
less precision.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/e16f9768/attachment.html>