[Python-Dev] Minutes from the Numeric Coercion dev-day session

[Tim Peters]

[Guido]
> I'd like to argue about that.  I think the extent to which HWFP
> doesn't work for newbies is mostly related to the change we made in
> 2.0 where repr() (and hence the interactive prompt) show full
> precision, leading to annoyances like repr(1.1) == '1.1000000000000001'.
>
> I've noticed that the number of complaints I see about this went way
> up after 2.0 was released.

Indeed yes, but I think that's a *good* thing.  We can't stop people from
complaining, but we can influence *what* they complain about, and it's
essential for newbies to learn ASAP that they have no idea how binary fp
arithmetic works.  Note that I spend a lot more of my life replying to these
complaints than you <wink>, and I can cut virtually all of them off early now
by pointing to the RepresentationError wiki page.  Before, it was an endless
sequence of "unique" complaints about assorted things that "didn't work
right", and that was much more time-consuming for me.  Of course, it's not a
positive help to the newbies so much as that scaring them early saves them
greater troubles later <no wink>.

Regular c.l.py posters can (& do!) handle this now too, thanks to hearing the
*same* complaint repeatedly now.  For example, over the past two days there
have been more than 40 messages on c.l.py about this, none of them stemming
from the conference or Moshe's PEP, and none of them written by me.  It's a
pattern:

+ A newcomer to Python complains about the interactive-prompt fp display.

+ People quickly uncover that's the least of their problems (that, e.g., they
truly *believe* Python should get dollars and cents exactly right all by
itself, and are programming as if that were true).

+ The fp display is the easiest of all fp surprises to explain fully and
truthfully (although the wiki page should make painfully clear that "easiest"
!= "easy" by a long shot), so is the quickest route toward disabusing them of
their illusions.

+ A few people suggest they use my FixedPoint.py instead; a few more that
they compute using cents instead (using ints or longs); and there's always
some joker who flames that if they're writing code for clients and have such
a poor grasp of fp reality, they should be sued for "technical incompetence".

Except for the flames, this is good in my eyes.

> I expect that most newbies don't use floating point in a fancy way,
> and would never notice it if it was slightly off as long as the output
> was rounded like it was before 2.0.

I couldn't disagree more that ignorance is to be encouraged, either in
newbies or in experts.  Computational numerics is a difficult field with
major consequences in real life, and if the language can't actively *help*
people with that, it should at least avoid encouraging a fool's confidence in
their folly.  If that isn't virulent enough for you <wink>, read Kahan's
recent "Marketing versus Mathematics" rant, here:

    http://www.cs.berkeley.edu/~wkahan/MktgMath.pdf

A point he makes over and over, illustrated with examples, is this:

    Decimal displays of Binary nonintegers cannot always be WYSIWYG.

    Trying to pretend otherwise afflicts both customers and
    implementors with bugs that go mostly misdiagnosed, so “fixing”
    one bug merely spawns others. …

In a specific example of a nasty real-life bug beginning on page 13, he calls
the conceit (& source of the bug) of rounding fp displays to 15 digits
instead of 17 "a pious fraud".  And he's right.  It spares the implementer
some shallow complaints at the cost of leading naive users down a garden
path, where they end up deeper and deeper in weeds over their heads.

Of course he acknowledges that 17-digit display "[annoys] users who expected
roundoff to degrade only the last displayed digit of simple expressions, and
[confuses] users who did not expect roundoff at all" -- but seeking to fuzz
those truths has worse consequences.

In the end, he smacks up against the same need to favor one group at the
expense of the other:

   Binary floating-point is best for mathematicians, engineers and most
   scientists, and for integers that never get rounded off.  For everyone
   else Decimal floating-point is best because it is the only way What
   You See can be What You Get, which is a big step towards reducing
   programming languages’ capture cross-section for programming errors.

He's wrong via omission about the latter, though:  rationals are also a way
to achieve that (so long as you stick to + - * /; decimal fp is still
arguably better once a sqrt or transcendental gets into the pot).

>> Presumably ABC used rationals because usability studies showed
>> they worked best (or didn't they test this?).

> No, I think at best the usability studies showed that floating point
> had problems that the ABC authors weren't able to clearly explain to
> newbies.  There was never an experiment comparing FP to rationals.

>> Presumably the TeachScheme! dialect of Scheme uses rationals for
>> the same reason.

> Probably for the same reasons.

Well, you cannot explain binary fp *clearly* to newbies in reasonable time,
so I can't fault any teacher or newbie-friendly language for running away
from it.  Heck, most college-age newbies are still partly naive about fp
numerics after a good one-semester numerical analysis course (voice of
experience, there).

>> 1/10 and 0.1 are indeed very different beasts to me).

> Another hard question: does that mean that 1 and 1.0 are also very
> different beasts to you?  They weren't to the Alice users who started
> this by expecting 1/4 to represent a quarter turn.

1/4 *is* a quarter turn, and exactly a quarter turn, under every alternative
being discussed (binary fp, decimal fp, rationals).  The only time it isn't
is under Python's current rules.  So the Alice users will (presumably) be
happy with any change whatsoever from the status quo.

They may not be so happy if they do ten 1/10 turns and don't get back to
where they started (which would happen under binary fp, but not decimal fp or
rationals).

Some may even be so unreasonable <wink> as to be unhappy if six 1/6 turns
wasn't a wash (which leaves only rationals as surprise-free).

Paul Dubois wants a way to tag fp literals (see his proposal).  That's
reasonable for his field.  DrScheme's Student levels have a way to tag
literals as inexact too, which allows students to get their toes wet with
binary fp while keeping their gonads on dry land.  Most people can't ride
rationals forever, but they're great training wheels; thoroughly adequate for
dollars-and-cents computations (the denominators don't grow when they're all
the same, so $1.23 computations don't "blow up" in time or space); and a
darned useful tool for dead-serious numeric grownups in sticky numerical
situations (rationals are immune to all of overflow, underflow, roundoff
error, and catastrophic cancellation, when sticking to + - * /).

Given that Python can't be maximally friendly to everyone here, and has a
large base of binary fp users I don't hate at all <wink>, the best I can
dream up is:

    1.3    binary fp, just like now

    1.3_r  exact rational (a tagged fp literal)

    1/3    exact rational

    1./3   binary fp

So, yes, 1.0 and 1 are different beasts to me:  the "." alone and without an
"_r" tag says "I'm an approximation, and approximations are contagious:
inexact in, inexact out".

Note that the only case where this changes the meaning of existing code is

    1/3

But that has to change anyway lest the Alice users stay stuck at 0 forever.

> You know where I'm leaning...  I don't know that newbies are genuinely
> hurt by FP.

They certainly are burned by binary FP if they go on to do any numeric
programming.  The junior high school textbook formula for solving a quadratic
equation is numerically unstable.  Ditto the high school textbook formula for
computing variance.  Etc.  They're *surrounded* by deep pits; but they don't
need to be, except for the lack of *some* way to spell a newbie-friendly
arithmetic type.

> If we do it right, the naive ones will try 11.0/10.0, see
> that it prints 1.1, and be happy;

Cool.  I make a point of never looking at my chest x-rays either <0.9 wink>.

> the persistent ones will try 1.1**2-1.21, ask for an explanation, and
> get a introduction to floating point.  This *doesnt'* have to explain all
> the details, just the two facts that you can lose precision and that 1.1
> isn't representable exactly in binary.

Which leaves them where?  Uncertain & confused (as you say, they *don't* know
all the details, or indeed really any of them -- they just know "things go
wrong", without any handle on predicting the extent of the problems, let
alone any way of controlling them), and without an alternative they *can*
feel confident about (short of sticking to integers, which may well be the
most frequent advice they get on c.l.py).  What kind of way is that to treat
a poor newbie?

I'll close w/ Kahan again:

    Q. Besides its massive size, what distinguishes today’s market for
       floating-point arithmetic from yesteryears’ ?

    A. Innocence
       (if not inexperience, naïveté, ignorance, misconception,
        superstition, … )

non-extended-binary-fp-is-an-expert's-tool-ly y'rs  - tim