[Numpy-discussion] RE: Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)

[Tim Peters]

[Huaiyu Zhu]
> On the issue of whether Python should ignore over/underflow on
> IEEE-enabled platforms:
>
> It can be argued that on IEEE enabled systems the proper thing to do for
> overflow is simply return Inf.  Raising exception is WRONG.  See below.

Python was not designed with IEEE-754 in mind, and *anything* that happens
wrt Infs, NaNs, and all other 754 features in Python is purely an accident
that can and does vary wildly across platforms.  And, as you've discovered
in this case, can vary wildly also across even a single library, depending
on config options.  We cannot consider this to be a bug since Python has had
no *intended* behavior whatsoever wrt 754 gimmicks.  We can and do consider
gripes about these accidents to be feature requests.

[Guido]
> Incidentally, math.exp(800) returns inf in 1.5, and raises
> OverflowError in 2.0.  So at least it's consistent.

[Huaiyu]
> That is not consistent at all.

Please read with an assumption of good faith.  Guido was pointing out that--
all in the specific case of gcc+glibc on Linux (these don't hold on other
platforms) --exp(800) returning Inf in 1.5 and OverflowError in 2.0 is
consistent *with* that exp(-800) returns 0 in 1.5 and raises an exception in
2.0.  He's right; indeed, he is in part agreeing with you.

[Guido
> 1.5.2 links with -lieee while 2.0 doesn't.  Removing -lieee from the
> 1.5.2 link line makes is raise OverflowError too.  Adding it to the
> 2.0 link line makes it return 0.0 for exp(-1000) and inf for
> exp(1000).

[Huaiyu]
> If using ieee is as simple as setting such a flag, there is no
> reason at all not to use it.

The patch to stop setting -lieee was contributed by a Python user who
claimed it fixed bugs on *their* platform.  That's "the reason".  We don't
want to screw them either.

> Here are some more examples:
> ...

I understand that 754 semantics can be invaluable.  So does Guido.  There's
no argument about that.  But Python doesn't yet support them, and wishing it
did doesn't make it so.  If linking with -lieee happens to give you the
semantics you want on your platform today, you're welcome to build with that
switch.  It appears to be a bad choice for *most* Python users, though (more
below), so we don't want to do it in the std 2.0 distro.

> ...
> In practice this simply means Python would not be suitable for numerical
> work at all.

Your biggest obstacle in getting Python support for 754 will in fact be
opposition from number-crunchers.  I've been slinging fp for 21 years, 15 of
those writing compilers and math libraries for "supercomputer" vendors.  754
is a Very Good Thing, but is Evil in subset form (see Kahan's (justified!)
vilification of Java on this point); ironically, 754 is hardest to sell to
those who could benefit from it the most.

> What about the other way round?  No problem.  It is easy to write
> functions like isNaN, isInf, etc.

It's easy for a platform expert to write such functions for their specific
platform of expertise, but it's impossible to write them in a portable way
before C99 is implemented (C99 requires that library suppliers provide them,
rendering the question moot).

> ...
> The language should never take over or subvert decisions based on
> numerical analysis.

Which is why a subset of 754 is evil.  754 requires that the user be able to
*choose* whether or not, e.g., overflow signals an exception.  Your crusade
to insist that it never raise an exception is as least as bad (I think it's
much worse) as Python's most common accidental behavior (where overflow from
libm usually raises an exception).  One size does not fit all.

[Tim]
> Ignoring ERANGE entirely is not at all the same behavior as 1.5.2, and
> current code certainly relies on detecting overflows in math functions.

[Huaiyu]
> As Guido observed ERANGE is not generated with ieee, even for
> overflow.  So it is the same behavior as 1.5.2.

You've got a bit of a case of tunnel vision here, Huaiyu.  Yes, in the
specific case of gcc+glibc+Linux, ignoring ERANGE returned from exp() is
what 1.5.2 acted like.  But you have not proposed to ignore it merely from
ERANGE returned from exp() in the specific case of gcc+glibc+Linux.  This
code runs on many dozens of platforms, and, e.g., as I suggested before, it
looks like HPUX has always set errno on both overflow and underflow.  MS
Windows sets it on overflow but not underflow.  Etc.  We have to worry about
the effects on all platforms.

> Besides, no correct numerical code should depend on exceptions like
> this unless the machine is incapable of handling Inf and NaN.

Nonsense.  754 provides the option to raise an exception on overflow, or
not, at the user's discretion, precisely because exceptions are sometimes
more appropriate than Infs of NaNs.  Kahan himself isn't happy with Infs and
NaNs because they're too often too gross a clue (see his writings on
"presubstitution" for a more useful approach).

[Tim]
> In no case can you expect to see overflow ignored in 2.0.

[Huaiyu]
> You are proposing a dramatic change from the behavior of 1.5.2.
> This looks like to me to need a PEP and a big debate.  It would break
> a LOT of numerical computations.

I personally doubt that, but realize it may be true.  That's why he have
beta releases.  So far yours is the only feedback we've heard (thanks!), and
as a result we're going to change 2.0 to stop griping about underflow, and
do so in a way that will actually work across all platforms.  We're probably
going to break some HPUX programs as a result; but they were relying on
accidents too.

[Guido]
> No, the configure patch is right.  Tim will check in a change that
> treats ERANGE with a return value of 0.0 as underflow (returning 0.0,
> not raising OverflowError).

[Huaiyu]
> What is the reason to do this?  It looks like intetionally subverting
> ieee even when it is available.  I thought Tim meant that only logistical
> problems prevent using ieee.

Python does not support 754 today.  Period.  I would like it to, but not in
any half-assed, still platform-dependent, still random crap-shoot, still
random subset of 754 features, still rigidly inflexible, way.  When it does
support it, Guido & I will argue that it should enable (what 754 calls) the
overflow, divide-by-0 and invalid operation exceptions by default, and
disable the underflow and inexact exceptions by default.  This ensures that,
under the default, an infinity or NaN can never be created from
non-exceptional inputs without triggering an exception. Not only is that
best for newbies, you'll find it's the *only* scheme for a default that can
be sold to working number-crunchers (been there, done that, at Kendall
Square Research).  It also matches Guido's original, pre-754, *intent* for
how Python numerics should work by default (it is, e.g., no accident that
Python has always had an OverflowError exception but never an UnderflowError
one).

And that corresponds to the change Guido says <wink> I'm going to make in
mathmodule.c:  suppress complaints about underflow, but let complaints about
overflow go through.  This is not a solution, it's a step on a path towards
a solution.  The next step (which will not happen for 2.0!) is to provide an
explicit way to, from Python, disable overflow checking, and that's simply
part of providing the control and inquiry functions mandated by 754.  Then
code that would rather deal with Infs than exceptions can, at its explicit
discretion.

> If you do choose this route, please please please ignore ERANGE entirely,
> whether return value is zero or not.

It should be clear that isn't going to happen.  I realize that triggering
overflow is going to piss you off, but you have to realize that not
triggering overflow is going to piss more people off, and *especially* your
fellow number-crunchers.  Short of serious 754 support, picking "a winner"
is the best we can do for now.  You have the -lieee flag on your platform du
jour if you can't bear it.

[Paul Dubois]
> I don't have time to follow in detail this thread about changed behavior
> between versions.

What makes you think we do <0.5 wink>?

> These observations are based on working with hundreds of code authors. I
> offer them as is.

FWIW, they exactly match my observations from 15 years on the HW vendor
side.

> a. Nobody runs a serious numeric calculation without setting underflow-to-
> zero, in the hardware. You can't even afford the cost of software checks.
> Unfortunately there is no portable way to do that that I know of.

C allows libm implementations a lot of discretion in whether to set errno to
ERANGE in case of underflow.  The change we're going to make is to ignore
ERANGE in case of underflow, ensuring that math.* functions will *stop*
raising underflow exceptions on all platforms (they'll return a zero
instead; whether +0 or -0 will remain a platform-dependent crap shoot for
now).  Nothing here addresses underflow exceptions that may be raised by fp
hardware, though; this has solely to do with the platform libm's treatment
of errno.

So in this respect, we're removing Python's current unpredictability, and in
the way you favor.

> b. Some people use Inf but most people want the code to STOP so they can
> find out where the INFS started. Otherwise, two hours later you have big
> arrays of Infs and no idea how it happened.

Apparently Python on libc+glibc+Linux never raised OverflowError from math.*
functions in 1.5.2 (although it did on most other platforms I know about).
A patch checked in to fix some other complaint on some other Unixish
platform had the side-effect of making Python on libc+glibc+Linux start
raising OverflowError from math.* functions in 2.0, but in both overflow and
underflow cases.  We intend to (as above) suppress the underflow exceptions,
but let the overflow cases continue to raise OverflowError.  Huaiyu's
original complaint was about the underflow cases, but (as such things often
do) expanded beyond that when it became clear he would get what he asked for
<wink>.

Again we're removing Python's current unpredictability, and again in the way
you favor -- although this one is still at the mercy of whether the platform
libm sets errno correctly on overflow (but it seems that most do these
days).

> Likewise sqrt(-1.) needs to stop, not put a zero and keep going.

Nobody has proposed changing anything about libm domain (as opposed to
range) errors (although Huaiyu probably should if he's serious about his
flavor of 754 subsetism -- I have no idea what gcc+glibc+Linux did here on
1.5.2, but it *should* have silently returned a NaN (not a zero) without
setting errno if it was *self*-consistent -- anyone care to check that
under -lieee?:

    import math
    math.sqrt(-1)

NaN or ValueError?  2.0 should raise ValueError regardless of what 1.5.2 did
here.).

just-another-day-of-universal-python-harmony-ly y'rs  - tim