[Python-Dev] RE: [Numpy-discussion] RE: Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)

[Huaiyu Zhu]

[Paul Dubois]
> > 
> > a. Nobody runs a serious numeric calculation without setting
> > underflow-to-zero, in the hardware. You can't even afford the cost of
> > software checks. Unfortunately there is no portable way to do that that I
> > know of.

Amen.

> > 
> > b. Some people use Inf but most people want the code to STOP so they can
> > find out where the INFS started. Otherwise, two hours later you have big
> > arrays of Infs and no idea how it happened. Likewise sqrt(-1.) needs to
> > stop, not put a zero and keep going.

$ /usr/bin/python
Python 1.5.2 (#1, Sep 17 1999, 20:15:36)  [GCC egcs-2.91.66 19990314/Linux
(egcs- on linux-i386
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from math import *
>>> exp(777)
inf
>>> exp(-777)
0.0
>>> sqrt(-1)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
OverflowError: math range error

This was sane behavior.  Are we saying that Python 2.0 has invented
something better than IEEE 754?

[Guido van Rossum]

> Thanks, Paul!  This behavior has always been what I wanted Python to
> do (even though it's not always what Python did, depending on the
> platform) and also what Tim's proposed patch will implement for the
> specific case of math.exp() (and other math functions that may
> underflow or overflow), on most reasonable platforms.

Guido, with due respect to your decisions on Python issues, I simply have to
fight this one.  It is one thing to accomodate for naive users, but it is
another to dumb down every one else.

Case 1.  Someone writes a flawed numerical routine.  Two hours later he
finds his array filled with Inf and NaN.

Case 2.  Someone writes a perfect numerical routine.  Two hours later he
gets an exception, because the error is near zero.

Solution for case 1.  Use better algorithm.  Use better error control.
Raise exceptions when error is too large.  These are proper solutions.  
They are easy and efficient to implement.  They are needed anyway - If
something's wrong, you want to raise exceptions far earlier than Inf,
certainly before you get arrays filled with elements like 1e300.

Solution for case 2.  Almost impossible.  The division between under- and
over-flow is artificial.  What about 1/x or similar functions?  The only way
to work on such a platform is to abandon vectorized computation.

> There are still lots of places where the platform gives Python no
> choice of creating NaN and Inf, and because there's no
> platform-independent way to test for these, they are hard to avoid in
> some cases; but eventually, Tim will find a way to root them out.  And
> for people like Huaiyu, who want to see Inf, there will (eventually)
> be a way to select this as a run-time option; and ditto for whoever
> might want underflow to raise an exception.

I can understand that exceptions are the only available choices if IEEE is
not available.  But is there a compelling reason that Python should behave
"better" than IEEE when it's in fact available?

If the reason is to protect naive users, I can think of several responses:

1. For people doing one-off interactive work, returning Inf is in fact more
   informative. 

2. For users iterative numerical computations, they need to be educated
   about error control.  Otherwise they won't get corrent results anyway.

3. For really serious work, we could provide good numerical modules so that
   they don't need to write themselves.  To make this happen fast the fewer
   debacles like this one the better.

Case in point: Someone asked for regession modules two weeks ago.  I was
trying to convert my old matlab programs, which only took a few hours.  But
I wasted a week of (spare) time fighting for some mysterious "overflow".
Turns out that a Guassian is near zero when it's far from center, and Python
does not like it.

In practice, Inf may be generated more often as a proper value than by
mistake.

This is not an issue about whether someone "prefers" Inf or exception.  It
is about whether there is a choice to do proper computation.  Returning Inf
does not prevent someone to raise exception.  Raising exception
automatically prevents perfect algorithms to work properly.

As Kevin has volunteered to help with IEEE implementation and made a plan,
is there a strong reason to drop IEEE for Linux in 2.0?  If there is
insufficient time to carry out his plan, wouldn't it be prudent to keep
things as they were in 1.5.2?

Huaiyu