On 14 Sep 2007, at 15:54, Lou Pecora wrote:

I thought this is what the linspace function was written for in numpy. Why not use that?

AFAIK, linspace() is written to generate N evenly spaced numbers between start and stop inclusive. Similar but not quite the same as arange().

It works just like you would want always including the final point.

The example I gave was actually meant to _avoid_ inclusion of the last point. E.g. In [93]: arange(0.0, 0.4+0.2, 0.1) Out[93]: array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6]) In [94]: myrange(0.0, 0.4+0.2, 0.1) Out[94]: array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5]) where myrange() is an ad hoc replacement for arange(): def myrange(start, stop, step): r = finfo(double).resolution N = min(ceil((stop-start)/step), ceil((stop-start)/step-r)) return start + arange(N) * step I'm not 100% sure that the above version of myrange() wouldn't generate surprising results in some cases. If it doesn't, why not include it in (the C-version of) arange()? I don't think users actually count on the inclusion of the end point in some cases, so it would not break code. It would, however, avoid some surprises from time to time. From the example of Lorenzo, it seems that Matlab is always including the endpoint. How exactly is their arange version defined? Joris

On 14/09/2007, Robert Kern <robert.kern@gmail.com> wrote:

Ed Schofield wrote:

...

Using arange in this way is a fundamentally unreliable thing to do, but is there anything we want to do about this?

Tell people to use linspace(). Yes, it does a slightly different thing; that's why it works. Most uses of floating point arange() can be cast using linspace() more reliably.

I would like to point out in particular that numpy's linspace can leave out the last point (something I often want to do): Definition: linspace(start, stop, num=50, endpoint=True, retstep=False) Docstring: Return evenly spaced numbers. Return num evenly spaced samples from start to stop. If endpoint is True, the last sample is stop. If retstep is True then return the step value used. This is one of those cases where "from pylab import *" is going to bite you, though, because its linspace doesn't. You can always fake it with linspace(a,b,N+1)[:-1]. Anne

On 9/14/07, Joris De Ridder <Joris.DeRidder@ster.kuleuven.be> wrote:

...

the question is how to reduce user astonishment.

IMHO this is exactly the point. There seems to be two questions here: 1) do we want to reduce user astonishment, and 2) if yes, how could we do this? Not everyone seems to be convinced of the first question, replying that in many cases linspace() could well replace arange(). In many cases, yes, but not all. For some cases arange() has its legitimate use, even for floating point, and in these cases you may get bitten by the inexact number representation. If Matlab seems to be able to avoid surprises, why not numpy?

Perhaps because it's a bad idea? This case may be different, but in general in cases where you try to sweep the surprising nature of floating point under the rug, you are never entirely successful. The end result is that, although surprises crop up with less regularity, they are much, much harder to diagnose and understand when they do crop up. If arange can be "fixed" in a way that's easy to understand, then great. However, if the algorithm for deciding the points is anything but dirt simple, leave it alone. Or, perhaps, deprecate floating point values as arguments. I'm not very convinced by the arguments advanced thus far that arange with floating point has legitimate uses. I've certainly used it this way myself, but I believe that all of my uses could easily be replaced with either linspace or arange with integer arguments. I suspect that cases where the exact properties of arange are required are far between and it's easy enough to simulate the current behaviour if needed. An advantage to that is that the potential pitfalls become obvious when you roll your own version. -- . __ . |-\ . . tim.hochberg@ieee.org

Charles R Harris wrote:

In the case of arange it should be possible to determine when the result is potentially ambiguous and issue a warning. For instance, if the argument of the ceil function is close to its rounded value.

What's "close"? The appropriate tolerance depends on the operations that would cause error. For literal inputs, where the only source of error is representation error, 1 eps would suffice, but then so would linspace(). For results of other computations, you might need more than 1 eps. But if you're doing computations, then it oughtn't to matter whether you get the endpoint or not (since you don't know what the values are anyway). -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Joris De Ridder wrote:

...

the question is how to reduce user astonishment.

IMHO this is exactly the point. There seems to be two questions here: 1) do we want to reduce user astonishment, and 2) if yes, how could we do this? Not everyone seems to be convinced of the first question, replying that in many cases linspace() could well replace arange(). In many cases, yes, but not all. For some cases arange() has its legitimate use, even for floating point, and in these cases you may get bitten by the inexact number representation. If Matlab seems to be able to avoid surprises, why not numpy?

Here's the thing: binary floating point is intrinsically surprising to people who are only accustomed to decimal. The way to not be surprised is to not use binary floating point. You can hide some of the surprises, but not all of them. When you do try to hide them, all you are doing is creating complicated, ad hoc behavior that is also difficult to predict; for those who have become accustomed to binary floating point's behavior, it's not clear what the "unastonishing" behavior is supposed to be, but binary floating point's is well-defined. Binary floating point is a useful tool for many things. I'm not interested in making numpy something that hides that tool's behavior in order to force it into a use it is not appropriate for. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On 15/09/2007, Christopher Barker <Chris.Barker@noaa.gov> wrote:

Oh, and could someone post an actual example of a use for which FP arange is required (with fudges to try to accommodate decimal to binary conversion errors), and linspace won't do?

Well, here's one: evaluating a function we know to be bandlimited to N harmonics and positive trying to bracket a maximum. We know it doesn't change much faster than T/N, so I might use xs = arange(0,T,1/float(4*N)) and then evaluate the function there. Of course, I don't care how many points there are, so no fudges please. But floating-point arange is certainly useful here; to use linspace or integer arange I'd have to write it in a much clumsier way. (Okay, a little clumsier.) In fact, reluctant as I am to provide arguments in favour of godawful floating-point fudges, if I have the harmonics I can use irfft to evaluate my function. I'll then have to carefully calculate the x-values where irfft evaluates, and an off-by-one problem is going to cause my program to fail. I would use integer arange and scale as appropriate, but there's something to be said for using floating-point arange. linspace(...,endpoint=False) is fine, though. Anne

On 14 Sep 2007, at 23:51, Robert Kern wrote:

You can hide some of the surprises, but not all of them.

I guess it's impossible to make a bullet-proof "fix". When arange() gets a 'stop' value of 0.60000000000000009, it cannot possibly know whether this stop value is supposed to be 0.6, or whether this value is the result of a genuine computation that has nothing to do with inexact number representation. In the latter case, I would definitely want arange() to be working as it does now. It seems, though, if linspace() is the better equivalent of arange(), its default endpoint=True option seems a little bit inconvenient (but by no means a problem), as you would always have to reset it to emulate arange() behaviour. Joris Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

On 14/09/2007, Charles R Harris <charlesr.harris@gmail.com> wrote:

Since none of the numbers are exactly represented in IEEE floating point, this sort of oddity is expected. If you look at the exact values, (.4 + .2)/.1 > 6 and .6/.1 < 6 . That said, I would expect something like ceil(interval/delta - relatively_really_small_number) would generally return the expected result. Matlab probably plays these sort of games. The downside is encouraging bad programming habits. In this case, the programmer should be using linspace..

There is actually a context in which floating-point arange makes sense: when you want evenly-spaced points and don't much care how many there are. No reason to play games in this context of course; the question is how to reduce user astonishment. Anne

On Friday 14 September 2007 20:12, Charles R Harris wrote:

Since none of the numbers are exactly represented in IEEE floating point, this sort of oddity is expected. If you look at the exact values, (.4 + .2)/.1 > 6 and .6/.1 < 6 . That said, I would expect

You hit send too fast! The fractions that can be represented exactly in binary are: 1/2, 1/4, 1/8, ... and not 2/10, 4/10, 8/10 .... See here: In [1]:0.5 == .25+.25 Out[1]:True In [2]:.5 Out[2]:0.5 In [3]:.25 Out[3]:0.25 In [4]:.125 Out[4]:0.125 In [8]:.375 == .25 + .125 Out[8]:True Regards, Eike.