[Python-Dev] More int/long integration issues

[Guido van Rossum]

> I'm working on a patch that allows both range() and xrange() to work
> with large (PyLong) values.

I'm not interested for xrange().  As I said, xrange() is a crutch and
should not be given features that make it hard to kill.

For range(), sure, upload to SF.

> I noticed the need for a least supporting long values when I found some
> bugs in code that did things like:
> 
> a = 1/1e-5
> range( a-20, a)

This should be a TypeError.  I'm sorry it isn't.  range() is only
defined for ints, and unfortunately if you pass it a float it
truncates rather than failing.

> or
> 
> a = 1/1e-6
> b = 1/1e-5
> c = 1/1e-4
> range(a, b, c)

Ditto.

(BTW why don't you write this as 1e6, 1e5, 1e4???)

> Now, this example is hardcoded, but in graphing software, or other
> numerical work, the actual values come from the data set.  All of a
> sudden, you could be dealing with very small numbers (say, because you
> want to examine error values), and you get:
> 
> a = 1/1e-21
> b = 1/1e-20
> c = 1/1e-19
> range(a, b, c)
> 
> And your piece of code now fails.  By the comments I've seen, this
> failure tends to come as a big surprise (people are simply expecting
> range to be able to work with PyLong values, over short lengths).

But 1/1e-21 is not a long.  It's a float.  You're flirting with
disaster here.

> Also, someone who is working with large files (> C long on his machine)
> claimed to be having problems w/ xrange() failing (although, if he is
> indexing the xrange object, my patch can't help anyway)

That's a totally different problem.  Indeed you can't use xrange()
with values > sys.maxint.  But it should be easy to recode this
without xrange.

> I've seen enough people asking in the newsgroups about this behavior (at
> least four in the past 5 months or so), and I've submitted some
> application patches to make things work for these cases (ie. by
> explicitly subtracting out the large common base of each parameter, and
> adding it back in after the list is generated), so I decided to make a
> patch to change the range behavior.
> 
> Fixing range was relatively easy, and could be done with no performance
> penalty (the code to handle longs ranges is only invoked after the
> existing code path fails; the common case is unaltered).  Fixing
> xrange() is trickier, and I'm opting to maintain backwards compatibility
> as much as possible.
> 
> In any case, I should have the patch ready to submit within the next
> week or so (just a few hours more work is needed, for testing and
> cleanup)
> 
> Then the argument about whether it should ever be included can begin in
> earnest.  But I have seen enough examples of people being surprised that
> ranges of long values (where the range length is well within the
> addressable limit, but the range values must be PyLongs) that I think at
> least range() should be fixed.

Yes.

> And if range() is fixed, then sadly,
> xrange() should be fixed as well (IMO).

No.

> BTW, I'm all for deprecating xrange() with all deliberate speed.  Doing
> so would only make updating range behavior easier.

It can't be deprecated until we have an alternative.  That will have
to wait until Python 2.4.  I fought its addition to the language long
and hard, but the arguments from PBP (Practicality Beats Purity) were
too strong.

--Guido van Rossum (home page: http://www.python.org/~guido/)