[Python-Dev] More int/long integration issues
Guido van Rossum
Thu, 13 Mar 2003 21:53:03 -0500
> I'm working on a patch that allows both range() and xrange() to work
> with large (PyLong) values.
I'm not interested for xrange(). As I said, xrange() is a crutch and
should not be given features that make it hard to kill.
For range(), sure, upload to SF.
> I noticed the need for a least supporting long values when I found some
> bugs in code that did things like:
> a = 1/1e-5
> range( a-20, a)
This should be a TypeError. I'm sorry it isn't. range() is only
defined for ints, and unfortunately if you pass it a float it
truncates rather than failing.
> a = 1/1e-6
> b = 1/1e-5
> c = 1/1e-4
> range(a, b, c)
(BTW why don't you write this as 1e6, 1e5, 1e4???)
> Now, this example is hardcoded, but in graphing software, or other
> numerical work, the actual values come from the data set. All of a
> sudden, you could be dealing with very small numbers (say, because you
> want to examine error values), and you get:
> a = 1/1e-21
> b = 1/1e-20
> c = 1/1e-19
> range(a, b, c)
> And your piece of code now fails. By the comments I've seen, this
> failure tends to come as a big surprise (people are simply expecting
> range to be able to work with PyLong values, over short lengths).
But 1/1e-21 is not a long. It's a float. You're flirting with
> Also, someone who is working with large files (> C long on his machine)
> claimed to be having problems w/ xrange() failing (although, if he is
> indexing the xrange object, my patch can't help anyway)
That's a totally different problem. Indeed you can't use xrange()
with values > sys.maxint. But it should be easy to recode this
> I've seen enough people asking in the newsgroups about this behavior (at
> least four in the past 5 months or so), and I've submitted some
> application patches to make things work for these cases (ie. by
> explicitly subtracting out the large common base of each parameter, and
> adding it back in after the list is generated), so I decided to make a
> patch to change the range behavior.
> Fixing range was relatively easy, and could be done with no performance
> penalty (the code to handle longs ranges is only invoked after the
> existing code path fails; the common case is unaltered). Fixing
> xrange() is trickier, and I'm opting to maintain backwards compatibility
> as much as possible.
> In any case, I should have the patch ready to submit within the next
> week or so (just a few hours more work is needed, for testing and
> Then the argument about whether it should ever be included can begin in
> earnest. But I have seen enough examples of people being surprised that
> ranges of long values (where the range length is well within the
> addressable limit, but the range values must be PyLongs) that I think at
> least range() should be fixed.
> And if range() is fixed, then sadly,
> xrange() should be fixed as well (IMO).
> BTW, I'm all for deprecating xrange() with all deliberate speed. Doing
> so would only make updating range behavior easier.
It can't be deprecated until we have an alternative. That will have
to wait until Python 2.4. I fought its addition to the language long
and hard, but the arguments from PBP (Practicality Beats Purity) were
--Guido van Rossum (home page: http://www.python.org/~guido/)