...

and that there's no definition of range other than the built-in one (this requires a subtle change of language rules); it can then substitute an internal equivalent to xrange().

That sounds good. What sort of subtle language change do you have in mind which would permit this deduction?

An official prohibition on inserting names in other namespaces that shadow built-ins. The prohibition needn't be enforced (although that would be nice). A program that does import foomod foomod.range = ... would be invalid, but an implementation might not be able to catch all cases, e.g. import foomod foomod.__dict__['range'] = ... It could be enforced, mostly, by making a module's __dict__ attribute return a read-only proxy like a new-style class's __dict__ attribute does, and putting an explicit ban on setting certain names in the module setattr implementation. But the module itself could also play games, e.g. it could do exec "range = ..." in globals() Another module could also do from foomod import f # a function f.func_globals['range'] = ... All these things would be illegal without necessarily being enforced. (The only way I see for total enforcement would be to change the dict implementation to trap certain assignments.) BTW, import foomod foomod.foo = ... would still be allowed -- it's only setting previously unset built-in names (or maybe built-in names that are known to be used by the module) that would be prohibited. Also, foomod could explicit allow setting an attribute by doing something like range = range # copy the built-in into a global to disable the optimization. I.e. setting something that's already set should always be allowed. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:

...

The bytecode compiler should be clever enough to see that you're writing

for i in range(...): ...

and that there's no definition of range other than the built-in one (this requires a subtle change of language rules); it can then substitute an internal equivalent to xrange().

Ouch! What happens to:

def foo(seq): for x in seq: ...

foo(xrange(small, really_big))

if xrange dies??

Good point. I guess xrange() can't die until range() becomes an iterator (which can't be before Python 3.0). Hm, maybe range() shouldn't be an iterator but an interator generator. No time to explain; see the discussion about restartable iterators. --Guido van Rossum (home page: http://www.python.org/~guido/)

...

Hm, maybe range() shouldn't be an iterator but an interator generator. No time to explain; see the discussion about restartable iterators.

I think what you mean is fairly obvious. list et al. are iterator generators, right? It's just a thing with an __iter__ function which produces an iterator?

If so, I tend to agree that's the right behavior for range(). range(x,y,z) should be an immutable object.

Yes. Idioms like this are used fairly often: seq = range(...) for i in seq: ... for i in seq: ... # etc. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, 2003-03-21 at 06:55, Guido van Rossum wrote:

...

...

...

Hm, maybe range() shouldn't be an iterator but an interator generator. No time to explain; see the discussion about restartable iterators.

[Chad Netzer]

Hmmm. Now that've uploaded my patch extending range() to longs,

(And now that I've checked it in. :-)

I'd like to work on this. I've already written a C range() iterator (incorporating PyLongs), and it would be very nice to have it automatically be a lazy range() when used in a loop.

In any case, assuming you are quite busy, but would consider this for the 2.4 timeframe, I will do some work on it. If it is already being covered, I'll gladly stay away from it. :)

range() can't be changed from returning a list until at least Python 3.0. xrange() already is an iterator well. So I'm not sure there's much to do, especially since I think making xrange() support large longs goes against the design goal for xrange(), which is to be a lightweight alternative for range() when speed is important. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Chad Netzer]

...

...

I'd like to work on this. I've already written a C range() iterator (incorporating PyLongs), and it would be very nice to have it automatically be a lazy range() when used in a loop.

In any case, assuming you are quite busy, but would consider this for the 2.4 timeframe, I will do some work on it. If it is already being covered, I'll gladly stay away from it. :)

[Guido]

...

range() can't be changed from returning a list until at least Python 3.0.

[MAL]

Is this change really necessary ? Instead of changing the semantics of range() why not have the byte code compiler optimize it's typical usage:

for i in range(10): pass

In the above case, changing the byte code compiler output would not introduce any change in semantics. Even better, the compiler could get rid off the function call altogether.

Right. That's nice, and can be done before 3.0 (as soon as we change the rules so that adding a 'range' attribute to a module object is illegal). My musing about making range() an iterator or iterator well comes from the observation that if I had had iterators as a concept from day one, I would have made several things iterators that currently return lists, e.g. map(), filter(), and range(). The need for the concrete list returned by range() (outside the tutorial :-) is rare; in those rare cases you could say list(range(...)). Whether this is indeed worth changing in 3.0 isn't clear, that depends on the scope of 3.0, which isn't defined yet (because I haven't had time to work on it, really). I certainly plan to eradicate xrange() in 3.0 one way or another: TOOWTDI. --Guido van Rossum (home page: http://www.python.org/~guido/)