xrange question

[Carlos Ribeiro]

At 03:14 07/05/01 +1000, John Flynn wrote:
>The question is: what am I missing? What does 'xrange' do that 'while'
>cannot do better?

It's a question of performance - both in speed and memory usage 
-consistency, and aesthetics (the Python way of doing things - loops in 
this case).

The idiom used for loops over integer ranges in Python was:

for x in range(limit):
     do_something(x)

but this does poes a problem. The range() function generates the entire 
list even before the loops is called. This is expensive, mainly in terms of 
memory usage, because the list has to be allocated in memory; and also in 
terms of speed. There also a possibility that the loop is going to be 
terminated by a break statement; in this case, all the elements that were 
not iterated (because of the break) are not going to be used anymore, and 
all the time/memory that was used to initialize them is lost.

So we got xrange() - a special function call that builds a "lazy" list, 
that is able to returns each member without creating all the list members 
first. It solves the speed problem and keeps the idiom that we all are used:

for x in xrange(limit):
     do_something(x)

So why can't we use the while statement to solve the same problem, as you 
suggested?

x= 0
while x < 1000:
     x = x + 1
     do_something(x)

Try this compared to the for loop, and you'll be surprised by the speed 
loss. Why it is so? Because the explicit increment is done by the 
interpreted code, that is slower than the increment done inside the Python 
library - that's C code, running natively. Even without the speed loss - 
the 'for' idiom is very clear and compact, and the xrange solution is a 
elegant way to keep the idiom and provide good performance.

I hope that this does clarify things for you...


Carlos Ribeiro