[Tutor] list comprehension, efficiency?

[Steven D'Aprano]

On Tue, 28 Sep 2010 01:57:23 pm Bill Allen wrote:

> I can now see that quite a bit of the code I write dealing with lists
> can be done with list
> comprehensions.   My question is this, is the list comprehension
> styled code generally
> more efficient at runtime?  If so, why?


List comprehensions *are* lists, or rather, they produce lists.

A list comprehension 

result = [expr for x in seq] 

is just syntactic sugar for a for-loop:

result = []
for x in seq:
    result.append(x)

except that in Python 3, the "x" variable is hidden. (In Python 2 it is 
not, but that was an accident.) The end result is exactly the same 
whether you use a list comp or a for-loop.

The advantage of for-loops is that you can do much more complex code. 
That complexity comes at a cost, and consequently for-loops tend to be 
a little bit slower than list comprehensions: some of the work can be 
done under the hood, faster than regular Python code. But for most 
cases, this difference is relatively small and won't make any real 
difference. What does it matter if your program runs in 24 milliseconds 
instead of 22 milliseconds? Or five hours and seventeen minutes instead 
of five hours and sixteen minutes? Who cares? Write the code that is 
most natural and easy to read and understand, and only worry about such 
tiny savings when you absolutely have to.

But there is one case where for-loops are potentially MUCH faster than 
list comps. List comps always run all the way to the end, but for-loops 
can break out early. If your problem lets you break out of the loop 
early, this is a good thing. So this for-loop:


for x in xrange(10000000):
    result.append(x+1)
    if x > 5: break


will beat the pants off this list comp:

[x+1 for x in xrange(10000000) if x <= 5]

There's no way to break out of the list comp early -- it has to keep 
going past 5 all the way to the end.


-- 
Steven D'Aprano