Style in list comprehensions

[Alex Martelli]

Dave Kuhlman wrote:
   ...
>     def func1(x):
>         return x*3
>     def t1(lst):
>         return [func1(x) for x in lst]
>     def t2(lst):
>         result = []
>         for x in lst:
>             result.append(func1(x))
>         return result

With Python 2.3's timeit.py, I think the performance differences are
more easily and reliably measured -- both those stemming directly from
using list comprehensions rather than loops of .append calls, and the
ones often enabled by a list comprehension's better suitability for
"inlining" whatever operations you're performing.  To wit:

bash-2.05b$ cd /usr/lib/python2.3
bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
> 'result=[]' 'for x in range(10000): result.append(f1(x))'
10 loops, best of 3: 3.7e+04 usec per loop

bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
> 'result=[f1(x) for x in range(10000)]'
10 loops, best of 3: 2.94e+04 usec per loop

bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
> 'result=[x*3 for x in range(10000)]'
100 loops, best of 3: 1.79e+04 usec per loop

In some (rare) bottleneck situations, these performance differences
might perhaps be significant.  Similarly for good old map...:

bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
> 'result=map(f1, range(10000))'
100 loops, best of 3: 1.87e+04 usec per loop

bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
> 'result=map(lambda x:x*3, range(10000))'
100 loops, best of 3: 1.84e+04 usec per loop

No "inlining" (as you can see, pseudo-inlining via lambda does
not buy you any performance advantage), but if you're stuck in a
situation where you just can't inline map may well be faster than
list comprehensions -- perhaps significantly so, if this happens
to fall right on a performance bottleneck of your program.

Finally, in some cases, you can "preallocate" the result list to
the length it will have in the end, and then rebind its items one
by one, rather than "growing" the result list as you go.  When
this is feasible, it's often the fastest approach you can possibly
use, to wit:

bash-2.05b$ python2.3 timeit.py -s'result=10000*[None]' \
> 'for i in range(10000): result[i] = i*3'
100 loops, best of 3: 9.75e+03 usec per loop

...almost twice as fast than the best map or list comprehension,
in this specially favourable case.

Of course, talking of specially favourable cases, when some time
consuming operation DOES represent such a case one might be well
advised to keep an eye out for totally different possibilities...:

bash-2.05b$ python2.3 timeit.py 'result=range(0,30000,3)'
100 loops, best of 3: 2.22e+03 usec per loop

Yes, this IS "cheating" -- but it does produce the same 'result'
list as each of the other approaches, and it does so 4 or 5 times
faster than the second-best approach (preallocated list, taking
some advantage of this being a special case), almost 10 times
faster than the best "generalized" approach.  Any time you need
to produce any kind of arithmetic progression, do consider using
just a special-purpose 'range' call, if you care about speed!-)


Alex