Python 2.0b1 List comprehensions are slow

[Tim Peters]

[Kevin O'Connor]
> ...
> Unfortunately, I'm a little concerned about the speed of the new list
> feature.  According to the web site, "[a comprehension] is more efficient
> than map() with a lambda."  However, this is not true in my test
> programs - they show comprehensions to be 50% slower than map with
> a lambda
>
> I hope this can be resolved before 2.0 is released.  :-)

It's unlikely anything will change here before 2.1, because betas are for
bug-fixing and this isn't a bug (unless we want to call it a bug in the
docs!).

You may enjoy <wink> puzzling over timings when using this scenario instead
(just a sketch):

N = 100000

from random import random as pick
points = [(pick(), pick(), pick()) for i in xrange(N)]

def one():
    from math import sqrt
    return [sqrt(x**2 + y**2 + z**2) for x, y, z in points]

def two():
    from math import sqrt
    return map(lambda (x, y, z), sqrt=sqrt: sqrt(x**2 + y**2 + z**2),
                                            points)

If your machine is anything like mine, these run at virtually the same
speed.  That's actually what I expected "on general principle":

1. Map has to invoke a lambda for each element, but saves all but
   one <wink> trip around the eval loop and 100 thousand appends
   ("map" preallocates the result list, and uses an indexed store
   to record each result -- cheaper).

2. Listcomps save all the lambda invocations, but make many trips
   around the eval loop for each element generated, and makes a call
   to append for each element.

lambdas aren't cheap in Python, but neither are making 100 thousand append
calls and many 100s of thousands of trips around the eval loop.  So "should
be more-or-less close to a wash".

Then why did you see such a large difference?  Primarily because you're
picking on operator.* functions.  You could spend a whole day fully
understanding the difference in speed between

    operator.add(i, j)

and

    i + j

in *isolation*.  The primitive binary operators are optimized in
supernatural ways throughout the implementation, and operator.add doesn't
always take the same path thru the internals as infix "+".  They're not
really the same!  Not under the covers.  Although they should have the same
visible behavior.

use-it-for-clarity-speed-may-come-later-but-even-if-it-never-does-
    clarity-is-worth-more-to-me-ly y'rs  - tim