map vs for (was RE: ANN: Gadfly bugfix test release)

Tim Peters tim_one at email.msn.com
Sun Aug 8 20:00:08 CEST 1999


[Aaron Watters, times some loops]

>     for i in range(len(range1)):
>         result[i] = (range1[i], 'x')
> [vs]
>     result = map(None, range1, ['x']*len(range1))

yielding

> testing loop
> elapsed 1.10000002384
>
> testing map
> elapsed 0.330000042915

and

>     for i in range(len(range1)):
>         result[i] = range1[i]-3
> [vs]
>     from operator import sub
>     result = map(sub, range1, [3]*len(range1))

yielding

> testing loop
> elapsed 0.439999938011
>
> testing map
> elapsed 0.330000042915

Then:

> Does this mean that map is always better than for
> loops even when it looks stupid?

It's complicated.  map has only one speed advantage all the time:  it does
the loop overhead at C speed, that is without the overhead of making trips
around the Python eval loop.  Beyond that, "it depends".

map (or filter) applied to None is supernaturally fast, as your first test
showed.  Apart from zippering sequences together (or filtering out false
elements) there's no use for it, though.

Anything other than that is case-dependent.  map (or filter) usually wins if
it can be applied to something from the operator module, as your second test
showed.  But map (or filter) usually loses if applied to a Python lambda (or
function) the guts of which can be written inline when using a for-loop; in
that case map saves a bit of loop overhead at the relatively enormous
expense of making Python-level function calls the for-loop avoids.

Another complication:  in your second test, it so happens that the Python
eval loop special-cases integer "-", doing it inline.  But operator.sub goes
thru a polymorphic __sub__ interface, making a method call for each subtract
(albeit a method call all at C speed).  So the relative advantage of using
map+operator.sub vs for+"integer -" is smaller than it is for many other
operators in the operator module.  So non-None operator.something map often
has a larger advantage than your second test showed.

And one more:  if the loop trip count gets large, building up a giant
vector(s) for map (or filter) to chew over can toast your cache and start
tickling your disk.  Then it's faster to write down your problem and mail it
to c.l.py, waiting for someone else to compute the answer <wink>.

the-only-killer-advantage-is-None-ly y'rs  - tim






More information about the Python-list mailing list