[Python-3000] map() Returns Iterator

[Guido van Rossum]

On 8/3/07, Kurt B. Kaiser <kbk at shore.net> wrote:
> Although there has been quite a bit of discussion on dropping reduce()
> and retaining map(), filter(), and zip(), there has been less discussion
> (at least that I can find) on changing them to return iterators instead
> of lists.

That's probably because over the years that this has been on my list
of things I'd change most people agreed silently.

> I think of map() and filter() as sequence transformers.  To me, it's
> an unexpected semantic change that the result is no longer a list.

Well, enough people thought of them as iteratables to request imap(),
ifilter() and izip() added to the itertools library.

> In existing Lib/ code, it's twice as likely that the result of map()
> will be assigned than to use it as an iterator in a flow control
> statement.

Did you take into account the number of calls to imap()?

> If the statistics on the usage of map() stay the same, 2/3 of the time
> the current implementation will require code like
>
>         foo = list(map(fcn, bar)).

And the 2to3 tool should make this transformation (unless it can tell
from context that it's unnecessary, e.g. in a for-loop, or in a call
to list(), tuple() or sorted().

We didn't write the 2to3 transform, but it's easier than some others
we already did (e.g. keys()).

> map() and filter() were retained primarily because they can produce
> more compact and readable code when used correctly.  Adding list() most
> of the time seems to diminish this benefit, especially when combined with
> a lambda as the first arg.

When you have a lambda as the first argument the better translation is
*definitely* a list comprehension, as it saves creating stack frames
for the lambda calls.

> There are a number of instances where map() is called for its side
> effect, e.g.
>
>         map(print, line_sequence)
>
> with the return result ignored.

I'd just call that bad style.

> In py3k this has caused many silent
> failures.  We've been weeding these out, and there are only a couple
> left, but there are no doubt many more in 3rd party code.

And that's why the 2to3 tool needs improvement.

> The situation with filter() is similar, though it's not used purely
> for side effects.

Same reply though.

Also, have you ever liked the behavior that filter returns a string if
the input is a string, a tuple if the input is a tuple, and a list for
all other cases? That really sucks IMO.

> zip() is infrequently used.

It was especially designed for use in for-loops (to end the fruitless
discussions trying to come up with parallel iteration syntax). If it
wasn't for the fact that iterators hadn't been invented yet at the
time, zip() would definitely have returned an iterator right from the
start, just as enumerate().

> However, IMO for consistency they should all act the same way.

That's not much of consistency.

> I've seen GvR slides suggesting replacing map() et. al. with list
> comprehensions, but never with generator expressions.

Depends purely on context. Also, it's easier to talk about "list
comprehensions" than about "list comprehensions or generator
expressions" all the time, so I may have abbreviated my suggestions
occasionally.

> PEP 3100: "Make built-ins return an iterator where appropriate
> (e.g. range(), zip(), map(), filter(), etc.)"
>
> It makes sense for range() to return an iterator.  I have my doubts on
> map(), filter(), and zip().  Having them return iterators seems to
> be a premature optimization.  Could something be done in the ast phase
> of compilation instead?

Not likely, the compiler doesn't know enough about the state of builtins.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)