[Python-3000] map() Returns Iterator

[Guido van Rossum]

On 8/6/07, Kurt B. Kaiser <kbk at shore.net> wrote:
> "Guido van Rossum" <guido at python.org> writes:
[...pushback...]

> However, IMHO eliminating the strict versions of map() and filter() in
> favor of the lazy versions from itertools kicks the degree of
> sophistication necessary to understand these functions up a notch (or
> three).

I wonder how bad this is given that range() and dict.keys() and
friends will also stop returning lists? I don't think you ever saw any
of my Py3k presentations (the slides of the latest one at here:
http://conferences.oreillynet.com/presentations/os2007/os_vanrossum.ppt).
I've always made a point of suggesting that we're switching to
returning iterators instead of lists from as many APIs as makes sense
(I stop at str.split() though, as I can't think of a use case where
the list would be so big as to be bothersome).

> To say nothing of remembering having to use
>
> 3>> foo = (list(map(bar)))
>
> most the time.

I think you're overreacting due to your experience with conversion of
existing code. I expect that new use cases where a list is needed will
generally be written using list comprehensions in Py3k-specific
tutorials, and generator expressions for situations where a list isn't
needed (as a slightly more advanced feature). Then map() and filter()
can be shown as more advanced optimizations of certain end cases.

> I'd say keep map(), filter(), imap() and ifilter(), and
> use the latter when you're working with streams.
>
> "Explicit is better than implicit."
>
> Then there's the silent failure to process the side-effects of
>
> 3>> map(print, lines)
>
> which is rather unexpected.  To me, this code is quite readable and not
> at all pathological (no more than any print statement :).  It may not be
> Pythonic in the modern idiom (that pattern is found mostly in code.py
> and IDLE, and it's very rare), but it's legal and it's a little
> surprising that it's necessary to spell it
>
> 3>> list(map(print, lines))
>
> now to get any action.

Aren't you a little too fond of this idiom? I've always found it a
little surprising when I encountered it, and replaced it with the more
straightforward

 for line in lines:
    print(line)

> It took me awhile to track down the failures in
> the interactive interpreter emulator because that pattern was being used
> to print the exceptions; the thing just produced no output at all.

I think that's just a side effect of the conversion. I take it you
didn't use 2to3?

> The alternatives
>
> 3>> print('\n'.join(lines))
>
> or
>
> 3>> (print(line) for line in lines)  # oops, nothing happened
> 3>> [print(line) for line in lines]
>
> aren't much of an improvement.

Well duh. Really. What's wrong with writing it as a plain old for-loop?

> >> In existing Lib/ code, it's twice as likely that the result of map()
> >> will be assigned than to use it as an iterator in a flow control
> >> statement.
> >
> > Did you take into account the number of calls to imap()?
>
> No.  Since the py3k branch is partially converted, I went back to 2.6,
> where skipping Lib/test/, there are (approximately!!):
>
> 87  assignments of the output of map(), passing a list
> 21  assignments involving map(), but not directly.  Many of these involve
>     'tuple' or 'join' and could accept an iterator.
> 58  return statements involving map() (39 directly)
> 1   use to construct a list used as an argument
> 2   for ... in map()   (!!)   and 1 for ... in enumerate(map(...))
> 1   use as map(foo, bar) == baz_list
> 5   uses of imap()

I'm not sure what the relevant of assignments is. I can assign an
iterator to a variable and do stuff with it and never require it to be
a list. I can also pass a map() call into a function and then it
depends on what the function does to that argument.

> [...]
>
> > We didn't write the 2to3 transform, but it's easier than some others
> > we already did (e.g. keys()).
>
> I see a transform in svn.

I guess I didn't look well enough.

> As an aside, is there any accepted process
> for running these transforms over the p3yk branch?  Some parts of Lib/
> are converted, possibly by hand, possibly by 2to3, and other parts are
> not.

(Aside: Please skip the p3yk branch and use the py3k-struni branch --
it's the way of the future.)

I tend to do manual conversion of the stdlib because it's on the
bleeding edge. At times I've regretted this, and gone back and run a
particular transform over some of the code. I rarely use the full set
of transforms on a whole subtree, although others sometimes do that.
Do note the options that help convert doctests and deal with print()
already being a function.

[zip()]
> It's used only fifteen times in 2.6 Lib/ and four of those are
> izip(). Eight are assignments, mostly to build dicts.

I don't understand. What's an "assignment" to build a dict? Do you
mean something like

  dict(zip(keys, values))

? That's an ideal use case for an iterator.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)