[Python-3000] map() Returns Iterator

[Kurt B. Kaiser]

"Guido van Rossum" <guido at python.org> writes:

>> I think of map() and filter() as sequence transformers.  To me, it's
>> an unexpected semantic change that the result is no longer a list.
>
> Well, enough people thought of them as iteratables to request imap(),
> ifilter() and izip() added to the itertools library.

Agreed.  When processing (possibly very long) streams, the lazy versions
have great advantages.

3>> def ones():
        while True:
                yield 1

3>> a = map(lambda x: x+x, ones())
3>> b = map(lambda x: x+1, a)
3>> b.__next__()
3

If you try that in 2.6 you fill memory.

However, IMHO eliminating the strict versions of map() and filter() in
favor of the lazy versions from itertools kicks the degree of
sophistication necessary to understand these functions up a notch (or
three).

3>> c, d = map(int, ('1', '2'))
3>> c, d
(1, 2)
3>> e = map(int, ('1', '2'))
3>> f, g = e
3>> f, g
(1, 2)
3>> f, g = e
Traceback (most recent call last):
  File "<pyshell#15>", line 1, in <module>
    f, g = e
ValueError: need more than 0 values to unpack


To say nothing of remembering having to use

3>> foo = (list(map(bar)))

most the time.  I'd say keep map(), filter(), imap() and ifilter(), and
use the latter when you're working with streams.

"Explicit is better than implicit."

Then there's the silent failure to process the side-effects of

3>> map(print, lines)

which is rather unexpected.  To me, this code is quite readable and not
at all pathological (no more than any print statement :).  It may not be
Pythonic in the modern idiom (that pattern is found mostly in code.py
and IDLE, and it's very rare), but it's legal and it's a little
surprising that it's necessary to spell it

3>> list(map(print, lines)) 

now to get any action.  It took me awhile to track down the failures in
the interactive interpreter emulator because that pattern was being used
to print the exceptions; the thing just produced no output at all.

The alternatives

3>> print('\n'.join(lines))

or

3>> (print(line) for line in lines)  # oops, nothing happened
3>> [print(line) for line in lines]

aren't much of an improvement.

>> In existing Lib/ code, it's twice as likely that the result of map()
>> will be assigned than to use it as an iterator in a flow control
>> statement.
>
> Did you take into account the number of calls to imap()?

No.  Since the py3k branch is partially converted, I went back to 2.6,
where skipping Lib/test/, there are (approximately!!):

87  assignments of the output of map(), passing a list
21  assignments involving map(), but not directly.  Many of these involve
    'tuple' or 'join' and could accept an iterator.
58  return statements involving map() (39 directly)
1   use to construct a list used as an argument
2   for ... in map()   (!!)   and 1 for ... in enumerate(map(...))
1   use as map(foo, bar) == baz_list
5   uses of imap()

[...]

> We didn't write the 2to3 transform, but it's easier than some others
> we already did (e.g. keys()).

I see a transform in svn.  As an aside, is there any accepted process
for running these transforms over the p3yk branch?  Some parts of Lib/
are converted, possibly by hand, possibly by 2to3, and other parts are
not.

[...]

> When you have a lambda as the first argument the better translation is
> *definitely* a list comprehension, as it saves creating stack frames
> for the lambda calls.

Thanks, good tip.

[...]

> Also, have you ever liked the behavior that filter returns a string if
> the input is a string, a tuple if the input is a tuple, and a list for
> all other cases? That really sucks IMO.

If you look at map() and filter() as sequence transformers, that makes
some sense: preserve the type of the sequence if possible.  But clearly
map() and filter() should act the same way!

>> zip() is infrequently used.
>
> It was especially designed for use in for-loops (to end the fruitless
> discussions trying to come up with parallel iteration syntax). If it
> wasn't for the fact that iterators hadn't been invented yet at the
> time, zip() would definitely have returned an iterator right from the
> start, just as enumerate().
>
>> However, IMO for consistency they should all act the same way.
>
> That's not much of consistency.

It's used only fifteen times in 2.6 Lib/ and four of those are
izip(). Eight are assignments, mostly to build dicts. Six are in
for-loops. One is a return.

>> Could something be done in the ast phase of compilation instead?
>
> Not likely, the compiler doesn't know enough about the state of builtins.

OK, thanks for the reply.

-- 
KBK