Real-world use cases for map's None fill-in feature?

Peter Otten __peter__ at
Tue Jan 10 04:16:04 EST 2006

Raymond Hettinger wrote:

> Alternately, the need can be met with existing tools by pre-padding the
> iterator with enough extra values to fill any holes:
>     it = chain(iterable, repeat('', group_size-1))
>     result = izip_longest(*[it]*group_size)
> Both approaches require a certain meaure of inventiveness, rely on
> advacned tricks, and forgo readability to gain the raw speed and
> conciseness afforded by a clever use of itertools.  They are also a
> challenge to review, test, modify, read, or explain to others.

Is this the author of itertools becoming its most articulate opponent? What
use is this collection of small functions sharing an underlying concept if
you are not supposed to combine them to your heart's content? You probably
cannot pull off some of those tricks until you have good working knowledge
of the iterator protocol, but that is becoming increasingly important to
understand all Python code.

> In contrast, a simple generator is trivially easy to create and read,
> albiet less concise and not as speedy:
>     it = iter(iterable)
>     while 1:
>         row = tuple(islice(it, group_size))
>         if len(row) == group_size:
>             yield row
>         else:
              if row:
                  yield row + ('',) * (group_size - len(row))
>             break
> The generator version is plain, simple, boring, and uninspirational.

I Can't argue with that :-) But nobody spotted the bug within a day; so
dumbing down the code didn't pay off. Furthermore, simple code like above
is often inlined and therefore harder to test and an impediment to
modification. Once you put the logic into a separate function/generator it
doesn't really matter which version you use. You can't get the
chain/repeat/izip variant to meet your (changing) requirements? Throw it
away and just keep the (modified) test suite.

A newbie, by the way, would have /written/ neither. The it = iter(iterable)
voodoo isn't obvious and the barrier to switch from lst[:group_size] to
islice(it, group_size) to /improve/ one's is code high. I expect to see an
inlined list-based solution. The two versions are both part of a learning
experience and both worth the effort.

Regarding the thread's topic, I have no use cases for a map(None, ...)-like
izip_longest(), but occasionally I would prefer izip() to throw a
ValueError if its iterable arguments do not have the same "length".


More information about the Python-list mailing list