itertools.izip brokeness

[Paul Rubin]

bonono at gmail.com writes:
> But that is exactly the behaviour of python iterator, I don't see what
> is broken.

What's broken is the iterator interface is insufficient to deal with
this cleanly.

> And because python iterator can only go in one direction, those
> consumed do lose in the zip/izip calls.

Yes, that's the problem.  It's proven useful for i/o streams to support
a pushback operation like ungetc.  Maybe something like it can be done
for iterators.

> I think you need to use map(None,...) which would not drop anything,
> just None filled. Though you don't have a relatively lazy version as
> imap(None,...) doesn't behave like map but a bit like zip.

I don't understand what you mean by this?  None is not callable.

How about this (untested):

  def myzip(iterlist):
    """return zip of smaller and smaller list of iterables as the
individual iterators run out"""
    sentinel = object()  # unique sentinel
    def sentinel_append(iterable):
       return itertools.chain(iterable, itertools.repeat(sentinel))
    for i in itertools.izip(map(sentinel_append, iterlist)):
       r = [x for x in i.next() if x is not sentinel]
       if r: yield r
       else: break