itertools.izip brokeness

Diez B. Roggisch deets at nospam.web.de
Tue Jan 3 08:52:26 EST 2006


> What's broken is the iterator interface is insufficient to deal with
> this cleanly.

I don't consider it broken. You just think too much in terms of the OPs
problems or probably other fields where the actual data is available for
"rewinding".

But as iterators serve as abstraction for lots of things - especially
generatiors - you can't enhance the interface.

> 
> Yes, that's the problem.  It's proven useful for i/o streams to support
> a pushback operation like ungetc.  Maybe something like it can be done
> for iterators.

No. If you want that, use

list(iterable)

Then you have random access. If you _know_ there will be only so much data
needed to "unget", write yourself a buffered iterator like this:

buffered(iterable, size)

Maybe something like that _could_ go in the itertools. But I'm not really
convinced, as it is too tied to special cases - and besides that very
easily done.


> How about this (untested):
> 
>   def myzip(iterlist):
>     """return zip of smaller and smaller list of iterables as the
> individual iterators run out"""
>     sentinel = object()  # unique sentinel
>     def sentinel_append(iterable):
>        return itertools.chain(iterable, itertools.repeat(sentinel))
>     for i in itertools.izip(map(sentinel_append, iterlist)):
>        r = [x for x in i.next() if x is not sentinel]
>        if r: yield r
>        else: break

If that fits your semantics - of course. But the general zip shouldn't
behave that way.

Regards,

Diez



More information about the Python-list mailing list