inverse of izip
sharidas at zeomega.com
Fri Aug 20 06:55:44 CEST 2004
Thanks for the explanation. I understand izip a little better now.
Steven Bethard wrote:
>Satchidanand Haridas <sharidas at zeomega.com> wrote in message news:<mailman.1941.1092909660.5135.python-list at python.org>...
>>Could you expand on what you mean by exhaust the iterators too early?
>>The reason I ask is that the * operator is applied to
>>((1,1),(2,2),....(9,9)). The operation of the
>>itertools.izip(range10(),range10()) is completed before the * operation
>>is applied. And the iter() simply converts the result of the inverse
>>izip operation into an iterator. I hope the above was not too
>Yeah, the difference is a little subtle here. What we have before you
>use the * operator is an iterator that will yield (1,1) then (2,2) up
>to (9,9). Note that we don't actually have the tuple
>((1,1),(2,2),....(9,9)) yet, just an iterator that will produce the
>same elements. If your list is very large and you don't want to keep
>it all in memory at once, it's crucial that we have the iterator here,
>not the tuple.
>When you use the * operator, Python converts the iterable following
>the * into the argument list of the function. This means that if
>you're using an iterable, it reads all of the elements of the iterable
>into memory at once. That's why my range10 iterators printed
>"exhausted" after the * application -- all their elements had been
>read into memory. Again, if your list is very large, this is a bad
>thing because you now have all the elements of the list in memory at
>the same time. My other solution (well, Peter Otten's correction of
>my solution) never has the whole list in memory at the same time --
>each time enumerate generates a tuple and it's index, each of the
>iterators returned by starzip generates their appropriate items.[*]
>[*] Of course, if you exhaust one of the iterators before the others,
>itertools.tee's implicit cache will actually store all the elements,
>so starzip would really only be efficient if you wanted to iterate
>through the sub-iterators in lockstep. This means you'd probably want
>to itertools.izip them back together at some point, but being able to
>starzip them means you can wrap the individual iterators with extra
>functionality if necessary.
More information about the Python-list