Generator oddity

[Steven D'Aprano]

On Fri, 01 May 2009 04:58:16 -0700, opstad wrote:

> I'm a little baffled by the inconsistency here. Anyone have any
> explanations?
> 
>>>> def gen():
> ...   yield 'a'
> ...   yield 'b'
> ...   yield 'c'
> ...
>>>> [c1 + c2 for c1 in gen() for c2 in gen()]
> ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
> 
>>>> list(c1 + c2 for c1 in gen() for c2 in gen())
> ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
> 
>>>> it1 = gen()
>>>> it2 = gen()
>>>> list(c1 + c2 for c1 in it1 for c2 in it2)
> ['aa', 'ab', 'ac']
> 
> Why does this last list only have three elements instead of nine?


Ah, good one! That had me puzzled for a while too.

The answer is to write it out as a nested for-loop, using print in place 
of yield. Here's the first way:

for c1 in gen():
    for c2 in gen():
        print c1 + c2

And the second:

it1 = gen()
it2 = gen()
for c1 in it1:
    for c2 in it2:
        print c1 + c2


In the first example, the inner loop gets refreshed each time through the 
outer loop with a brand new instance of the generator. Expanding the 
loops in full:

# First method:
c1 = 'a'
call gen() to make an iterable
step through the fresh iterable, giving 'aa' 'ab' 'ac'
c1 = 'b'
call gen() to make an iterable
step through the fresh iterable, giving 'ba' 'bb' 'bc'
c1 = 'c'
call gen() to make an iterable
step through the fresh iterable, giving 'ca' 'cb' 'cc'


# Second method:
c1 = 'a'
inner iterable already exists
step through the iterable, giving 'aa' 'ab' 'ac'
c1 = 'b'
inner iterable is exhausted, so do nothing
c1 = 'c'
inner iterable is exhausted, so do nothing


And there you have it. A nice Gotcha for the books.



-- 
Steven