revive a generator

[Carl Banks]

On Thursday, October 20, 2011 6:23:50 AM UTC-7, Yingjie Lan wrote:
> Hi,
> 
> it seems a generator expression can be used only once:
> 
> >>> g = (x*x for x in range(3))
> >>> for x in g: print x
> 0
> 1
> 4
> >>> for x in g: print x #nothing printed
> >>>
> 
> Is there any way to revive g here?

Revive is the wrong word for what you want.  Once an iterator (be it a generator or some other kind of iterator) is done, it's done.  What you are asking for is, given a generator, to create a new generator from the same expression/function that created the original generator.  This is not reviving, but recreating.

I have two objections to this: a major ideological one and a minor practical one.

The practical drawback to allowing generators to be recreated is that it forces all generators to carry around a reference to the code object that created it.

    if random.random() > 5:
        g = (x*x for x in xrange(3))
    else:
        g = (x+x for x in xrange(3))
    for y in g:
        print x
    revive(g)  # which generator expression was it?
               # need to carry around a reference to be able to tell
    for y in g:
        print x

Carrying a reference to a code object in turn carries around any closures used in the generator expression or function, so it can potentially keep a large amount of data alive.  Given that the vast majority of generators would never be recreated, this is quite wasteful.

My ideological objection is that it forces the programmer to be wary of the effects of recreation.  Right now, if someone writes a generator expression, they can rely on the fact that it can only be iterated through once (per time the generator expression is evaluated).  But if you allow a downstream user to recreate the generator at will, then the writer will always have to be wary of adverse side-effects if the generator is iterated through twice.

So, although I can see it being occasionally useful, I'm going to opine that it is more trouble than it's worth.


Carl Banks