[Python-Dev] [872326] generator expression implementation

[Jeremy Hylton]

On Mon, 2004-01-12 at 21:53, jiwon wrote:
> I'm the originator of the patch (SF patch 872326 - generator expression).
> If "immediate evaluation of expr in generator expression" means that
> generator expression:
> 
> (x for x in range())
> 
> should be equivalent to:
> 
> def __foo(_range=range()):
>     for x in _range:
>         yield x
> 
> it seems to me that generator expression has almost no merit over list
> comprehension.
> I think the merit of generator is that it does not need to allocate all the
> memory that it needs in the initial time, right?
> Or, am I confusing something? :)

The range() case isn't all the interesting, because you wouldn't delay
the memory allocation very long.  The first time an element was consumed
from the g.expr it would materialize the whole list.  So you would only
delay for the likely brief time between creating the generator and
consuming its first element.

There are two more common cases.  One is that the expression is already
a fully materialized sequence.  Most of the examples in PEP 289 fall
into this category:  You've got a list of objects and you want to sum
their spam_score attributes.  The other case is that the expression is
itself a generator, e.g.

    dotproduct = sum(x*y for x,y in itertools.izip(x_vector, y_vector))

For all these cases, the benefit is that the result list is never
materialized, not that the input list is never materialized.

Jeremy