[Python-Dev] accumulator display syntax

[Tim Peters]

[Tim]
>    squares = (f(x)**2 for x in inputs)  # assuming reiterability here
> ...
>    for f in math.sin, math.cos, math.tan:
>        plot(squares)

[Skip Montanaro]
> How much more expensive

Stop right there.  I must have been unclear.  The only point of the example
was semantic, not cost:  even if generator expressions used closure
semantics, the example *still* wouldn't work the way it appears to read, and
because generator expressions aren't reiterable.  What the example would do
under closure semantics:

1. Plot the square of math.sin(x), for each x in inputs.

then

2. Probably nothing more than that.  The "squares" GE is exhausted after
   #1 completes, and no matter often it's run again it's simply going to
   raise StopIteration at once each time it's tried.  A reasonable plot()
   would probably do nothing when fed an exhausted iterable, but maybe it
   would raise an exception.  That's up to plot().  What it *won't* do
   under any scheme here is go on to plot the squares of math.cos(x) and
   math.tan(x) over the inputs too.

The lack of reiterability (which is fine by me!) thus seems to make a
plausible use for closure semantics hard to imagine.  The example was one
where closure semantics suck despite misleading appearance.

Closures are very often used (in languages other than Python, and in Python
too by people who haven't yet learned to write Python <0.9 wink>) to hold
mutable state in outer scopes, for the use of functions in inner scopes,
very much like an instance's data attributes hold mutable state for the use
of methods defined in the instance's class.  In those common cases, the
power comes from being able to run functions (methods) more than once, or to
reuse the mutable state among functions (methods).  But generator
expressions are always one-shot computations (you get to run a GE to
completion no more than once).  There may be some use for closure semantics
in a collection of GEs that reference each other (similar to instance data
being visible to multiple methods), but so far I've failed to dream up a
plausible case of that nature either.

> would this be than
>
>     for f in math.sin, math.cos, math.tan:
>         squares = (f(x)**2 for x in inputs)
>         plot(squares)

Despite the similar appearance, that does something very different, plotting
all 3 functions (not just math.sin), and regardless of whether closure or
capture semantics are used.  I expect the body of the loop in real life
would be a one-liner, though:

        plot(f(x)**2 for x in inputs)

> which would work without reiterability, right?

Yup.

> The underlying generator function could still be created at compile-time
> and it (or its code object?) stored in the current function's constants.
> 'f' is simply an argument to it when the iterator is instantiated.

Guido expanded on that already.  The code is compiled only once (at "compile
time"), and there's a small runtime cost per outer-loop iteration to build a
function object from the (pre-compiled) code object, and a possibly larger
runtime cost per outer-loop iteration to start the GE.  Passing 'f' and
'inputs' may be part of either of those costs, depending on how it's
implemented -- but giving the synthesized generator function some
initialized locals is the least of the runtime costs.