[Python-Dev] accumulator display syntax

[Guido van Rossum]

> > > Actually, I consider Samuele's example a good argument in *favor* of
> > > the idea.  Because of the similarity between listcomps and generator
> > > expressions (gen-X's? ;) ) it seems late binding of locals would
> > > lead to people thinking the behavior is a bug.  Since a genex is not
> > > a function (at least in form) a late binding would be very
> > > non-obvious and counterintuitive relative to other kinds of
> > > expressions.
> >
> >Hm.  We do late binding of globals.  Why shouldn't we do late binding
> >of locals?
> 
> Wha?  Oh, you mean in a function.

No, everywhere.  Global in generator expressions also have late
binding:

   A = 1
   def f():
       return (x+A for x in range(3))
   g = f()
   A = 2
   print list(g)   # prints [2, 3, 4]; not [1, 2, 3]

> But that's what I'm saying, it's *not* a 
> function.  Sure, it's implemented as one under the hood, but it doesn't 
> *look* like a function.  In any normal (non-lambda) expression, whether a 
> variable is local or global, its value is retrieved immediately.

That's because the expression is evaluated immediately.  When passing
generator expressions around that reference free variables (whether
global or from a function scope), the expression is evaluated when it
is requested.  Note that even under your model,

  A = []
  g = (A for x in range(3))
  A.append(42)
  print list(g)    # prints [[42], [42], [42]]

> Also, even though there's a function under the hood, that function
> is *called* and its value returned immediately.  This seems
> consistent with an immediate binding of parameters.

But it's a generator function, and the call suspends immediately, and
continues to execute only when the next() method on the result is
called.

> >   There are lots of corners or the language where if you
> >expect something else the actual behavior feels like a bug, until
> >someone explains it to you.  That's no reason to compromise.  It's an
> >opportunity for education about scopes!
> 
> So far, I haven't seen you say any reason why the "arguments"
> approach is bad, or why the "closure" approach is good.  Both are
> certainly Pythonic in some circumstances, but why do you feel that
> one is better than the other, here?

Unified semantic principles.  I want to be able to explain generator
expressions as a shorthand for defining and calling generator
functions.  Invoking default argument semantics makes the explanation
less clean: we would have to go through the trouble of finding all
references to fere variables.  Do you want globals to be passed via
default arguments as well?  And what about builtins?  (Note that the
compiler currently doesn't know the difference.)

> I will state one pragmatic reason for using the default arguments
> approach: code converted from using a listcomp to a genex can
> immediately have bugs as a result of rebinding a local.  Those bugs
> won't happen if rebinding the local has no effect on the genex's
> evaluation.  (Obviously, an aliasing problem can still be created if
> one modifies a mutable used in the genex, but there's no way to
> remove that possibility and still end up with a lazy iterator.)
> 
> Given that one of the big arguments in favor of genexes is to make 
> "upgrading" from listcomps easy, it shouldn't fail so quickly and 
> obviously.  E.g., converting from:
> 
> x = {}
> for i in range(10):
>      x[i] = [y^i for y in range(10)]
> 
> to:
> 
> x = {}
> for i in range(10):
>      x[i] = (y^i for y in range(10))
> 
> Shouldn't result in all of x's elements iterating over the same values!

Hm.  I think most generator expressions should be finished before
moving on to the next line, as in

  for n in range(4):
    print sum(x**n for x in range(1, 11))

Saving a generator expression for later use should be something you
rarely do, and you should really think of it as a shorthand for a
generator function just as lambda is a shorthand for a regular
function.

--Guido van Rossum (home page: http://www.python.org/~guido/)