Why doesn't eval of generator expression work with locals?

Gabriel Genellina gagsl-py2 at yahoo.com.ar
Thu Jan 29 16:51:52 EST 2009


En Thu, 29 Jan 2009 05:14:41 -0200, Hendrik van Rooyen  
<mail at microcorp.co.za> escribió:
> "Gabriel Genellina" <gagsl-py2 at yahoo.com.ar> wrote:
> En Wed, 28 Jan 2009 16:00:43 -0200, Scott David Daniels

>> (I *think* this has to do with free variables in the "right side" (after
>> the "in" keyword) of a generator expression; they appear to be evaluated
>> when the expression is *defined*, not when is is *used*. By contrast,  
>> free
>> variables in the "left side" appear to be evaluated when the expression  
>> is
>> used.)
>>
>
> Yikes! this is, IMO, far too complicated a way of looking at it.

Sure? "Make everything as simple as possible, but not simpler." (att. A.  
Einstein). I cannot find a simpler way to explain it that is still true...

> I think that if you rewrite the "comprehensions" as loops,
> you will see what is happening. - in the one case, the locals()
> returned is from the outer scope, while in the other the
> locals function is called from inside the loop - a different scope,
> because there must be *somewhere* where the state is kept to yield
> the next value.

But a loop doesn't define a new scope (only "def" and "class" used to  
define one; now generator expressions do too). The new scope is not the  
issue, but the fact that the right and left parts of a gen.expr. are  
evaluated at different times. This wasn't obvious to me -- and still  
isn't. If you rewrite the generator expression as a generator function you  
don't get the same behaviour:

print "genexpr"
A = [1,2,3]
B = 1
g = (x+B for x in A)
A = [4,5,6]
B = 10
print list(g)
# output: [11,12,13]
# A is evaluated at the time g is *defined*
# B is evaluated at the time g is *iterated*

print "genfunc"
A = [1,2,3]
B = 1
def f():
   for x in A:
     yield x+B
A = [4,5,6]
B = 10
g = f()
A = [7,8,9]
B = 100
print list(g)
# output: [107,108,109]
# A and B are evaluated at the time g is *iterated*

Of course this is clearly stated in the Language Reference "Variables used  
in the generator expression are evaluated lazily in a separate scope when  
the next() method is called for the generator object (in the same fashion  
as for normal generators). However, the in expression of the leftmost for  
clause is immediately evaluated in the current scope..." -- but this  
behaviour is still surprising and not obvious to me. ("not obvious" means  
that things could have been different, choosing this was a design  
decision).

As far as I can tell, this is the only case in Python where parts of the  
same expression are evaluated in different contexts at different times.

> I find it very confusing because locals is always the same
> function - but what it returns varies depending on where you
> call it from. So if you want to pass it around, it's no good
> passing the function - you have to pass the returned result,
> *called from the scope of interest*.  Subtle stuff.

Yep, the fact that locals() returns a different thing depending on  
when/where you call it, was the initial clue to this issue.

-- 
Gabriel Genellina




More information about the Python-list mailing list