# [Python-ideas] free variables in generator expressions

Arnaud Delobelle arno at marooned.org.uk
Thu Dec 13 00:01:42 CET 2007

```On 12 Dec 2007, at 21:56, Georg Brandl wrote:

> Brett Cannon schrieb:
[...]
>
>>
>> Consider what your genexp, ``(x for x in P if x % p)``, really is::
>>
>>  def _genexp():
>>      for x in P:
>>        if x % p:
>>          yield x
>
> Actually it is
>
> def _genexp(P):
>    for x in P:
>        if x % p:
>            yield x
>
> IOW, the outmost iterator is not a free variable, but passed to the
> invisible function object.

I see. 'P' gets frozen but not 'p', so I should be able to write:

def gen2_sieve(n):
"Generate all primes less than n"
P =  xrange(2, n)
while True:
for p in P:
yield p
P = (lambda p: (x for x in P if x % p))(p)
break
else:
return

>>> list(gen2_sieve(20))
[2, 3, 5, 7, 11, 13, 17, 19]

It seems to work.  Ok then in the same vein, I imagine that

(x + y for x in A for y in B)

becomes:

def _genexp(A):
for x in A:
for y in B:
yield x + y

Let's test this (python 2.5):

>>> A = '12'
>>> B = 'ab'
>>> gen = (x + y for x in A for y in B)
>>> A = '34'
>>> B = 'cd'
>>> list(gen)
['1c', '1d', '2c', '2d']

So in the generator expression, A is remains bound to the string '12'
but B gets rebound to 'cd'.  This may make the implementation of
generator expressions more straighforward, but from the point of view
of a user of the language it seems rather arbitrary. What makes A so
special as opposed to B?  Ok it belongs to the outermost loop, but
conceptually in the example above there is no outermost loop.

At the moment I still think it makes more sense for the generator
expressions to generate as much as possible a sequence which is the
same as what the corresponding list comprehension would have been,
i.e.:

l = [f(x, y) for x in A for y in B(x) if g(x, y)]
g = [f(x, y) for x in A for y in B(x) if g(x, y)]
<code, maybe binding A, B, f, g to new objects>
assert list(g) == l

to work as much as possible.

Perhaps I should go and see how generator expressions are generated in
the python source code.

--
Arnaud

```