[Python-Dev] Seeming unintended difference between list comprehensions and generator expressions...

[Nick Coghlan]

Josiah Carlson wrote:
> The behavior of 3.0 WRT list comprehensions behaving the same way as
> generator expressions is expected, but why generator expressions
> (generally) don't keep a reference to the class scope during execution
> seems to be unintended.

It's intended. While arguably not ideal, it would require some pretty
major changes to the lexical scoping rules to make them behave any
differently.

The translation of (i*i for i in x) is conceptually along the lines of:

  def _ge(arg):
    for i in arg:
      yield i*i

  <expr_value> = _ge(x)

Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
translated as:

  def _lc(arg):
    result = []
    for i in arg:
      result.append(i*i)
    return result

  <expr_value> = _lc(x)

Like any function scope inside a class namespace, the body of a genexp
(and, in 3.x, comprehension) doesn't have direct access to the class
namespace because classes don't play any part in the lexical scoping rules.

Basically, if a generator or 3.x comprehension needs access to a value
from a containing class scope anywhere other than the outermost
iterator, then it needs to be put into a temporary function and given
the extra value as an argument:

.>> class C:
...   x = {}
...   def _init_constants(d, itr):
...     d.update((i, d.get(i, None)) for i in itr)
...   _init_constants(x, range(10))
...   del _init_constants
...
.>> C.x
{0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None,
8: None, 9: None}

(in this toy case, of course, it would be simpler to make the temporary
function create and return the constants dictionary, but the above
approach with multiple arguments being passed in applies more generally
when you need to access multiple existing values from the class scope)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------