
[Chris Angelico <rosuav@gmail.com>]
... You're correct. The genexp is approximately equivalent to:
def genexp(): for p in small_primes: thisp = p yield n % thisp == 0 while any(genexp()): n //= thisp
With generator expressions, since they won't necessarily be iterated over immediately, I think it's correct to create an actual nested function; you need the effects of closures. With comprehensions, it's less obvious, and what you're asking for might be more plausible. The question is, how important is the parallel between list(x for x in iter) and [x for x in iter] ? Guido likes it, and to create that parallel, list comps MUST be in their own functions too.
I don't care how they're implemented here; I only care here about the visible semantics.
I have a long history of arguing that magically created lexically nested anonymous functions try too hard to behave exactly like explicitly typed lexically nested functions, but that's the trendy thing to do so I always lose ;-) The problem: in a magically created nested function, you have no possibility to say _anything_ about scope; at least when you type it by hand, you can add `global` and/or `nonlocal` declarations to more-or-less say what you want.
That's a fair point. But there is another equally valid use-case for assignment expressions inside list comps:
values = [y + 2 for x in iter if (y := f(x)) > 0]
In this case, it's just as obvious that the name 'y' should be local to the comprehension, as 'x' is.
There's a difference, though: if `y` "leaks", BFD. Who cares? ;-) If `y` remains inaccessible, there's no way around that.
Since there's no way to declare "nonlocal y" inside the comprehension, you're left with a small handful of options:
i leapt straight to #3:
1) All names inside list comprehensions are common with their surrounding scope. The comprehension isn't inside a function, the iteration variable leaks, you can share names easily. Or if it *is* inside a function, all its names are implicitly "nonlocal" (in which case there's not much point having the function).
DOA. Breaks old code.
2) All names are local to their own scope. No names leak, and that includes names made with ":=".
Saying "local to their own scope" _assumes_ what you're trying to argue _for_ - it's circular. In fact it's impossible to know what the user intends the scope to be.
3) Some sort of rule like "iteration variables don't leak, but those used with := are implicitly nonlocal".
Explicitly, because "LHS inherits scope from its context" (whether global, nonlocal, or local) is part of what ":=" is defined to _mean_ then.
Would create odd edge cases eg [x for x in iter if x := x] and that would probably result in x leaking.
Don't care.
4) A special adornment on local names if you don't want them to leak
5) A special adornment on local names if you DO want them to leak
Probably also DOA.
6) A combination of #3 and #4: "x := expr" will be nonlocal, ".x := expr" will be local, "for x in iter" will be local. Backward compatible but a pain to explain.
Definitely DOA. ...
Since there's no way to explicitly identify the desired scope, I suggest that ":=" inside magically created nested functions do the more-useful-more-often thing: treat the name being bound as if the binding had been spelled in its enclosing context instead. So, in the above, if `thisp` was declared `global`, also `global` in the genexp; if `nonlocal`, also `nonlocal`; else (almost always the case in real life) local to the containing code (meaning it would be local to the containing code, but nonlocal in the generated function).
Is it really more useful more often?
I found no comprehensions of any kind in my code where binding expressions would actually be of use unless the name "leaked". Other code bases may, of course, yield different guesses. I'm not a "cram a lot of stuff on each line" kind of coder. But the point above remains: if they don't leak, contexts that want them to leak have no recourse. If they do leak, then the other uses would still work fine, but they'd possibly be annoyed by a leak they didn't want.
No, I didn't have much use for `for` target names becoming magically local to invisible nested functions either, but I appreciate that it's less surprising overall. Using ":=" is much more strongly screaming "I'm going way out of my way to give a name to this thing, so please don't fight me by assuming I need to be protected from the consequences of what I explicitly asked for".
Personally, I'd still like to go back to := creating a statement-local name, one that won't leak out of ANY statement. But the tide was against that one, so I gave up on it.
Part of that is because - as the existence of this thread attests to - we can't even control all the scopes gimmicks Python already has. So people are understandably terrified of adding even more ;-)