On Thu, 19 Nov 2020 18:53:01 +1300 Greg Ewing email@example.com wrote:
Starting a new thread for this as suggested by Guido.
On 18/11/20 7:20 pm, Guido van Rossum wrote:
On Tue, Nov 17, 2020 at 22:12 Greg Ewing <firstname.lastname@example.org mailto:email@example.com> wrote:
If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that
capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction.
I wonder if we should try that idea again? The problem may be backwards compatibility — there’s always someone who depends on such a thing (even if by accident, it would be a pain to debug). Maybe we could invent new syntax to signal this behavior, though that might defeat the purpose.
I think it can be made close enough to backwards compatible, although I'm not greatly opposed to new syntax if it's thought desirable.
Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding.
Note that this is *not* the same as introducing a new scope.
And that's very sad. That means instead of solving the root of the problem, adhoc workarounds are proliferated.
All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished.
In CPython, it would be implemented by creating a new cell for each iteration,
Million cells for million of iterations? Truly a CPython-style solution!
instead of changing the value of an existing cell. For other implementations, I don't know -- it would depend on how those implementations currently deal with nested functions.
Do we need new syntax? My feeling is, probably not. I believe that the vast majority of existing code, perhaps all of it, would be unaffected by this change. All the existing workarounds for the current behaviour would continue to work. The only things that would break would be code that somehow relies on capturing the same instance of the loop variable every time, and I find it hard to imagine code that needs to do that.
If we do want new syntax, my suggestion would be
for new x in some_iterator: ...
I'd suggest that it should be "for let", that it should introduce a new scope, and the scoping should be optimized to always treat "x" (as the "for" control variable) by-value, instead of breeding billions of cells.
Using "let" will then give natural way to introduce proper block-level lexical scoping in Python:
def foo(): let x = 1 if bar: let x = 2 ... # x is 1 again here
Using "let" will also allow for a natural counterpart of "const" keyword syntax, vs annotation, like it has to be now (https://firstname.lastname@example.org/thread/SQTOWJ6...).
Actually, thinking about it, the syntax should be "for const" in the first place:
for const x in some_iterator: ...
That will make it obvious that "x" is captured by value, and that:
for const x in some_iterator: ... x = 1
Doesn't make sense (should lead to error - assigning to a const).
The downside of requiring special syntax is that we would still regularly get people asking why their lambdas in for statements don't do what they expect. We would have a better answer for them, but the questions wouldn't go away.
So, there shouldn't be "special syntax". There should be generic syntax to solve Python's block scoping and const'ness issues.