[Python-ideas] Making the for statement work better with nested functions

Starting a new thread for this as suggested by Guido. On 18/11/20 7:20 pm, Guido van Rossum wrote:

On Tue, Nov 17, 2020 at 22:12 Greg Ewing mailto:greg.ewing@canterbury.ac.nz> wrote:

If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction.

I wonder if we should try that idea again? The problem may be backwards compatibility — there’s always someone who depends on such a thing (even if by accident, it would be a pain to debug). Maybe we could invent new syntax to signal this behavior, though that might defeat the purpose.

I think it can be made close enough to backwards compatible, although I'm not greatly opposed to new syntax if it's thought desirable. Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding. Note that this is *not* the same as introducing a new scope. All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished. In CPython, it would be implemented by creating a new cell for each iteration, instead of changing the value of an existing cell. For other implementations, I don't know -- it would depend on how those implementations currently deal with nested functions. Do we need new syntax? My feeling is, probably not. I believe that the vast majority of existing code, perhaps all of it, would be unaffected by this change. All the existing workarounds for the current behaviour would continue to work. The only things that would break would be code that somehow relies on capturing the same instance of the loop variable every time, and I find it hard to imagine code that needs to do that. If we do want new syntax, my suggestion would be for new x in some_iterator: ... The downside of requiring special syntax is that we would still regularly get people asking why their lambdas in for statements don't do what they expect. We would have a better answer for them, but the questions wouldn't go away. -- Greg