On Sun, Jun 24, 2018 at 04:33:38PM +1000, Nick Coghlan wrote: [...]
Making the intentional choice to use an assignment expression is not really "implicit" in any meaningful sense.
No, it's actually implicit: there's an extra "global NAME" or "nonlocal NAME" in the equivalent code for a comprehension that isn't there in the as-written source code, and doesn't get emitted for a regular assignment expression or for the iteration variable in a comprehension - it only shows up due to the defined interaction between comprehensions and assignment expressions.
You seem to be talking about an implementation which could change in the future. I'm talking semantics of the proposed language feature. As a programmer writing Python code, I have no visibility into the implementation. The implementation could change ten times a day for all I care, so long as the semantics remain the same.
I want the desired semantics to drive the implementation, not the other way around.
You seem to want the implementation to drive the semantics, by eliminating the proposed feature because it doesn't match your deep understanding of the implementation as a nested function.
I want this feature because its useful, and without it the use-cases for assignment expressions are significantly reduced.
As far as "implicit", for the sake of the discussion, I'll grant you that one. Okay, the proposed behaviour will implicitly enable comprehensions to export their state.
Now what? Is that a good thing or a bad thing?
If "implicit" (with or without the scare quotes) is such a bad thing to be avoided, why are comprehensions implemented using an implicit function?
The problem I have with PEP 572 is that it proposes *breaking that otherwise universal pattern* - instead of having assignment expressions in comprehensions implicitly declare the name as local in the nested comprehension scope, it instead has them:
You talk about "nested comprehension scope", and that's a critical point, but I'm going to skip answering that for now. I have a draft email responding to another of your posts on that topic, which I hope to polish in the next day.
- implicitly declare the name as global or as nonlocal in the
comprehension (or else raise an exception), depending on the nature of the parent scope where the comprehension is used 2. in the nonlocal reference case, amend the symbol table analysis to act like there was a preceding "if 0:\n for NAME in ():\n pass" in the parent scope (so the compiler knows which outer function scope to target)
If it is okay for you to amend the list comprehension to behave as if it were wrapped in an implicit nested function, why shouldn't it be okay to behave as if assignments inside the comprehension included an implicit nonlocal declaration?
The rationale being given for why that is OK is:
- "Everyone" thinks comprehensions are just a for loop (even though
that hasn't been true for the better part of a decade, and was never true for generator expressions)
Obviously "everyone" is an exaggeration, but, yes, I stand by that -- most people don't even give comprehension scope a thought until they get bitten by it.
Either because (Python 2) they don't realise the loop variable is local to their current scope:
or (Python 3) they get bitten by the change:
(As is so often the case, whatever behaviour we choose, we're going to surprise somebody.)
It is hardly surprising that people don't think too hard about scoping of comprehensions. Without a way to perform assignments inside comprehensions, aside from the loop variables themselves, there's nothing going on inside a comprehension where it makes a visible difference whether it is a local scope or a sublocal scope.
*IF* assignment expressions are introduced, that is going to change. We have some choices:
1. Keep assignment expressions encapsulated in their implicit function, and be prepared for people to be annoyed because (with no way to declare them global or non-local inside an expression), they can't use them to get data in and out of the comprehension.
2. Allow assignment expressions to be exported out of the comprehension, and be prepared for people to be annoyed because they clobbered a local.
(But for the reasons Tim Peters has already set out, I doubt this will happen often.)
3. Allow some sort of extra comprehension syntax to allow global/nonlocal declarations inside comprehensions.
x = 1 [nonlocal x := x+i for i in sequence]
(Hmmm... I thought I would hate that more than I actually do.)
4. Have some sort of cunning plan whereby if the variable in question exists in the local scope, it is implicitly local inside the comprehension:
x = 1 [x := i+1 for i in (1, 2)] assert x == 3
but if it doesn't, then the variable is implicitly sublocal inside the comprehension:
del x [x := i+1 for i in (1, 2)] x # raises NameError
Remember, the driving use-case which started this (ever-so-long) discussion was the ability to push data into a comprehension and then update it on each iteration, something like this:
x = initial_value() results = [x := transform(x, i) for i in sequence]
Please, Nick, take your implementor's hat off, forget everything you know about the implementation of comprehensions and their implicit nested function, and tell me that doesn't look like it should work.