[This is my one response today]

On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy <tjreedy@udel.edu> wrote:
On 6/24/2018 7:25 PM, Guido van Rossum wrote:
> I'd wager that the people who might be most horrified about it

the (b) scoping rule change

> would be people who feel strongly that the change to the
> comprehension scope rules in Python 3 is a big improvement,

I might not be one of those 'most horrified' by (b), but I increasingly
don't like it, and I was at best -0 on the comprehension scope change.
To me, iteration variable assignment in the current scope is a
non-problem.  So to me the change was mostly useless churn.  Little
benefit, little harm.  And not worth fighting when others saw a benefit.

Fair enough, and by itself this might not have been enough reason to make the change. But see below.
 
However, having made the change to nested scopes, I think we should
stick with them.  Or repeal them.  (I believe there is another way to
isolate iteration names -- see  below).  To me, (b) amounts to half
repealing the nested scope change, making comprehensions half-fowl,
half-fish chimeras.

That depends on how you see it -- to me (b) just means that there's an implicit nonlocal[1] to make the assignment have the (desirable) side-effect.

The key thing to consider here is whether that side-effect is in fact desirable. For me, the side-effect of the comprehension's loop control variable was never desirable -- it was just an implementation detail leaking out. (And that's different from leaking a regular for-loop's control variable -- since we have 'break' (and 'else') there are some legitimate use cases. But comprehensions try to be expressions, and here the side effect is at best useless and at worst a nasty surprise.)
 
> and who are familiar with the difference in implementation
> of comprehensions (though not generator expressions) in Python 2 vs. 3.

That I pretty much am, I think.  In Python 2, comprehensions (the fish)
were, at least in effect, expanded in-line to a normal for loop.
Generator expressions (the fowls) were different.  They were, and still
are, expanded into a temporary generator function whose return value is
dropped back into the original namespace.  Python 3 turned
comprehensions (with 2 news varieties thereof) into fowls also,
temporary functions whose return value is dropped back in the original
namespace.  The result is that a list comprehension is equivalent to
list(generator_ expression), even though, for efficiency, it is not
implemented that way.  (To me, this unification is more a benefit than
name hiding.)

Right, and this consistency convinced me that the change was worth it. I just really like to be able to say "[... for ...]" is equivalent to "list(... for ...)", and similar for set and dict.
 
(b) proposes to add extra hidden code in and around the temporary
function to partly undo the isolation.

But it just adds a nonlocal declaration. There's always some hidden code ('def' and 'return' at the very least).
 
list comprehensions would no
longer be equivalent to list(generator_expression), unless
generator_expressions got the same treatment, in which case they would
no longer be equivalent to calling the obvious generator function.
Breaking either equivalence might break someone's code.

Ah, there's the rub! I should probably apologize for not clarifying my terminology more. In the context of PEP 572, when I say "comprehensions" I include generators! PEP 572 states this explicitly (https://github.com/python/peps/blame/master/pep-0572.rst#L201-L202).

Certainly PEP 572 intends to add that implicit nonlocal to both comprehensions and generator expressions. (I just got really tired of writing that phrase over and over, and at some point I forgot that this is only a parenthetical remark added in the PEP's latest revision, and not conventional terminology -- alas. :-)

Part (b) of PEP 572 does several things of things to *retain* consistency:

- The target of := lives in the same scope regardless of whether it occurs in a comprehension, a generator expression, or just in some other expression.

- When it occurs in a comprehension or generator expression, the scope is the same regardless of whether it occurs in the "outermost iterable" or not.

If we didn't have (b) the target would live in the comprehension/genexpr scope if it occurred in a comprehension/genexp but outside its "outermost iterable", and in the surrounding scope otherwise.
 
---

How loop variables might be isolated without a nested scope: After a
comprehension is parsed, so that names become strings, rename the loop
variables to something otherwise illegal.  For instance, i could become
'<i>', just as lambda becomes '<lambda>' as the name of the resulting
function.  Expand the comprehension as in Python 2, except for deleting
the loop names along with the temporary result name.

Assignment expressions within a comprehension would become assignment
expressions within the for loop expansion and would automatically add or
replace values in the namespace containing the comprehension.  In other
words, I am suggesting that if we want name expressions in
comprehensions to act as they would in Python 2, then we should consider
reverting to an altered version of the Python 2 expansion.

Possibly this is based on a misunderstanding of my use of "comprehensions". Also, since your trick can only be used for list/set/dict comprehensions, but not for generator expressions (at least I assume you don't want it there) it would actually *reduce* consistency between list/set/dict comprehensions and generator expressions.
 
---

In any case, I think (b) should be a separate PEP linked to a PEP for
(a).  The decision for (a) could be reject (making (b) moot), accept
with (b), or accept unconditionally (but still consider (b)).

For me personally, (b) makes the PEP more consistent, so I'm not in favor of breaking up the PEP. But we can certainly break up the discussion -- that's why I started using the labels (a) and (b).
 
----------
[1] Sometimes it's an implicit global instead of an implicit nonlocal -- when there's already a global for the same variable in the target scope.

--
--Guido van Rossum (python.org/~guido)