On Sat, Jun 23, 2018 at 12:22:33AM +1000, Nick Coghlan wrote:
- for the reactions to my description of the currently proposed parent
local scoping behaviour in comprehensions, I'd use the word "horrified", and feel I wasn't overstating the response :)
Without knowing how you worded the question, and the reasons for this horrified reaction, I'm afraid that isn't really helpful. It is nothing more than an appeal to emotion:
Such strong emotions as "horrified" are typically a sign of an immediate, emotional gut reaction, not careful thought. We often see those sorts of reactions attached to the most objectively trivial matters. Immediate gut reactions are rarely a good guide because they tend to over-value the status quo, exaggerate the difficulty and costs of change, and under-estimate the benefits.
Speaking personally, I've learned to question my immediately gut reaction. (And I remember to do so at least half the time.) PEP 572 is an example: when the issue was first raised back in February, my gut reaction was "Not in MY Python!!!" but by taking it seriously and running through some examples over the course of the discussion, I realised that, actually, I cautiously favour the idea.
Of course, matters of *personal taste* cannot be anything but gut reaction, but in those matters, what one person holds strongly another can legitimately reject strongly. We ought to try to look beyond personal taste, and try (even if only imperfectly) to consider rational reasons for and against a proposal. If we do, reactions like "horrified" are rarely justified. It's just a minor feature in a programming language, the world will go on one way or the other, and Python already has trickier gotchas.
While I try to account for the fact that I implemented the current comprehension semantics for the 3.x series, and am hence biased towards considering them the now obvious interpretation,
While we certainly don't want to make "non-obvious" a virtue for its own sake, obviousness (obvious to who?) ought to take a distant second place to *useful*. Otherwise we'd have to give up an awful lot of existing Python, starting with the fundamental execution model.
(Oh, the number and length of arguments about whether Python uses call by value or call by reference, why mutable defaults and []*3 are "broken"... if you think Python's execution model is "obvious" you've been using Python too long ;-)
But as Tim Peters has said on a number of occasions, nobody is suggesting changing the interpretation of current comprehension semantics. Comprehension loop variables will continue to remain isolated to the comprehension.
(And for the record, that makes *comprehensions* a weird special case, not assignment expressions. All other expressions run in the current lexical scope. Comprehensions introduce an implicit, invisible, sub-local scope that doesn't match up with a change in indentation as class and def statements do.)
The behaviour in question is a matter of *assignment expression* semantics, not comprehensions. And honestly, I don't see why the proposed behaviour is "horrifying". Here's the high-level overview:
- at the top level of a module, assignment expressions assign in the global scope;
- inside a class, assignment expressions assign in the class scope;
- inside a function, assignment expressions assign in the function local scope (unless declared global or nonlocal);
- inside a comprehension, assignment expressions assign in the surrounding lexical scope (the surrounding function, class or module).
The first three are the same as ordinary statement assignment. The last one is what you would expect if you treat comprehensions as any other expression which run in the current lexical scope. (The current function or class or module.) Even if we treat it as a "weird special case" (I don't think it is, but for the sake of the argument let's say it is) its not hard to explain.
As I discuss below, you can get a very long way indeed working with comprehensions without once thinking about the scope they run in. By the time you need to think about comprehension scope, it shouldn't be hard to deal with the rule:
- loop variables are hidden in a comprehension private scope; - explicit assignment expression variables are not.
This is not async, or metaclasses, or even Unicode.
plenty of functional-language-inspired documentation to instead encourage folks to view comprehensions as tightly encapsulated declarative container construction syntax.
I can't say I've done a broad survey, but the third-party documentation I've read on comprehensions typically glosses over the scoping issues without mentioning them. To the extent that scoping is even hinted at, comprehensions are treated as expressions which are exactly equivalent to re-writing them as a for-loop in the current scope.
This is a typical example, found as the top result on googling for "python comprehensions":
Nothing is mentioned about scope, and it repeats the inaccurate but simple equivalency:
for item in list: if conditional: expression
But perhaps that tutorial is too old. Okay this recent one is only a little more than a year old:
Again, no mention of scoping issues, comprehensions are simply expressions which presumably run in the same scope as any other expression.
I think you over-estimate how many newcomers to Python are even aware that the scope of comprehensions is something to consider.
I'm currently working on a concept proposal at https://github.com/ncoghlan/peps/pull/2 that's much closer to PEP 572 than any of my previous `given` based suggestions:
I look forward to reading it, and I promise I won't go by my gut reaction :-)