[Chris Barker]

...

...

So what about:

l = [x:=i for i in range(3)]

vs

g = (x:=i for i in range(3))

Is there any way to keep these consistent if the "x" is in the regular

local scope?

[Tim]

...

I'm not clear on what the question is. The list comprehension would bind ` l ` to [0, 1, 2] and leave the local `x` bound to 2. The second example binds `g` to a generator object, which just sits there unexecuted. That has nothing to do with the PEP, though.

If you go on to do, e.g.,

l = list(g)

then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local

`x` will

...

be left bound to 2.

[Chris]

OK, it has been said that the priority is that

list(a_gen_expression)

Behave the same as

[the_same_expression]

That's certainly desirable.

So we’re good there. And maybe it’s correct that leaving the running of the gen_exp ‘till later is pretty uncommon, particularly for newbies,

Common or not, I have no idea why anyone would write a genexp like the one you gave, except to contrive an example of silly behavior exhibited by silly code ;-)

It's really not interesting to me to make up code as goofy as you can conceive of - the interesting questions are about plausible code (including plausible coding errors).

but:

If the execution of the gen_exp is put off, it really confuses things — that name being changed would happen at some arbitrary tone, and at least in theory, the gen_exp could be passed off to somewhere else in the code, and be run or not run completely remotely from where the name is used.

Sure.

So while this is technically the same as the comprehension, it is not the same as a generator function which does get its own scope.

It is the same as a generator function with appropriate scope declarations - a generator expression is, after all, implemented _by_ a nested generator function. You can write a workalike to your code above today, but nobody worries about that because nobody does that ;-)

def f(): def bashx(outermost): nonlocal x for i in outermost: x = i yield i

x = 12 g = bashx(range(3)) print("x before", x) L = list(g) print("L", L) print("x after", x)

Then calling `f()` prints:

x before 12 L [0, 1, 2] x after 2

And we should be clear how it will work — after all, in py2, the handling of the looping name was handled differently in gen_exp vs comprehensions.

The PEP specifies the semantics. If it's accepted, that will be folded into the docs.

So I think a local scope for all comprehension-like things would be the way to go.

But getting back to the original thread topic — python has a number of places that you can only use expressions — adding the ability to bind a name in all these places complicates the language significantly.

Did adding ternary `if` (truepart if expression else falsepart) complicate the language significantly? Python has rarely expanded the number of expression forms, but whenever it has the sky didn't actually fall despite earnest warnings that disaster was inevitable ;-)

...

Put a body B in a listcomp and any side effects due to executing B

Maybe it’s just me, but re-binding a name seems like a whole new category of side effect.

With no trickery at all, you've always been able to rebind attributes, and mutate containers, in comprehensions and genexps. Because `for` targets aren't limited to plain names; e.g.,

g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))

is already "legal", and will stomp all over the complex `for` targets when executed - there's nothing "local" about them. But nobody worries about that because nobody does stuff like that.

And as in my goofy code above, mucking with binding of plain names is also possible today. Indeed, straightforward if that's what you _want_ to do. But nobody does.

It's just not one of Python's goals to make it impossible to write useless code ;-)

[Chris]

yes, it was a contrived example, but the simplest one I could think of off the top of my head that re-bound a name in the loop -- which was what I thought was the entire point of this discussion?

But why off the top of your head? There are literally hundreds & hundreds of prior messages about this PEP, not to mention that you could also find examples in the PEP. Why make up a senseless example?

If we think hardly anyone is ever going to do that -- then I guess it

doesn't matter

how it's handled.

So look at real examples. One that's been repeated at least a hundred times wants a local to "leak into" a listcomp:

total = 0 cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with UnboundLocalError instead? Do you currently teach that comprehensions and genexps are implemented via invisible magically generated lexically nested functions? If not, you're going to have to start for people to even begin to make sense of UnboundLocalError if `total` _doesn't_ "leak into" that example. My belief is that just about everyone who doesn't know "too much" about the current implementation will be astonished & baffled if that example doesn't "just work".

In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes): n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction: they don't leak and there's no way to make them leak, not even when that's wanted. Which _is_ wanted in the last example above, which would be clearer still written as:

while any(n % p == 0 for p in small_primes): n //= p

But that ship has sailed.

... And "nonlocal" is not used that often, and when it is it's for careful

closure

trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment expressions _overall_ will be used more often than ternary `if` but significantly less often than augmented assignment. I expect their use in genexps and comprehensions will be minimal. There are real use cases for them, but the vast majority of genexps and comprehensions apparently have no use for them at all.

And, of course, when a newbie encounters it, they can google it and see

what

it means -- far different that seeing a := in a comprehension and

understanding

(by osmosis??) that it might make changes in the local scope.

Which relates to the above: how do you teach these things? The idea that "a newbie" even _suspects_ that genexps and listcomps have something to do with lexically nested scopes and invisible nested functions strikes me as hilarious ;-)

Regardless of how assignment expressions work in listcomps and genexps, this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

How then are you going to explain that this seemingly trivial variation _doesn't_?

[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block containing the brackets. So let them.

Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment expression targets are treated in "most of" a comprehension, because the expression defining the iterable of the outermost "for" _is_ evaluated in the containing block (it is _not_ evaluated in the scope of the synthetic function).

That's not a special case for targets if they all "leak", but is if they don't.

And I don't think you can even do that with generator expressions now --

as

they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do anything whatsoever.

Which is my point -- this would allow the local namespace to be

manipulated

in places it never could before.

As above, not true. However, it would make it _easier_ to write senseless code mucking with the local namespace - if that's what you want to do.

Maybe it's only comprehensions, and maybe it'll be rare to have a

confusing

version of those, so it'll be no big deal, but this thread started

talking about

educators' take on this -- and as an educator, I think this really does complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)

Python got much of it's "fame" by being "executable pseudo code" -- its

been

moving farther and farther away from those roots. That's generally a good

thing,

as we've gain expressiveness in exchangel, but we shouldn't pretend it

isn't

happening, or that this proposal doesn't contribute to that trend.

I didn't say a word about that one way or the other. I mostly agree, but at the start Guido was aiming to fill a niche between shell scripting languages and C. It was a very "clean" language from the start, but not aimed at beginners. Thanks to his experience working on ABC, it carried over some key ideas that were beginner-friendly, though.

I view assignment expressions as being aimed at much the same audience as augmented assignments: experienced programmers who already know the pros and cons from vast experience with them in a large number of other widely used languages. That's also a key Python audience.

... Well, I've been surprised by what confused students before, and I will

again. But I

dont hink there is any doubt that Python 3.7 is a notably harder to learn

that

Python 1.5 was...

Absolutely. It doesn't much bother me, though - at this point the language and its widely used libraries are so sprawling that I doubt anyone is fluent in all of it. That's a sign of worldly success.

... and this:

In [55]: x = 0 In [56]: [x for x in range(3)] Out[56]: [0, 1, 2] In [57]: x Out[57]: 0

doesn't change x in the local scope --

In Python 3, yes; in Python 2 it rebinds `x` to 2.

if that was a good idea, why is a good idea to have := in a comprehension effect the local scope??

Because you can't write a genexp or comprehension AT ALL without specifying `for` targets, and in the overwhelming majority of genexps and comprehensions anyone ever looked at, "leaking" of for-targets was not wanted. "So don't let them leak" was pretty much a no-brainer for Python 3.

But assignment expressions are NEVER required to write a genexp or comprehension, and there are only a handful of patterns known so far in which assignment expressions appear to be of real value in those contexts. In at least half those patterns, leaking _is_ wanted - indeed, essential. In the rest, leaking isn't.

So it goes. Also don't ignore other examples given before, showing how having assignment expressions _at all_ argues for "leaking" in order to be consistent with what assignment expressions do outside of comprehensions and genexps.

But maybe it is just me.

Nope. But it has been discussed so often before this is the last time I'm going to repeat it all again ;-)

... One of these conversations was started with an example something like

this:

[(f(x), g(f(x))) for x in an_iterable]

The OP didn't like having to call f() twice. So that would become:

[ (temp:=f(x), g(temp)) for x in an_iterable]

so now the question is: should "temp" be created / changed in the

enclosing local scope?

This sure looks a lot like letting the iteration name (x in this example)

leak out -

so I'd say no.

In that example, right, leaking `temp` almost certainly isn't wanted. So it goes.

And I don't think this kind of thing would be rare.

I do. It's dead easy to make up examples to "prove" anything people like, but I'm unswayed unless examples come from real code, or are obviously compelling.

Since we're not going to get a way to explicitly say which targets (neither `for` nor assignment expression) do and don't leak, it's a reasonably satisfying compromise to say that one kind never leaks and the other kind always leaks. The pick your poison accordingly.

In the example above, note that they _could_ already do, e.g.,

[(fx, g(fx)) for x in an_iterable for fx in [f(x)]]

Then nothing leaks (well, unless f() or g() do tricky things). I personally wouldn't care that `temp` leaks - but then I probably would have written that example as the shorter (& clearer to my eyes):

[(v. g(v)) for v in map(f, an_iterable)]

to begin with.

On Jun 28, 2018, at 8:21 PM, Tim Peters tim.peters@gmail.com wrote:

Seems it’s all been said, and Tim’s latest response made an excellent case for consistency.

But:

Regardless of how assignment expressions work in listcomps and genexps, this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

This reinforces my point that it’s not just about comprehensions, but rather that the local namespace can be altered anywhere an expression is used — which is everywhere.

That trivial example is unsurprising, but as soon as your line of code gets a bit longer, it could be far more hidden.

I’m not saying it’s not worth it, but it a more significant complication than simply adding a new feature like augmented assignment or terniary expressions, where the effect is seen only where it is used.

A key problem with thinking about this is that we can scan existing code to find places where this would improve the code, and decide if those use-cases would cause confusion.

But we really can’t anticipate all the places where it might get used (perhaps inappropriately) that would cause confusion. We can hope that people won’t tend to do that, but who knows?

Example: in a function argument:

result = call_a_func(arg1, arg2, kwarg1=x, kwarg2=x:=2*y)

Sure, there are always ways to write bad code, and most people wouldn’t do that, but someone, somewhere, that thinks shorter code is better code might well do it. Or something like it.

After all, expressions can be virtually anywhere in your code.

Is this a real risk? Maybe not, but it is a complication.

-CHB

On 30 June 2018 at 09:49, Chris Barker - NOAA Federal via Python-Dev python-dev@python.org wrote:

...

On Jun 28, 2018, at 8:21 PM, Tim Peters tim.peters@gmail.com wrote:

Seems it’s all been said, and Tim’s latest response made an excellent case for consistency.

But:

...

Regardless of how assignment expressions work in listcomps and genexps, this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

This reinforces my point that it’s not just about comprehensions, but rather that the local namespace can be altered anywhere an expression is used — which is everywhere.

That trivial example is unsurprising, but as soon as your line of code gets a bit longer, it could be far more hidden.

The significant semantic differences between "{x : 1}" and "{x := 1}" are also rather surprising :)

Cheers, Nick.

On 29 June 2018 at 08:42, Chris Barker via Python-Dev python-dev@python.org wrote:

On Thu, Jun 28, 2018 at 9:28 AM, Tim Peters tim.peters@gmail.com wrote:

...

Did adding ternary `if` (truepart if expression else falsepart) complicate the language significantly?

I don't think so -- no. For two reasons:

1. the final chosen form is kind of verbose, but therefor more like

"executable pseudo code" :-) As apposed to the C version, for instance.

2. it added one new construct, that if, when someone sees it for the first

(or twenty fifth) time and doesn't understand it, they can look it up, and find out. and it only effects that line of code.

So adding ANYTHING does complicate the language, by simply making it a bit larger, but some things are far more complicating than others.

It's worth noting that without the bug prone "C and A or B" construct (which gives the wrong result when "not A" is True), we'd likely never have gotten "A if C else B" (which gives the right result regardless of the truth value of A). In the case of PEP 308, the new construction roughly matched the existing idiom in expressive power, it just handled it correctly by being able to exactly match the developer's intent.

"NAME := EXPR" exists on a different level of complexity, since it adds name binding in arbitrary expressions for the sake of minor performance improvement in code written by developers that are exceptionally averse to the use of vertical screen real estate, and making a couple of moderately common coding patterns (loop-and-a-half, if-elif-chains with target binding) more regular, and hence easier to spot.

I think the current incarnation of PEP 572 does an excellent job of making the case that says "If we add assignment expressions, we should add them this particular way" - there are a lot of syntactic and semantic complexities to navigate, and it manages to make its way through them and still come out the other side with a coherent and self-consistent proposal that copes with some thoroughly quirky existing scoping behaviour.

That only leaves the question of "Does the gain in expressive power match the increase in the cognitive burden imposed on newcomers to the language?", and my personal answer to that is still "No, I don't think it does". It isn't my opinion on that that matters, though: I think that's now going to be a conversation between Guido and folks that are actively teaching Python to new developers, and are willing to get their students involved in some experiments.

Cheers, Nick.

P.S. It does make me wonder if it would be possible to interest the folks behind https://quorumlanguage.com/evidence.html in designing and conducting fully controlled experiments comparing the comprehensibility of pre-PEP-572 code with post-PEP-572 code *before* the syntax gets added to the language :)

On Sat, Jun 30, 2018 at 9:43 AM Tim Peters tim.peters@gmail.com wrote:

The attractions are instead in the areas of reducing redundancy, improving clarity, allowing to remove semantically pointless indentation levels in some cases, indeed trading away some horizontal whitespace in otherwise nearly empty lines for freeing up a bit of vertical screen space, and in the case of comprehensions/genexps adding straightforward ways to accomplish some conceptually trivial things that at best require trickery now (like emulating a cell object by hand).

The examples you provided (some were new in this thread, I think) are compelling. While my initial reaction to the proposal was mild horror, I'm not troubled by the scoping questions.

Issues still bothering me: 1. Initial reactions from students was confusion over := vs = 2. This seems inconsistent with the push for type hints

To be fair, I felt a similar gut reaction to f-strings, and now I can't live without them. Have I become a cranky old man, resistant to change? Your examples have put me into the "on the fence, slightly worried" category instead of "clearly a bad idea".

On scoping, beginners seem more confused by UnboundLocalError than by variables bleeding between what they perceive as separate scopes. The concept of a scope can be tricky to communicate. Heck, I still make the mistake of looking up class attributes in instance methods as if they were globals. Same-scope is natural. Natural language is happy with ambiguity. Separate-scope is something programmers dreamed up. Only experienced C, Java, etc. programmers get surprised when they make assumptions about what syntax in Python creates separate scopes, and I'm not so worried about those folks. I remind them that the oldest versions of C didn't have block scopes (1975?) and they quiet down.

The PEP lists many exclusions of where the new := operator is invalid [0]. I unfortunately didn't have a chance to read the initial discussion over the operator. I'm sure it was thorough :-). What I can observe is that each syntactical exclusion was caused by a different confusion, probably teased out by that discussion. Many exclusions means many confusions.

My intuition is that the awkwardness stems from avoiding the replacement of = with :=. Languages that use := seem to avoid the Yoda-style comparison recommendation that is common to languages that use = for assignment expressions. I understand the reluctance for such a major change to the appearance of Python code, but it would avoid the laundry list of exclusions. There's some value in parsimony.

Anyway, we've got some time for testing the idea on live subjects.

Have a good weekend, everyone. -- Michael

PS. Pepe just tied it up for Portugal vs Uruguay. Woo! ... and now Cavani scored again :-(

[0] https://www.python.org/dev/peps/pep-0572/#exceptional-cases

[Nick Coghlan]

...

...

"NAME := EXPR" exists on a different level of complexity, since it adds name binding in arbitrary expressions for the sake of minor performance improvement in code written by developers that are exceptionally averse to the use of vertical screen real estate,

...

...

...

...

[Tim]

...

Note that PEP 572 doesn't contain a single word about "performance"

(neither

...

...

that specific word nor any synonym), and I gave only one thought to it

when

...

...

writing Appendix A: "is this going to slow anything down

significantly?".

...

...

The answer was never "yes", which I thought was self-evident, so I never

...

...

mentioned it. Neither did Chris or Guido.

...

...

...

...

Best I can recall, nobody has argued for it on the grounds of

"performance".

...

...

except in the indirect sense that sometimes it allows a more compact

way of

...

...

reusing an expensive subexpression by giving it a name. Which they

already

...

...

do by giving it a name in a separate statement, so the possible

improvement

...

...

would be in brevity rather than performance.

[Nick]

...

The PEP specifically cites this example as motivation:

The PEP gives many examples. Your original was a strawman mischaracterization of the PEP's _motivations_ (note the plural: you only mentioned "minor performance improvement", and snipped my listing of the major motivations).

group = re.match(data).group(1) if re.match(data) else None

...

...

That code's already perfectly straightforward to read and write as a

...

single line,

I disagree. In any case of textual repetition, it's a visual pattern-matching puzzle to identify the common substrings (I have to visually scan that line about 3 times to be sure), and then a potentially difficult conceptual puzzle to figure out whether side effects may result in textually identical substrings evaluating to different objects. That's why "refererential transparency" is so highly valued in functional languages ("if subexpressions are spelled the same, they evaluate to the same result, period" - which isn't generally true in Python - to get that enormously helpful (to reasoning) guarantee in Python you have to ensure the subexpression is evaluated exactly once).

And as you of all people should be complaining about, textual repetition is also prone to "oops - forgot one!" and "oops! made a typo when changing the second one!" when code is later modified.

so the only reason to quibble about it

I gave you three better reasons to quibble about it just above ;-)

is because it's slower than the arguably less clear two-line alternative:

...

...

_m = re.match(data)

...

group = _m.group(1) if _m else None

I find that much clearer than the one-liner above: the visual pattern matching is easier because the repeated substring is shorter and of much simpler syntactic structure; it guarantees _by construction_ that the two instances of `_m` evaluate to the same object, so there's no possible concern about that (it doesn't even matter if you bound `re` to some "non-standard" object that has nothing to do with Python's `re` module); and any later changes to the single instance of `re.match(data)` don't have to be repeated verbatim elsewhere. It's possible that it runs twice as fast too, but that's the least of my concerns.

All of those advantages are retained in the one-liner too if an assignment expression can be used in it.

Thus the PEP's argument is that it wants to allow the faster version

...

to remain a one-liner that preserves the overall structure of the

...

version that repeats the subexpression:

...

...

group = _m.group(1) if _m := re.match(data) else None

...

...

That's a performance argument, not a readability one (as if you don't

...

care about performance, you can just repeat the subexpression).

How does that differ from the part of what I said that you did retain above?

...

sometimes it allows a more compact way of reusing an expensive subexpression by giving it a name. Which they already do by giving it a name in a separate statement, so the possible improvement would be in brevity rather than performance.

You already realized the performance gain could be achieved by using two statements. The _additional_ performance gain by using assignment expressions is at best trivial (it may save a LOAD_FAST opcode to fetch the object bound to `_m` for the `if` test).

So, no, gaining performance is _not_ the motivation here. You already had a way to make it "run fast'. The motivation is the _brevity_ assignment expressions allow while _retaining_ all of the two-statement form's advantages in easier readability, easier reasoning, reduced redundancy, and performance.

As Guido said, in the PEP, of the example you gave here:

Guido found several examples where a programmer repeated a subexpression, slowing down the program, in order to save one line of code

It couldn't possibly be clearer that Guido thought the programmer's motivation was brevity ("in order to save one line of code"). Guido only happened to mention that they were willing to slow down the code to get that brevity, but, as above, they were also willing to make the code harder to read, reason about, and maintain. With the assignment expression, they don't have to give up any of the latter to get the brevity they mistakenly _think_ ;-) they care most about - and, indeed, they can make it even briefer.

I sure don't count it against the PEP that it may trick people overly concerned with brevity into writing code that's clearer and faster too, but that's a tiny indirect part of the PEP's motivation_s_ (note the plural again).

I think I'll bow out of this now. It's just too tedious.

Like here:

[Nick]

I never said the motivation was to gain performance relative to the two-statement version - I said the motivation given in the PEP is to gain performance relative to the *repeated subexpression* version, *without* making the transition to the already supported two-statement version.

This is what you wrote:

...

"NAME := EXPR" exists on a different level of complexity, since it adds name binding in arbitrary expressions for the sake of minor performance improvement in code written by developers that are exceptionally averse to the use of vertical screen real estate,

I'm not telepathic, so took it for it what said. It didn't say a word about "repeated expressions", nor a word about "two-statement versions"

It did say "minor" performance improvements, which sure suggested to me that you had in mind the tiny changes in bytecode that can result from squashing a current two-statement spelling into a one-statement form with an embedded assignment. That fit both "minor performance improvement" and "vertical screen space", which were the only the two clues I had to go on.

In the example you eventually gave in a later message, the performance difference was more on the order of a factor 2 (regexp searches can be expensive indeed), which matches nobody's idea of "minor". So the use of "minor" wholly ruled out in my mind that you had _anything_ akin to your eventual example in mind.

For the rest, sure, words could be added to the PEP without end. At least some of the points I covered were telegraphically mentioned in my Appendix, although I'll grant that nobody else is telepathic either ;-)

For the question:

...

The remaining point of contention is only the "Inevitable cost of

...

change" one: given the level of disruption this will cause in the way

...

that Python gets taught to new users, is it giving a commensurate

...

pay-off in increased semantic expressiveness?

...

...

My answer to that question remains "No", while your answer is either

...

"Yes" or "I don't see why that should matter" (I'm genuinely unsure

...

which).

I don't know, and I'm not qualified to guess - I don't teach Python to new users for a living. Decades ago I tutored "advanced" engineering undergrads in a variety of science-y subjects, and was routinely surprised by what broke their brains.

I have noted that assignment expressions have been part of a great many languages for decades (this isn't cutting edge tech), questions about them are conspicuous by absence on StackOverflow (everyone else seems to teach them effectively), and skimming various online teaching materials for other languages convinced me that none of those professional educators thought it needed more than a page to teach (and to teach them with relatively few words). There's really not much to them: people have to learn what binding means in Python regardless, pretty much starting in the first hour, yes? "Binding" on its own is the hard part.

If they don't drop out and stick around for the 10-minute lesson on assignment expressions 3 weeks later, will _that_ finally break their brains? "Wow - it not only binds the name, but ALSO returns the object that was bound?! I just can't take it - please, let's go back to the lesson on metaclasses" just doesn't seem likely to me ;-)

At heart, ":=" could make a good case for being the simplest of all Python's operators. It does no computation at all, and you don't even have to worry about a dunder method changing its meaning depending on context.

So, ya, when someone claims they'll make Python significantly harder to teach, I'm skeptical of that claim. Which does not mean I believe it's wrong - it means I'm skeptical. I would also be skeptical of a claim that teaching them would be no trouble at all, except nobody has made such a claim ;-)

On Sun, Jul 1, 2018 at 8:35 AM, Michael Selik mike@selik.org wrote:

As Mark and Chris said (quoting Mark below), this is just one straw in the struggle against piling too many things on the haystack. Unlike some changes to the language, this change of such general use that it won't be an optional topic. Once widely used, it ain't optional.

Exactly -- and I also emphasis that this would complicate the language in a broad way -- a much bigger deal than adding a self contained new expression or even the nonlocal keyword.

But to be clear about my take -- it will make the language that little bit harder to teach, but it will also add a bit of complexity that will effect us non-newbies as well.

Is it worth it? maybe. I know I like a better way to express the loop-and-a-half concept in particular.

-CHB

Is this message some kind of joke or did you just send it to the wrong mailing-list/recipient?

On Sun, 1 Jul 2018 20:21:19 -0700 Matt Arcidy marcidy@gmail.com wrote:

This cynical view on students is shocking! Everyone on this list has been a student or a learner for far longer than an educator, and the perspective from students and learners are far more important than educators to assess this angle regardless. Can anyone adequately explain why this specific modality of learning, a student-in-a-seat based educator, must outweigh all other modalities learners use to increase knowledge and skill, from the perspectives of policy, tool creation, and each of our time spent learning?

Shortest story: Teach not to re-use names.

Short story:

1. What about the full mosaic of learning vs. this myopic view on

seat-based student-educator interaction? 2) What about smart, motivated, diligent and cautious students? 3) What weight should educator opinion be given with respect to providing convenience to professional Python programmers? 4) Who is this Student Stupid von Densemeister anyways? 5) Are assignment expressions convenience and is any danger the pose unmitagatble? 6) Consider adding an "Important Topics not Covered" or "Further Reading" reading section to your class description 7) Creating examples showing this effect is easy, especially when not actually re-using the name in the expression for explanatory purposes. it's the same as creating examples showing how re-use works in comprehensions.

Let's stop constructing these fake Students. They only work as appeals to the people we have come across whose lack of understanding has made our life painful. This construction is actively filtering all the good students for the sake of influencing this decision, yet again punishing or discounting the intelligent, quick, and diligent.

And what of this underlying premise that educator's should _significantly_ influence language development? Limiting Python's tools to Student Straw-man's ability to learn is just dissonant, they have nothing to do with each other, nor does this cause-effect relationship actually exist. Let's evaluate this reductionist statement: "I understand X, but this other person is not capable of understanding X, therefore X should not exist" Is has there ever been an X for which this is true, let alone the backwardation necessary to fully close the statement?

The actual argument is far less reductionist, yet even more ridiculous: "I understand X, this other person may take time to learn X, and may use X wrong, therefore X should not exist" "I understand assignment expressions, but this other class of person may take time to learn assignment expressions, and may use assignment expressions wrong, therefore assignment expressions should not be accepted"

Rhetorically I disagree with how teaching is being presented, to the point of near insult (for me lacking a better term). You are saying these statements about _my_ learning path, (though not personally of course.) Each of you occupied a role of student at some point, and each of these statements are being made about your path as well. Do these ring true of your student experience? What about your much broader experience as a _learner_? You think a tool shouldn't exist because it took you time to learn it and you wrote some hard to debug code, and possibly crashed production, got fired, lost your house and your pet snake, and crashed the planet into the sun?

Now I yield, I will accept this position: all/some students cannot learn this (or it's too complex to teach), but they must learn this during some class to quickly become effective python developers. How much weight should this position have in this decision? Let's appeal to the learner in us. How much of our learner's path, percentage of total time learning all things python related, has been in a seat listening to someone else, and that's the only place from which we gained the knowledge to meet the educator's objective? This time spent in a class, how does that compare to hours in other learning modalities? Is this percentage not exactly the weight assigned to that position? Are people hired from pure class-room based experience expected to require zero further learning? Are people more valuable based on classroom hours or work hours?

As for handling teaching the subject or not, this is easily remedied with how I do it: "Important Topics not Covered", with resources.

Anyone here can rightfully claim educator status by having taught another person something related to this language, which includes at-work mentoring, informal discussions, posting/replying on SO, blogging, etc. Are they not being solicited to comment as well? It's possible to answer this question while vehemently disagreeing with the PEP. This focus on people who are being ostensibly paid to teach is myopic.

Concretely, it's clear to me that parent-local effects can be dangerously non-obvious when reading and mimicking code without undertsanding. But when? And how to guard against? How about this: teach proper (i.e. not) re-using names. The name will still be ejected to the parent scope, but there won't be any use of it. Teach the explicit declare pattern first (as everyone does anyways), explain to not re-use. Regardless of when re-use is done, it is always (or only) as dangerous as the effect the bound value has anyways, and any re-use has the potential to trigger the same dangerous behaviors.

Must we continue this educator assessment of PEP 572? Educators are not gate-keepers, and the only measure of their success is what students learn, and students have a far larger and more fine-grained mosaic now than ever. Seat-based education plays a far smaller role in anyone's learning path than ever before, and even while in the seat they have access to Google. All this yield to tiling the outcome in the favor of the educator by assigning the goal meeting to them, not to the diligence of the student to learn. How unfair to the student.

I consider this position purposefully ignoring motivated self-learners of high ability and skill, or just plain old diligent programmers who learned to read specs before using tools.

I don't like posting to python-dev because it's not really my realm, but this topic is insanely tilted against PEP572 for the most ridiculous of reasons. I am Pro-572 , so I have decided to join critique of the educator position. I would rather do it on python-Ideas, and I want more types of educators solicited, as well as students and learners. And yes, lets assess your specific lesson plans if you will make claims it's not possible. Do you even teach the difference between assignments and expressions at all?

To have this raised in the this stage of this PEP, and on the dev list, illustrates how long it took educators to understand the tool to begin with, as opposed to those who understood even if they disagree. To have a room full of seat-based educators provide feedback on a tool they have had 30m to understand as critical to shaping the language is not defensible in these lower courts. This angle cannot withstand rigorous scrutiny because each premise is false and it rests the majority of students being dense. They aren't, and the vast majority of learners don't need class-room education anyways, so what is this weight being placed on the opinion of these educators? I agree it's important to hear it, but dimishingly.

This is not to say that PEP572 should be accepted otherwise. However, this educator angle, raised only now and depending on this platonically dumb student and a non-creative approach to education, is just pure straw-man to distract from the point at hand. And while I have repeated "straw-man" as the critique, I dislike leaning on such a debate-team crutch, and of course employing straw-man doesn't mean the point is invalid. it just means the rhetoric is bad. However, as the underlying premise is a severe minority opinion yet claiming to be broad, and the absolute percentage of solicited opinions is 0% (to the 17th place), I don't see any importance to the position of educators right now, especially since these educators in the thread are complaining about an increase in their personal work, for which it appears they were compensated (this is pretty bad straw-man, sorry).

And to repeat, what about the learners? Time spent in seat based education is so insanely small now. The fact that the name is ejected to the parent stop is not terribly difficult, and appropriately factored examples showing the the effect of scope ejection will be easy to construct, even if trivial for explanation purposes. So what, exactly, is the issue with learning it?

On Sun, Jul 1, 2018 at 4:35 PM Chris Barker via Python-Dev python-dev@python.org wrote:

...

On Sun, Jul 1, 2018 at 8:35 AM, Michael Selik mike@selik.org wrote:

...

As Mark and Chris said (quoting Mark below), this is just one straw in the struggle against piling too many things on the haystack. Unlike some changes to the language, this change of such general use that it won't be an optional topic. Once widely used, it ain't optional.

Exactly -- and I also emphasis that this would complicate the language in a broad way -- a much bigger deal than adding a self contained new expression or even the nonlocal keyword.

But to be clear about my take -- it will make the language that little bit harder to teach, but it will also add a bit of complexity that will effect us non-newbies as well.

Is it worth it? maybe. I know I like a better way to express the loop-and-a-half concept in particular.

-CHB

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/marcidy%40gmail.com

On Mon, Jul 2, 2018 at 2:34 AM Michael Selik mike@selik.org wrote:

On Sun, Jul 1, 2018 at 8:21 PM Matt Arcidy marcidy@gmail.com wrote:

...

[...] Can anyone adequately explain why this specific modality of learning, a student-in-a-seat based educator, must outweigh all other modalities [...]?

1. It doesn't.
2. It's a proxy for the other modes.

I hope this was an adequate explanation.

Absolutely, thank you. We agree it doesn't out weigh other methods. Clearly I disagree about the proxying.

On Sun, Jul 1, 2018 at 5:28 PM Steven D'Aprano steve@pearwood.info wrote:

On Sun, Jul 01, 2018 at 08:35:08AM -0700, Michael Selik wrote:

...

On Sun, Jul 1, 2018 at 12:39 AM Tim Peters tim.peters@gmail.com wrote:

...

So, ya, when someone claims [assignment expressions will] make Python significantly harder to teach, I'm skeptical of that claim.

I don't believe anyone is making that claim. My worry is that assignment expressions will add about 15 to 20 minutes to my class and a slight discomfort.

How do people who teach other languages deal with this?

Python may be in a unique situation in the history of programming. It wouldn't surprise me if more people learned Python last year than any other programming language.

Assignment expressions are hardly a new-fangled innovation of Python's. They're used in Java, Javascript, Ruby, Julia, R, PHP and of course pretty much the entire C family (C, C++, C# at least). What do teachers of those languages do?

Assignment expressions are not the issue. The real question is: How do open-source projects balance the addition of new features against the growth of complexity? It's the same as that "Remember the Vasa" thread.

[...] R [has] *four* different ways of doing assignment.

I think that's a good explanation of why I teach Python and not R. The first time someone asked me to teach a data science course, Python wasn't the clear winner. In fact, R may have been more popular among statisticians. I picked Python for the same reason it's more popular in the industry -- it's the easiest* to use.

* Easiest that gets the job done well.

As Mark and Chris said (quoting Mark below), this is just one straw in the

...

struggle against piling too many things on the haystack. Unlike some changes to the language, this change of such general use that it won't be an optional topic. Once widely used, it ain't optional.

Without knowing the details of your course, and who they are aimed at, we cannot possibly judge this comment.

I disagree. I think the sentiment holds for a great variety of courses and audiences.

Decorators are widely used, but surely you don't teach them in a one day introductory class aimed at beginners?

Most of the time, no. Once, yes, because that's what the team needed. I was pretty proud of myself for handling that one. Because I had to teach decorators early, many other important topics were excluded.

Here is the syllabus for a ten week course:

https://canvas.uw.edu/courses/1026775/pages/python-100-course-syllabus

Note that decorators and even regular expressions don't get touched until week ten. If you can't fit assignment expressions in a ten week course, you're doing something wrong. If you can't fit them in a two hour beginners course, there is so much more that you aren't covering that nobody will notice the lack.

It's not about any one particular topic, but the trade-offs between topics. A 10-week lecture course might be 30 hours of lecture, comparable to a 4-day "bootcamp" style course. I assure you that 4 days doesn't feel long enough when those last few hours are winding down. There's always more to say.

On Mon, 2 Jul 2018 10:25:42 +1000 Steven D'Aprano steve@pearwood.info wrote:

How do people who teach other languages deal with this?

Assignment expressions are hardly a new-fangled innovation of Python's. They're used in Java, Javascript, Ruby, Julia, R, PHP and of course pretty much the entire C family (C, C++, C# at least).

Those other languages don't have two different assignment operators, AFAIK. That's the main point of complication PEP 572 introduces, not the fact that assignment can now be used in an expression.

Admittedly R has the advantage that they don't have to teach a distinct assignment syntax and explain *why* it ought to be distinct. But countering that, they have *four* different ways of doing assignment.

I don't think R is easy to understand. It depends on the demographics. For a software engineer like me, it's pretty hard to wrap my head around R's nonsense. I think R is only easy if you accept to use it in a "tinker aimlessly until my code works" manner.

Regards

Antoine.

On Sun, Jul 1, 2018 at 11:36 PM Tim Peters tim.peters@gmail.com wrote:

[Michael Selik]

...

My worry is that assignment expressions will add about 15 to 20 minutes to my class and a slight discomfort.

So not intractable - which is my high-order bit ;-)

For those who want more bits of precision (perhaps Guido), while quantification is good, it needs context to provide insight. Like, out of how many class hours total?

Generally between 20 and 40 hours.

Is 15-20 minutes a little, a lot, par for the course ... compared to other

topics?

I guessed 15-20 minutes, because I'm mentally comparing it to things like ternary expressions. Odds and ends that make the code better, but not a major concept that deserves hours.

Will it require you to drop other topics?

Yes. It might not seem like much, but every minute counts. I'd probably try to ignore := unless some pesky student brings it up. It's like someone saying, "Hey, I heard that Python can't do threads?!" I always say, "Good question," but internally I'm thinking, "there goes a half hour. What can I cut today?"

Would you _save_ twice as much class time if we got rid of "is"? ;-)

Ha. You joke, but ``is`` takes about 5 minutes. About 5 or 10 minutes more if some clever student notices that ``1 is 1`` and I need to explain Singletons and interpreter optimizations versus language spec.

If it's accepted, do read the PEP

I've read it a few times now. I hope I didn't sound like I haven't read it. That'd be embarrassing.

Meta: About the Vasa, I'm not concerned.

Matt Arcidy brought up an interesting point, which I'll quote here: "... I don't see any importance to the position of educators right now, especially since these educators in the thread are complaining about an increase in their personal work, for which it appears they were compensated."

From my brief observations, it seems that the nattering nabobs of

negativism, such as myself, are mostly educators. I recently started to wonder if I'd care so much about the language if I didn't teach. I suspect that if I didn't worry about teaching new features, Python 4 could be announced tomorrow and I wouldn't really mind.

I suppose it is selfish. But I hope that you [Tim], Guido, and the so many others who have poured energy into this project will appreciate that it's not the current users, but the next billion (?!) Pythonistas that will really keep the language going. Maintaining popularity among educators is a big part of that.

On Sat, Jun 30, 2018 at 11:32:03PM -0500, Tim Peters wrote:

[...]

So, no, gaining performance is _not_ the motivation here. You already had a way to make it "run fast'. The motivation is the _brevity_ assignment expressions allow while _retaining_ all of the two-statement form's advantages in easier readability, easier reasoning, reduced redundancy, and performance.

I think the two of you (Tim and Nick) are in violent agreement :-)

In fairness to Nick performance is _a_ motivation here, in the sense of "how can we avoid making redundant multiple calls to a function in an expression without splitting it into multiple statements?".

If performance were the *only* concern, then I agree with your point that we already have a solution. Simply refactor this:

process(expensive_call(arg), expensive_call(arg) + 1)

to this:

useful_value = expensive_call(arg) process(useful_value, useful_value + 1)

But since performance is not the *only* concern, all the other factors you refer to (saving vertical space, DRY, side-effects, etc) count too.

Assignment expressions aren't valuable because they give us One Big Win. They're valuable because they give us Many Little Wins, which we (proponents of PEP 572) believe will outweigh the (minor) additional costs in complexity and opportunities for abuse.

The closest (in my opinion) assignment expressions comes to a One Big Win is to reduce the need for cascades of if... statements:

m = re.match(pattern, text) if m: ... else: m = re.match(other_pattern, text) if m: ... else: m = re.match(third_pattern, text) if m: ...

which wastes both vertical and horizontal space.

If that were the *only* win, I'd consider dedicated syntax for if/elif statements alone, but it isn't. Little wins include:

- use in while statement headers, "while x:= expr"

- DRY inside comprehensions, as an alternative to the neat but somewhat obscure idiom:

[(x, x+1, x*2) for a in it for x in (func(a),) if x]

- running totals and similar inside comprehensions, where we need a way to initialise the total on the first iteration

total = 0 [total := total + a for a in it]

- and examples such as Tim's use of Heron's Formula:

area = sqrt((s := (a+b+c)/2)*(s-a)*(s-b)*(s-c))

This last one is, I think, the only one where the postfix form reads better, but maybe I only say that because I'm more familiar with it:

area = sqrt(s*(s-a)*(s-b)*(s-c)) where s = (a+b+c)/2

On Thu, Jun 28, 2018 at 03:42:49PM -0700, Chris Barker via Python-Dev wrote:

If we think hardly anyone is ever going to do that -- then I guess it doesn't matter how it's handled.

That's how you get a language with surprising special cases, gotchas and landmines in its behaviour. (Cough PHP cough.)

It is one thing when gotchas occur because nobody thought of them, or because there is nothing you can do about them. But I do not think it is a good idea to *intentionally* leave gotchas lying around because "oh, I didn't think anyone would ever do that...".

*wink*

[...]

but here the keyword "nonlocal" is used -- you are clearly declaring that you are messing with a nonlocal name here -- that is a lot more obvious than simply using a :=

But from the point of view of somebody reading the code, there is no need for a nonlocal declaration, since the assignment is just a local assignment.

Forget the loop variable -- it is a special case where "practicality beats purity" and it makes sense to have it run in a sublocal scope. Everything else is just a local, regardless of whether it is in a comprehension or not.

And "nonlocal" is not used that often, and when it is it's for careful closure trickery -- I'm guessing := will be far more common. And, of course, when a newbie encounters it, they can google it and see what it means -- far different that seeing a := in a comprehension and understanding (by osmosis??) that it might make changes in the local scope.

I've given reasons why I believe that people will expect assignments in comprehensions to occur in the local scope. Aside from the special case of loop variables, people don't think of comprehensions as a separate scope.

There's no Comprehension Sublocal-Local-Enclosing Local-Global-Builtin scoping rule. (Do you want there to be?) Even *class scope* comes as an unfamiliar surprise to people.

I do not believe that people will "intuitively" expect assignments in a comprehension to disappear when the comprehension finishes -- I expect that most of the time they won't even think about it, but when they do, they'll expect it to hang around like *every other use* of assignment expressions.

And I don't think you can even do that with generator expressions now -- as they can only contain expressions.

It makes me cry to think of the hours I spent -- and the brownie points I lost with my wife -- showing how you can already simulate this with locals() or globals(). Did nobody read it? :-(

https://mail.python.org/pipermail/python-dev/2018-June/154114.html

Yes, you can do this *right now*. We just don't because playing around with locals() is a dodgy thing to do.

Maybe it's only comprehensions, and maybe it'll be rare to have a confusing version of those, so it'll be no big deal, but this thread started talking about educators' take on this -- and as an educator, I think this really does complicate the language.

See my recent post here:

https://mail.python.org/pipermail/python-dev/2018-June/154184.html

I strongly believe that the "comprehensions are local, like everything else" scenario is simpler and less surprising and easier to explain than hiding assignments inside a sublocal comprehension scope that hardly anyone even knows exists.

Especially if we end up doing it inconsistently and let variables sometimes leak.

Python got much of it's "fame" by being "executable pseudo code" -- its been moving farther and farther away from those roots. That's generally a good thing, as we've gain expressiveness in exchangel, but we shouldn't pretend it isn't happening, or that this proposal doesn't contribute to that trend.

I think there are two distinct forms of "complication" here.

1. Are assignment expressions in isolation complicated?

2. Given assignment expressions, can people write obfuscated, complex code?

Of course the answer to Q2 is yes, the opportunity will be there. Despite my general cynicism about my fellow programmers, I actually do believe that the Python community does a brilliant job of self-policing to prevent the worst excesses of obfuscatory one-liners. I don't think that will change.

So I think Q1 is the critical one. And I think the answer is, no, they're conceptually bloody simple. They evaluate the expression on the right, assign it to the name on the left, and return that value.

Here is a question and answer:

Question: after ``result = (x := 2) + 3``, what is the value of x?

Answer: 2.

Question: what if we put the assignment inside a function call? ``f((x:=2), x+3)``

Answer: still 2.

Question: how about inside a list display? ``[1, x:=2, 3]``

Answer: still 2.

Question: what about a dict display? ``{key: x:=2}`` A tuple? A set?

Answer: still 2 to all of them.

Question: how about a list comprehension?

Answer: ah, now, that's complicated, it depends on which bit of the comprehension you put it in, the answer could be that x is 2 as you would expect, or it could be undefined.

Yes, I can see why as an educator you would prefer that over *my* answer, which would be:

Answer: it's still ^&*@^!$ two, what did you think it would be???

*wink*

...

...

Maybe it’s just me, but re-binding a name seems like a whole new category of side effect.

With no trickery at all, you've always been able to rebind attributes, and mutate containers, in comprehensions and genexps. Because `for` targets aren't limited to plain names; e.g.,

g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))

sure, but you are explicitly using the names "object" and "a" here -- so while side effects in comprehension are discouraged, it's not really a surprised that namespaces specifically named are changed.

Trust me on this Chris, assuming the arguments over this PEP are finished by then (I give 50:50 odds *wink*) by the time 3.8 comes out you'll be saying

"sure, but you are explicitly assigning to a local variable with the := operator, it's not really a surprise that the local variable specifically named is being changed, it would be weird if it wasn't..."

and you'll have forgotten that there was ever a time we seriously discussed comprehension-scope as a thing.

:-)

[example with the loop variable of a comprehension]

doesn't change x in the local scope -- if that was a good idea, why is a good idea to have := in a comprehension effect the local scope??

But maybe it is just me.

Some of us don't think that it was a good idea *wink*

But I know when I'm outvoted and when "practicality beats purity".

Loop variables are semantically special. They tend to have short, often one-character names, taken from a small set of common examples (i, j, x are especially common). We don't tend to think of them as quite the same as ordinary variables: once the loop is complete, we typically ignore the loop variable.

It takes a conscious effort to remember that it actually still hangs around. I've written code like this:

for x in sequence: last_x = x do_stuff() # outside the loop process(last_x)

and then looked at it a day later and figuratively kicked myself. What the hell was I thinking?

So it is hardly surprising that people would sometimes write:

for x in sequence: L = [expression for x in something] process(x)

and be unpleasantly surprised in Python 2.

But this is not likely to happen BY ACCIDENT with assignment expressions, and if it does, well, the answer is, change the variable name to something a little less generic, or at least *different*:

for x in sequence: L = [x := a+1 for a in something] process(x)

I am reluctantly forced to agree that, purity be damned, burying the loop variable inside a hidden implicit function scope is the right thing to do, even if it is sometimes annoying. But the reasons for doing do don't apply to explicit assignment expressions.

We have no obligation to protect people from every accidental name clobbering caused by carelessness and poor choice of names.

py> import math as module, string as module, sys as module py> module <module 'sys' (built-in)>

I suppose we need to go back and look at the "real" examples of where/how folks think they'll use := in comprehensions, and see how confusing it may be.

One the simplest but most useful examples is as a debugging aide.

[function(x) for x in whatever]

crashes part of the way through and raises an exception. How to debug? If it were a for-loop, one simple solution would be to wrap it in a try...except and then print the last seen value of x.

try: for x in whatever: function(x) except: print(x)

Can't do that with a list comprehension, not in Python 3. (It works in Python 2.) Assignment expressions to the rescue!

try: [function(a := x) for x in whatever] except: print(a)

That's simple and lightweight enough that with a bit of polishing, you could even leave it in production code to log a hard-to-track down error.

Oh, and it doesn't easily translate to a map(), since lambda functions do execute in their own scope:

map(lambda x: function(a := x), whatever)

Another reason to prefer comprehensions to map.

One of these conversations was started with an example something like this:

[(f(x), g(f(x))) for x in an_iterable]

The OP didn't like having to call f() twice. So that would become:

[ (temp:=f(x), g(temp)) for x in an_iterable]

so now the question is: should "temp" be created / changed in the enclosing local scope?

The comprehension here is a red herring.

result = (temp := f(x), g(temp))

Should temp be a local, or should tuple displays exist in their own, special, sublocal scope?

In either case, there's no *specific* advantage to letting temp "leak" (scare quotes) out of the expression. But there's no disadvantage either, and if we pick a less prejudicial name, it might actually be advantagous:

result = (useful_data := f(x), g(useful_data)) process(useful_data)

Using a comprehension isn't special enough to change the rule that assignment expressions are local (unless declared global).

On 6/30/2018 5:35 AM, Steven D'Aprano wrote:

I've given reasons why I believe that people will expect assignments in comprehensions to occur in the local scope. Aside from the special case of loop variables, people don't think of comprehensions as a separate scope.

I think this is because comprehensions other than generator expressions were originally *defined* in terms of equivalent code in the *same* local scope, are still easily thought of in those terms, and, as I explained in my response to Guido, could, at least in simple cases, still be implemented in the local scope, so that assignment expressions would be executed and affect the expected local scope without 'nonlocal'.

Generator expressions, on the other hand, have always been defined in terms of equivalent code in a *nested* scope, and must be implemented that way, so some re-definition and re-implementation is needed for assignment expressions to affect the local scope in which the g.e is defined and for that effect to be comprehensible. It might be enough to add something like "any names that are targets of assignment expressions are added to global or nonlocal declarations within the implementation generator function."

If the equality [generator-expression] == list(generator-expression] is preserved, then it could serve as the definition of the list comprehension. The same could be true for set and dict comprehensions, with the understanding that the equality is modified to {a:b for ...} == dict((a,b) for ...). It should also be mentioned that the defining equality is not necessarily the implementation.