[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/28/2018 11:21 PM, Tim Peters wrote:

[somewhere below] this is the last time I'm going to repeat it all again ;-)

For me, this is your most convincing exposition and summary of why the 
proposal is at least ok.  Thank you.


> [Chris]
>  > yes, it was a contrived example, but the simplest one I could think 
> of off
>  > the top of my head that re-bound a name in the loop -- which was what I
>  > thought was the entire point of this discussion?
> 
> But why off the top of your head?  There are literally hundreds & 
> hundreds of prior messages about this PEP, not to mention that you could 
> also find examples in the PEP.  Why make up a senseless example?
> 
>  > If we think hardly anyone is ever going to do that -- then I guess it 
> doesn't matter
>  > how it's handled.
> 
> So look at real examples.  One that's been repeated at least a hundred 
> times wants a local to "leak into" a listcomp:
> 
> total = 0
> cumsums = [total ::= total + value for value in data]
> 
> As an educator, how are you going to explain that blowing up with 
> UnboundLocalError instead?  Do you currently teach that comprehensions 
> and genexps are implemented via invisible magically generated lexically 
> nested functions?  If not, you're going to have to start for people to 
> even begin to make sense of UnboundLocalError if `total` _doesn't_ "leak 
> into" that example.  My belief is that just about everyone who doesn't 
> know "too much" about the current implementation will be astonished & 
> baffled if that example doesn't "just work".
> 
> In other cases it's desired that targets "leak out":
> 
> while any(n % (divisor := p) == 0 for p in small_primes):
>      n //= divisor
> 
> And in still other cases no leaking (neither in nor out) is desired.
> 
> Same as `for` targets in that way,. but in the opposite direction:  they 
> don't leak and there's no way to make them leak, not even when that's 
> wanted.  Which _is_ wanted in the last example above, which would be 
> clearer still written as:
> 
> while any(n % p == 0 for p in small_primes):
>      n //= p
> 
> But that ship has sailed.
> 
> 
>  > ...
>  > And "nonlocal" is not used that often, and when it is it's for 
> careful closure
>  > trickery -- I'm guessing := will be far more common.
> 
> My guess (recorded in the PEP's Appendix A) is that assignment 
> expressions _overall_ will be used more often than ternary `if` but 
> significantly less often than augmented assignment.  I expect their use 
> in genexps and comprehensions will be minimal.  There are real use cases 
> for them, but the vast majority of genexps and comprehensions apparently 
> have no use for them at all.
> 
> 
>  > And, of course, when a newbie encounters it, they can google it and 
> see what
>  > it means -- far different that seeing a := in a comprehension and 
> understanding
>  > (by osmosis??) that it might make changes in the local scope.
> 
> Which relates to the above:  how do you teach these things?  The idea 
> that "a newbie" even _suspects_ that genexps and listcomps have 
> something to do with lexically nested scopes and invisible nested 
> functions strikes me as hilarious ;-)
> 
> Regardless of how assignment expressions work in listcomps and genexps, 
> this example (which uses neither) _will_ rebind the containing block's `x`:
> 
> [x := 1]
> 
> How then are you going to explain that this seemingly trivial variation 
> _doesn't_?
> 
> [x := 1 for ignore in "a"]
> 
> For all the world they both appear to be binding `x` in the code block 
> containing the brackets.  So let them.
> 
> Even worse,
> 
> [x for ignore in range(x := 1)]
> 
> will rebind `x` in the containing block _regardless_ of how assignment 
> expression targets are treated in "most of" a comprehension, because the 
> expression defining the iterable of the outermost "for" _is_ evaluated 
> in the containing block (it is _not_ evaluated in the scope of the 
> synthetic function).
> 
> That's not a special case for targets if they all "leak", but is if they 
> don't.
> 
> 
>  > And  I don't think you can even do that with generator expressions 
> now -- as
>  > they can only contain expressions.
> 
> Expressions can invoke arbitrary functions, which in turn can do 
> anything whatsoever.
> 
>  > Which is my point -- this would allow the local namespace to be 
> manipulated
>  > in places it never could before.
> 
> As above, not true.  However, it would make it _easier_ to write 
> senseless code mucking with the local namespace - if that's what you 
> want to do.
> 
> 
>  > Maybe it's only comprehensions, and maybe it'll be rare to have a 
> confusing
>  > version of those, so it'll be no big deal, but this thread started 
> talking about
>  > educators' take on this -- and as an educator, I think this really does
>  > complicate the language.
> 
> I'll grant that it certainly doesn't simplify the language ;-)
> 
> 
>  > Python got much of it's "fame" by being "executable pseudo code" -- 
> its been
>  > moving farther and farther away from those roots. That's generally a 
> good thing,
>  > as we've gain expressiveness in exchangel, but we shouldn't pretend 
> it isn't
>  > happening, or that this proposal doesn't contribute to that trend.
> 
> I didn't say a word about that one way or the other.  I mostly agree, 
> but at the start Guido was aiming to fill a niche between shell 
> scripting languages and C.  It was a very "clean" language from the 
> start, but not aimed at beginners.  Thanks to his experience working on 
> ABC, it carried over some key ideas that were beginner-friendly, though.
> 
> I view assignment expressions as being aimed at much the same audience 
> as augmented assignments:  experienced programmers who already know the 
> pros and cons from vast experience with them in a large number of other 
> widely used languages.  That's also a key Python audience.
> 
>  > ...
>  > Well, I've been surprised by what confused students before, and I 
> will again. But I
>  > dont hink there is any doubt that Python 3.7 is a notably harder to 
> learn that
>  > Python 1.5 was...
> 
> Absolutely.  It doesn't much bother me, though - at this point the 
> language and its widely used libraries are so sprawling that I doubt 
> anyone is fluent in all of it.  That's a sign of worldly success.
> 
>  > ...
>  > and this:
>  >
>  > In [55]: x = 0
>  > In [56]: [x for x in range(3)]
>  > Out[56]: [0, 1, 2]
>  >In [57]: x
>  > Out[57]: 0
>  >
>  > doesn't change x in the local scope --
> 
> In Python 3, yes; in Python 2 it rebinds `x` to 2.
> 
>  > if that was a good idea, why is a good idea
>  >  to have := in a comprehension effect the local scope??
> 
> Because you can't write a genexp or comprehension AT ALL without 
> specifying `for` targets, and in the overwhelming majority of genexps 
> and comprehensions anyone ever looked at, "leaking" of for-targets was 
> not wanted.  "So don't let them leak" was pretty much a no-brainer for 
> Python 3.
> 
> But assignment expressions are NEVER required to write a genexp or 
> comprehension, and there are only a handful of patterns known so far in 
> which assignment expressions appear to be of real value in those 
> contexts.  In at least half those patterns, leaking _is_ wanted - 
> indeed, essential.  In the rest, leaking isn't.
> 
> So it goes.  Also don't ignore other examples given before, showing how 
> having assignment expressions _at all_ argues for "leaking" in order to 
> be consistent with what assignment expressions do outside of 
> comprehensions and genexps.
> 
> 
>  > But maybe it is just me.
> 
> Nope.  But it has been discussed so often before this is the last time 
> I'm going to repeat it all again ;-)



>  > ...
>  > One of these conversations was started with an example something like 
> this:
>  >
>> [(f(x), g(f(x))) for x in an_iterable]
>  >
>  > The OP didn't like having to call f() twice. So that would become:
>  >
>> [ (temp:=f(x), g(temp)) for x in an_iterable]
>  >
>  > so now the question is: should "temp" be created / changed in the 
> enclosing local scope?
>  >
>  > This sure looks a lot like letting the iteration name (x in this 
> example) leak out -
>  > so I'd say no.
> 
> In that example, right, leaking `temp` almost certainly isn't wanted.  
> So it goes.
> 
> 
>  > And I don't think this kind of thing would be rare.
> 
> I do.  It's dead easy to make up examples to "prove" anything people 
> like, but I'm unswayed unless examples come from real code, or are 
> obviously compelling.
> 
> Since we're not going to get a way to explicitly say which targets 
> (neither `for` nor assignment expression) do and don't leak, it's a 
> reasonably satisfying compromise to say that one kind never leaks and 
> the other kind always leaks.  The pick your poison accordingly.
> 
> In the example above, note that they _could_ already do, e.g.,
> 
>     [(fx, g(fx)) for x in an_iterable for fx in [f(x)]]
> 
> Then nothing leaks (well, unless f() or g() do tricky things).  I 
> personally wouldn't care that `temp` leaks - but then I probably would 
> have written that example as the shorter  (& clearer to my eyes):
> 
>      [(v. g(v)) for v in map(f, an_iterable)]
> 
> to begin with.


-- 
Terry Jan Reedy