[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Chris]
> yes, it was a contrived example, but the simplest one I could think of off
> the top of my head that re-bound a name in the loop -- which was what I
> thought was the entire point of this discussion?

But why off the top of your head?  There are literally hundreds & hundreds
of prior messages about this PEP, not to mention that you could also find
examples in the PEP.  Why make up a senseless example?

> If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter
> how it's handled.

So look at real examples.  One that's been repeated at least a hundred
times wants a local to "leak into" a listcomp:

total = 0
cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with
UnboundLocalError instead?  Do you currently teach that comprehensions and
genexps are implemented via invisible magically generated lexically nested
functions?  If not, you're going to have to start for people to even begin
to make sense of UnboundLocalError if `total` _doesn't_ "leak into" that
example.  My belief is that just about everyone who doesn't know "too much"
about the current implementation will be astonished & baffled if that
example doesn't "just work".

In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes):
    n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction:  they
don't leak and there's no way to make them leak, not even when that's
wanted.  Which _is_ wanted in the last example above, which would be
clearer still written as:

while any(n % p == 0 for p in small_primes):
    n //= p

But that ship has sailed.


> ...
> And "nonlocal" is not used that often, and when it is it's for careful
closure
> trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment expressions
_overall_ will be used more often than ternary `if` but significantly less
often than augmented assignment.  I expect their use in genexps and
comprehensions will be minimal.  There are real use cases for them, but the
vast majority of genexps and comprehensions apparently have no use for them
at all.


> And, of course, when a newbie encounters it, they can google it and see
what
> it means -- far different that seeing a := in a comprehension and
understanding
> (by osmosis??) that it might make changes in the local scope.

Which relates to the above:  how do you teach these things?  The idea that
"a newbie" even _suspects_ that genexps and listcomps have something to do
with lexically nested scopes and invisible nested functions strikes me as
hilarious ;-)

Regardless of how assignment expressions work in listcomps and genexps,
this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

How then are you going to explain that this seemingly trivial variation
_doesn't_?

[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block
containing the brackets.  So let them.

Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment
expression targets are treated in "most of" a comprehension, because the
expression defining the iterable of the outermost "for" _is_ evaluated in
the containing block (it is _not_ evaluated in the scope of the synthetic
function).

That's not a special case for targets if they all "leak", but is if they
don't.


> And  I don't think you can even do that with generator expressions now --
as
> they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do anything
whatsoever.

> Which is my point -- this would allow the local namespace to be
manipulated
> in places it never could before.

As above, not true.  However, it would make it _easier_ to write senseless
code mucking with the local namespace - if that's what you want to do.


> Maybe it's only comprehensions, and maybe it'll be rare to have a
confusing
> version of those, so it'll be no big deal, but this thread started
talking about
> educators' take on this -- and as an educator, I think this really does
> complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)


> Python got much of it's "fame" by being "executable pseudo code" -- its
been
> moving farther and farther away from those roots. That's generally a good
thing,
> as we've gain expressiveness in exchangel, but we shouldn't pretend it
isn't
> happening, or that this proposal doesn't contribute to that trend.

I didn't say a word about that one way or the other.  I mostly agree, but
at the start Guido was aiming to fill a niche between shell scripting
languages and C.  It was a very "clean" language from the start, but not
aimed at beginners.  Thanks to his experience working on ABC, it carried
over some key ideas that were beginner-friendly, though.

I view assignment expressions as being aimed at much the same audience as
augmented assignments:  experienced programmers who already know the pros
and cons from vast experience with them in a large number of other widely
used languages.  That's also a key Python audience.

> ...
> Well, I've been surprised by what confused students before, and I will
again. But I
> dont hink there is any doubt that Python 3.7 is a notably harder to learn
that
> Python 1.5 was...

Absolutely.  It doesn't much bother me, though - at this point the language
and its widely used libraries are so sprawling that I doubt anyone is
fluent in all of it.  That's a sign of worldly success.

> ...
> and this:
>
> In [55]: x = 0
> In [56]: [x for x in range(3)]
> Out[56]: [0, 1, 2]
>In [57]: x
> Out[57]: 0
>
> doesn't change x in the local scope --

In Python 3, yes; in Python 2 it rebinds `x` to 2.

> if that was a good idea, why is a good idea
>  to have := in a comprehension effect the local scope??

Because you can't write a genexp or comprehension AT ALL without specifying
`for` targets, and in the overwhelming majority of genexps and
comprehensions anyone ever looked at, "leaking" of for-targets was not
wanted.  "So don't let them leak" was pretty much a no-brainer for Python 3.

But assignment expressions are NEVER required to write a genexp or
comprehension, and there are only a handful of patterns known so far in
which assignment expressions appear to be of real value in those contexts.
In at least half those patterns, leaking _is_ wanted - indeed, essential.
In the rest, leaking isn't.

So it goes.  Also don't ignore other examples given before, showing how
having assignment expressions _at all_ argues for "leaking" in order to be
consistent with what assignment expressions do outside of comprehensions
and genexps.


> But maybe it is just me.

Nope.  But it has been discussed so often before this is the last time I'm
going to repeat it all again ;-)

> ...
> One of these conversations was started with an example something like
this:
>
> [(f(x), g(f(x))) for x in an_iterable]
>
> The OP didn't like having to call f() twice. So that would become:
>
> [ (temp:=f(x), g(temp)) for x in an_iterable]
>
> so now the question is: should "temp" be created / changed in the
enclosing local scope?
>
> This sure looks a lot like letting the iteration name (x in this example)
leak out -
> so I'd say no.

In that example, right, leaking `temp` almost certainly isn't wanted.  So
it goes.


> And I don't think this kind of thing would be rare.

I do.  It's dead easy to make up examples to "prove" anything people like,
but I'm unswayed unless examples come from real code, or are obviously
compelling.

Since we're not going to get a way to explicitly say which targets (neither
`for` nor assignment expression) do and don't leak, it's a reasonably
satisfying compromise to say that one kind never leaks and the other kind
always leaks.  The pick your poison accordingly.

In the example above, note that they _could_ already do, e.g.,

   [(fx, g(fx)) for x in an_iterable for fx in [f(x)]]

Then nothing leaks (well, unless f() or g() do tricky things).  I
personally wouldn't care that `temp` leaks - but then I probably would have
written that example as the shorter  (& clearer to my eyes):

    [(v. g(v)) for v in map(f, an_iterable)]

to begin with.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180628/727dd1f8/attachment-0001.html>