[Python-Dev] PEP 572: Assignment Expressions

[Chris Angelico]

On Sat, Apr 21, 2018 at 10:26 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, Apr 21, 2018 at 05:46:44PM +1000, Chris Angelico wrote:
>> On Sat, Apr 21, 2018 at 5:11 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>
>> > So can you explain specifically what odd function-scope behaviour you
>> > are referring to? Give an example please?
>>
>> doubled_items = [x for x in (items := get_items()) if x * 2 in items]
>>
>> This will leak 'items' into the surrounding scope (but not 'x').
>
> The "not x" part is odd, I agree, but it's a popular feature to have
> comprehensions run in a separate scope, so that's working as designed.
>
> The "leak items" part is the behaviour I desire, so that's not odd, it's
> sensible *wink*
>
> The reason I want items to "leak" into the surrounding scope is mostly
> so that the initial value for it can be set with a simple assignment
> outside the comprehension:
>
>     items = (1, 2, 3)
>     [ ... items := items*2 ... ]
>
> and the least magical way to do that is to just make items an ordinary
> local variable.

You can't have your cake and eat it too. Iteration variables and names
bound by assignment expressions are both set inside the comprehension.
Either they both are local, or they both leak - or else we have a
weird rule like "the outermost iterable is magical and special".

>> [x for x in x if x] # This works
>
> The oddity is that this does work, and there's no assignment expression
> in sight.
>
> Given that x is a local variable of the comprehension `for x in ...` it
> ought to raise UnboundLocalError, as the expanded equivalent does:
>
>
> def demo():
>     result = []
>     for x in x: # ought to raise UnboundLocalError
>         if x:
>             result.append(x)
>     return result
>
>
> That the comprehension version runs (rather than raising) is surprising
> but I wouldn't call it a bug. Nor would I say it was a language
> guarantee that we have to emulate in similar expressions.

See, that's the problem. That is NOT how the comprehension expands. It
actually expands to this:

def demo(it):
    result = []
    for x in it:
        if x:
            result.append(x)
    return result
demo(iter(x))

PEP 572 corrects this by making it behave the way that you, and many
other people, expect. Current behaviour is surprising because the
outermost iterable is special and magical.

>> (x for x in 5) # TypeError
>> (x for _ in [1] for x in 5) # Works
>
> Now that last one is more than just odd, it is downright bizarre. Or at
> least it would, if it did work:
>
> py> list((x for _ in [1] for x in 5))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 1, in <genexpr>
> TypeError: 'int' object is not iterable
>
>
> Are you sure about this example?

Yes, I'm sure. You may notice that I didn't iterate over the genexps
in my example. The first one will bomb out, even without iteration;
the second one gives a valid generator object which, if iterated over
(or even stepped once), will bomb. This is because, again, the
outermost iterable is special and magical.

> In any case, since this has no assignment expression in it, I don't see
> why it is relevant.

Because an assignment expression in the outermost iterable would, if
the semantics are preserved, bind in the surrounding scope. It would
be FAR more logical to have it bind in the inner scope. Consider these
two completely different results:

def f(*prefix):
    print([p + name for p in prefix for name in locals()])
    print([p + name for name in locals() for p in prefix])

>>> f("* ", "$ ")
['* .0', '* p', '$ .0', '$ p', '$ name']
['* prefix', '$ prefix']

The locals() as seen by the outermost iterable are f's locals, and any
assignment expression there would be part of f's locals. The locals()
as seen by any other iterable, by a condition, or by the primary
expression, are the list comp's locals, and any assignment expression
there would be part of the list comp's locals.

ChrisA