Odd name shadowing in comprehension

[Steve D'Aprano]

On Sun, 23 Oct 2016 10:57 am, Chris Angelico wrote:

> This surprised me.
> 
> Python 3.4.2 (default, Oct  8 2014, 10:45:20)
> [GCC 4.9.1] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> y=6
>>>> [(x,y) for x in range(y) for y in range(3)]
> [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2,
> 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2), (5, 0), (5, 1),
> (5, 2)]

That surprises me too. I wouldn't have expected that y is both global and
local to the comprehension at the same time.

I think that this is a bug in list comprehensions and should at least give a
warning that y is being used as both local and non-local.


This is what happens if you try to access a local before it exists:

py> [(y, x) for x in (1, y) for y in (10, 20)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined


Swap the order, and it works:

py> [(y, x) for y in (10, 20) for x in (1, y)]
[(10, 1), (10, 10), (20, 1), (20, 20)]


But if y happens to exist as a global, the first version mysteriously works!

py> y = 999
py> [(y, x) for x in (1, y) for y in (10, 20)]
[(10, 1), (20, 1), (10, 999), (20, 999)]


Let's expand the list comp to a function:

y = 999
def list_comp():
    result = []
    for x in (1, y):
        for y in (10, 20):
            result.append((y, x))
    return result


Calling that function gives a NameError, specifically:

UnboundLocalError: local variable 'y' referenced before assignment


> Normally, a comprehension is described as being equivalent to an
> unrolled loop, inside a nested function. 

I don't think that's quite the right description, but something like that.

> That would be like this: 
> 
> def temp():
>     ret = []
>     for x in range(y):
>         for y in range(3):
>             ret.append((x,y))
>     return ret
> temp()

Indeed.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.