[Python-Dev] bug in grammar

[Guido van Rossum]

> As part of the implementation of PEP 227 (and in an attempt to reach
> some low-hanging fruit Guido mentioned on the types-sig long ago), I
> have been working on a compiler pass that generates a module-level
> symbol table.  I recently discovered a bug in the handling of list
> comprehensions that was giving me headaches.
> 
> I realize now that the problem is with the current grammar and/or
> compiler.  Here's a simple demonstration; try it in your friendly
> python 2.0 interpreter.
> 
> >>> [i for i in range(10)] = (1, 2, 3)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ValueError: unpack list of wrong size
> 
> The generated bytecode is:
> 
>           0 SET_LINENO               0
> 
>           3 SET_LINENO               1
>           6 LOAD_CONST               0 (1)
>           9 LOAD_CONST               1 (2)
>          12 LOAD_CONST               2 (3)
>          15 BUILD_TUPLE              3
>          18 UNPACK_SEQUENCE          1
>          21 STORE_NAME               0 (i)
>          24 LOAD_CONST               3 (None)
>          27 RETURN_VALUE        
> 
> I assume this isn't intended :-).  The compiler is ignoring everything
> after the initial atom in the list comprehension.  It's basically
> compiling the code as if it were:
> 
> [i] = (1, 2, 3)
> 
> I'm not sure how to try and fix this.  Should the grammar allow one to
> construct the example statement above?  If not, I'm not sure how to
> fix the grammar.  If not, I suppose the compiler should detect that
> the list comp is misplaced.  This seems fairly messy, since there are
> about 10 nodes between the expr_stmt and the list_for.
> 
> Or is this a cool way to use list comprehensions to generate
> ValueErrors?

Good catch!  Not everything cool deserves to be preserved.

It looks like this happens because the code that traverses lists on
the left-hand side of an assignment was never told about list
comprehensions.  You're right that the grammar can't be fixed; it's
for the same reason that it can't be fixed to disallow "f() = 1".

The solution is to add a test for this to the compiler that flags this
as an error.

--Guido van Rossum (home page: http://www.python.org/~guido/)