[ python-Bugs-1167751 ] Argument genexp corner case

[SourceForge.net]

Bugs item #1167751, was opened at 2005-03-21 09:47
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167751&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.5
Status: Closed
Resolution: Fixed
Priority: 8
Submitted By: John Ehresman (jpe)
Assigned to: Neal Norwitz (nnorwitz)
Summary: Argument genexp corner case

Initial Comment:
The following raises an unexpected exception; I would
expect it to either work or raise a SyntaxError.  It
seems that the grammar parses it, but the compiler does
not do the right thing.

>>> def foo(a): pass

>>> foo(a = i for i in range(10))

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
NameError: name 'i' is not defined

foo(a = (i for i in range(10)) works.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-22 17:44

Message:
Logged In: YES 
user_id=33168

Ok.  Backported.

----------------------------------------------------------------------

Comment By: Anthony Baxter (anthonybaxter)
Date: 2005-10-21 00:55

Message:
Logged In: YES 
user_id=29957

I can't see a problem with backporting this to 2.4


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-20 23:27

Message:
Logged In: YES 
user_id=33168

Checked in as:
 * Python/graminit.c 2.41
 * Lib/test/test_genexps.py 1.10
 * Grammar/Grammar 1.55
 * Misc/NEWS 1.1392 and 1.1391

Leaving it up to Anthony whether this should be backported.

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2005-10-11 06:43

Message:
Logged In: YES 
user_id=1038590

The problem is definitely on the parser end though:

Py> compiler.parse("foo(x=i for i in range(10))")
Module(None, Stmt([Discard(CallFunc(Name('foo'),
[Keyword('x', Name('i'))], None, None))]))

It's getting to what looks like a valid keyword argument in
"x=i" and throwing the rest of it away, when it should be
flagging a syntax error (the parser's limited lookahead
should be enough to spot the erroneous 'for' keyword and
bail out).

The problem's actually worse than the OP noted: consider
what will happen if there is a variable "i" visible from the
location of the function call (e.g. from a list
comprehension or for loop in a nested scope). Good luck
tracking that one down. . .


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-10 21:10

Message:
Logged In: YES 
user_id=33168

I definitely agree this is a big problem.

Here's what the code above generates:
2           0 LOAD_GLOBAL              0 (foo)
              3 LOAD_CONST               1 ('a')
              6 LOAD_GLOBAL              1 (i)
              9 CALL_FUNCTION          256
             12 POP_TOP
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE

If I put parens around the genexp, I get:
2           0 LOAD_GLOBAL              0 (foo)
              3 LOAD_CONST               1 ('a')
              6 LOAD_CONST               2 (<code object
<generator expression> at 0x2a960baae8, file "<stdin>", line 2>)
              9 MAKE_FUNCTION            0
             12 LOAD_GLOBAL              1 (range)
             15 LOAD_CONST               3 (10)
             18 CALL_FUNCTION            1
             21 GET_ITER
             22 CALL_FUNCTION            1
             25 CALL_FUNCTION          256
             28 POP_TOP
             29 LOAD_CONST               0 (None)
             32 RETURN_VALUE


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167751&group_id=5470