Nick Coghlan wrote:

Neal Norwitz wrote:

...

There's a problem with genexp's that I think really needs to get fixed. See http://python.org/sf/1167751 the details are below. This code: I agree with the bug report that the code should either raise a SyntaxError or do the right thing.

I agree it should be a SyntaxError - I believe the AST compiler actually raises one in this situation.

I was half right. Both the normal compiler and the AST compiler give a SyntaxError if you write: foo((a=i for i in range(10))) The problem is definitely on the parser end though: Py> compiler.parse("foo(x=i for i in range(10))") Module(None, Stmt([Discard(CallFunc(Name('foo'), [Keyword('x', Name('i'))], None, None))])) It's getting to what looks like a valid keyword argument in "x=i" and throwing the rest of it away, when it should be flagging a syntax error (the parser's limited lookahead should still be enough to spot the erroneous 'for' keyword and bail out). The error will be even more obscure if there is an "i" visible from the location of the function call. Whereas when it's parenthesised correctly, the parse tree looks more like this: Py> compiler.parse("foo(x=(i for i in range(10)))") Module(None, Stmt([Discard(CallFunc(Name('foo'), [Keyword('x', GenExpr(GenExprInner(Name('i'), [GenExprFor(AssName('i', 'OP_ASSIGN'), CallFunc(Name('range'), [Const(10)], None, None), [])])))], None, None))])) Cheers, Nick. P.S. I added a comment showing the parser output to the SF bug report. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 10/16/05, Neal Norwitz wrote:

On 10/10/05, Neal Norwitz wrote:

...

There's a problem with genexp's that I think really needs to get fixed. See http://python.org/sf/1167751 the details are below. This code:

...

...

...

foo(a = i for i in range(10))

I agree with the bug report that the code should either raise a SyntaxError or do the right thing.

The change to Grammar/Grammar below seems to fix the problem and all the tests pass. Can anyone comment on whether this fix is correct/appropriate? Is there a better way to fix the problem?

-argument: [test '='] test [gen_for] # Really [keyword '='] test +argument: test [gen_for] | test '=' test ['(' gen_for ')'] # Really [keyword '='] test

The other option would be changes in the Python/compile.c (somewhat) like following diff -r2.352 compile.c 6356c6356,6362 - if (TYPE(n) == argument && NCH(n) == 3) { --- + if (TYPE(n) == argument && NCH(n) == 4) { + PyErr_SetString(PyExc_SyntaxError, + "invalid syntax"); + symtable_error(st, n->n_lineno); + return; + } + else if (TYPE(n) == argument && NCH(n) == 3) { but IMO, changing the Grammar looks more obvious.

n _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com

Regarding this Grammar change; (last October) from argument: [test '=' ] test [gen_for] to argument: test [gen_for] | test '=' test ['(' gen_for ')'] - to raise error for "bar(a = i for i in range(10)) )" I think we should change it to argument: test [gen_for] | test '=' test instead of argument: test [gen_for] | test '=' test ['(' gen_for ')'] that is, without ['(' gen_for ')'] . We don't need that extra term, because "test" itself includes generator expressions - with all those parensises. Actually with that extra ['(' gen_for ')'] , foo(a= 10 (for y in 'a')) is grammartically correct ; although that error seems to be checked elsewhere. I tested without ['(' gen_for ')'] , and worked fine passing Lib/test/test_genexps.py -Jiwon On 10/20/05, Neal Norwitz wrote:

On 10/16/05, Neal Norwitz wrote:

...

On 10/10/05, Neal Norwitz wrote:

...

There's a problem with genexp's that I think really needs to get fixed. See http://python.org/sf/1167751 the details are below. This code:

...

...

...

foo(a = i for i in range(10))

I agree with the bug report that the code should either raise a SyntaxError or do the right thing.

The change to Grammar/Grammar below seems to fix the problem and all the tests pass. Can anyone comment on whether this fix is correct/appropriate? Is there a better way to fix the problem?

Since no one responded other than Jiwon, I checked in this change. I did *not* backport it since what was syntactically correct in 2.4.2 would raise an error in 2.4.3. I'm not sure which is worse. I'll leave it up to Anthony whether this should be backported.

BTW, the change was the same regardless of old code vs. new AST code.

n _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com