Disallow ambiguous syntax f(x for x in [1],)
Initially generator expressions always had to be written inside parentheses, as documented in PEP 289 [1]. The additional parenthesis could be omitted on calls with only one argument, because in this case the generator expression already is written inside parentheses. You could write just `list(x for x in [1])` instead of `list((x for x in [1]))`. The following code was an error:
list(x for x in [1], *[]) File "<stdin>", line 1 SyntaxError: invalid syntax list(x for x in [1],) File "<stdin>", line 1 SyntaxError: invalid syntax
You needed to add explicit parenthesis in these cases:
list((x for x in [1]), *[]) [1] list((x for x in [1]),) [1]
But in Python 2.5 the following examples were accepted:
list(x for x in [1], *[]) [1] list(x for x in [1], *{}) [1] list(x for x in [1],) [1]
However I haven't found anything about this change in the "What's New In Python 2.5" document [2]. The former two cases were found to be a mistake and it was fixed in Python 3.5.
list(x for x in [1], *[]) File "<stdin>", line 1 SyntaxError: Generator expression must be parenthesized if not sole argument list(x for x in [1], *{}) File "<stdin>", line 1 SyntaxError: Generator expression must be parenthesized if not sole argument
But `list(x for x in [1],)` still is accepted. I think it would be better it this raises a SyntaxError. 1. This syntax is ambiguous, because at first look it is not clear whether it is equivalent to `list((x for x in [1]),)` or to `list(x for x in ([1],))`. 2. It is bad from the aesthetic point of view, because this is the only case when the generator expression has not written inside parentheses. I believe that allowing to omit parenthesis in a call with a single generator expression argument was caused by aesthetic reasons. 3. I believe the trailing comma in function call was allowed because this simplified adding, removing and commenting out arguments. func(first_argument, second_argument, #third_argument, ) You shouldn't touch other lines by adding or removing a comma when add or remove arguments. But this reason is not applicable to the case of `list((x for x in [1]),)`, because the generator expression without parenthesis should be the only argument. Therefore there is no reasons to allow this syntax. 4. 2to3 didn't supported this syntax for recent times [4]. Finally it was changed, but I think that it would be better to disallow this syntax for reasons mentioned above. [1] https://www.python.org/dev/peps/pep-0289/ [2] https://docs.python.org/2.5/whatsnew/whatsnew25.html [3] https://docs.python.org/3.5/whatsnew/3.5.html#changes-in-python-behavior [4] https://bugs.python.org/issue27494
Sounds good to me. On Sun, Nov 12, 2017 at 7:17 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
Initially generator expressions always had to be written inside parentheses, as documented in PEP 289 [1]. The additional parenthesis could be omitted on calls with only one argument, because in this case the generator expression already is written inside parentheses. You could write just `list(x for x in [1])` instead of `list((x for x in [1]))`. The following code was an error:
list(x for x in [1], *[]) File "<stdin>", line 1 SyntaxError: invalid syntax list(x for x in [1],) File "<stdin>", line 1 SyntaxError: invalid syntax
You needed to add explicit parenthesis in these cases:
list((x for x in [1]), *[]) [1] list((x for x in [1]),) [1]
But in Python 2.5 the following examples were accepted:
list(x for x in [1], *[]) [1] list(x for x in [1], *{}) [1] list(x for x in [1],) [1]
However I haven't found anything about this change in the "What's New In Python 2.5" document [2].
The former two cases were found to be a mistake and it was fixed in Python 3.5.
list(x for x in [1], *[]) File "<stdin>", line 1 SyntaxError: Generator expression must be parenthesized if not sole argument list(x for x in [1], *{}) File "<stdin>", line 1 SyntaxError: Generator expression must be parenthesized if not sole argument
But `list(x for x in [1],)` still is accepted. I think it would be better it this raises a SyntaxError.
1. This syntax is ambiguous, because at first look it is not clear whether it is equivalent to `list((x for x in [1]),)` or to `list(x for x in ([1],))`.
2. It is bad from the aesthetic point of view, because this is the only case when the generator expression has not written inside parentheses. I believe that allowing to omit parenthesis in a call with a single generator expression argument was caused by aesthetic reasons.
3. I believe the trailing comma in function call was allowed because this simplified adding, removing and commenting out arguments.
func(first_argument, second_argument, #third_argument, )
You shouldn't touch other lines by adding or removing a comma when add or remove arguments. But this reason is not applicable to the case of `list((x for x in [1]),)`, because the generator expression without parenthesis should be the only argument. Therefore there is no reasons to allow this syntax.
4. 2to3 didn't supported this syntax for recent times [4]. Finally it was changed, but I think that it would be better to disallow this syntax for reasons mentioned above.
[1] https://www.python.org/dev/peps/pep-0289/ [2] https://docs.python.org/2.5/whatsnew/whatsnew25.html [3] https://docs.python.org/3.5/whatsnew/3.5.html#changes-in-pyt hon-behavior [4] https://bugs.python.org/issue27494
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% 40python.org
-- --Guido van Rossum (python.org/~guido)
12.11.17 18:57, Guido van Rossum пише:
Sounds good to me.
Thanks! Here is an implementation: https://bugs.python.org/issue32012. I have found that formally trailing comma after generator expression is not allowed by the grammar defined in the language reference: call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" But the actual Grammar file contains different rules.
It's hard to keep those two in sync, since the actual Grammar file is constrained by being strictly LL(1)... Can you get someone else to review the implementation?
13.11.17 02:00, Guido van Rossum пише:
It's hard to keep those two in sync, since the actual Grammar file is constrained by being strictly LL(1)... Can you get someone else to review the implementation?
I haven't change the grammar, just have changed checks in the CST to AST transformer. Maybe it would be better to change the grammar, but I'm not expert in this. Maybe Benjamin could provide better solution. There are other related differences between language specification and the implementation. The following examples are valid syntax: @deco(x for x in [1]) def f(): ... class C(x for x in [1]): ... The latter always raises a TypeError at runtime ("cannot create 'generator' instances"), but is compiled successfully.
FWIW, it is common to have syntax checks in ast.c. Especially situations like class ``C(x for x in [1]): ...`` I think would be hard to prohibit in Grammar. Since anyway we have many checks in ast.c already, I wouldn't care much about implementing these corner cases in Grammar. -- Ivan
participants (3)
-
Guido van Rossum -
Ivan Levkivskyi -
Serhiy Storchaka