Allow tuple unpacking in return and yield statements
First time contributing back -- if I should be filing a PEP or something like that for this, please let me know. Coming from https://bugs.python.org/issue32117, unparenthesized tuple unpacking is allowed in assignments: rest = (4, 5, 6) a = 1, 2, 3, *rest but not in yield or return statements (these result in SyntaxErrors): return 1, 2, 3, *rest yield 1, 2, 3, *rest The unpacking in assignments was enabled by a pre-3.2 commit that I haven't yet been able to track back to a discussion, but I suspect this asymmetry is unintentional. Here's the original commit: https://github.com/python/cpython/commit/4905e80c3d2f6abb613d212f0313d1dfe09... I've submitted a patch (CLA is signed and submitted, not yet processed), and Serihy said that since it changes the grammar I should have it reviewed here and have signoff by the BDFL. While I haven't had a need for this myself, it was brought up by a user on StackOverflow (https://stackoverflow.com/questions/47272460/python-tuple-unpacking-in-retur...). Thanks! Dave
On 11/25/2017 1:55 AM, David Cuthbert wrote:
First time contributing back -- if I should be filing a PEP or something like that for this, please let me know.
I don't think a PEP is needed.
Coming from https://bugs.python.org/issue32117, unparenthesized tuple unpacking is allowed in assignments:
rest = (4, 5, 6) a = 1, 2, 3, *rest
Because except for (), it is ',', not '()' that makes a tuple a tuple.
but not in yield or return statements (these result in SyntaxErrors):
return 1, 2, 3, *rest yield 1, 2, 3, *rest
To be crystal clear, a parenthesized tuple with unpacking *is* valid. return (1, 2, 3, *rest) yield (1, 2, 3, *rest) So is an un-parenthesized tuple without unpacking. Since return and yield are often the first half of a cross-namespace assignment, requiring the () is a bit surprising. Perhaps someone else has a good reason for the difference. Otherwise, +1 on the change.
The unpacking in assignments was enabled by a pre-3.2 commit that I haven't yet been able to track back to a discussion, but I suspect this asymmetry is unintentional. Here's the original commit: https://github.com/python/cpython/commit/4905e80c3d2f6abb613d212f0313d1dfe09...
I've submitted a patch (CLA is signed and submitted, not yet processed), and Serihy said that since it changes the grammar I should have it reviewed here and have signoff by the BDFL.
While I haven't had a need for this myself, it was brought up by a user on StackOverflow (https://stackoverflow.com/questions/47272460/python-tuple-unpacking-in-retur...).
Thanks! Dave
-- Terry Jan Reedy
I think the proposal is reasonable and won't require a PEP. On Nov 25, 2017 3:25 PM, "Terry Reedy" <tjreedy@udel.edu> wrote:
On 11/25/2017 1:55 AM, David Cuthbert wrote:
First time contributing back -- if I should be filing a PEP or something like that for this, please let me know.
I don't think a PEP is needed.
Coming from https://bugs.python.org/issue32117, unparenthesized tuple
unpacking is allowed in assignments:
rest = (4, 5, 6) a = 1, 2, 3, *rest
Because except for (), it is ',', not '()' that makes a tuple a tuple.
but not in yield or return statements (these result in SyntaxErrors):
return 1, 2, 3, *rest yield 1, 2, 3, *rest
To be crystal clear, a parenthesized tuple with unpacking *is* valid.
return (1, 2, 3, *rest) yield (1, 2, 3, *rest)
So is an un-parenthesized tuple without unpacking.
Since return and yield are often the first half of a cross-namespace assignment, requiring the () is a bit surprising. Perhaps someone else has a good reason for the difference. Otherwise, +1 on the change.
The unpacking in assignments was enabled by a pre-3.2 commit that I
haven't yet been able to track back to a discussion, but I suspect this asymmetry is unintentional. Here's the original commit: https://github.com/python/cpython/commit/4905e80c3d2f6abb613 d212f0313d1dfe09475dc
I've submitted a patch (CLA is signed and submitted, not yet processed), and Serihy said that since it changes the grammar I should have it reviewed here and have signoff by the BDFL.
While I haven't had a need for this myself, it was brought up by a user on
StackOverflow (https://stackoverflow.com/questions/47272460/python-tuple- unpacking-in-return-statement/47326859).
Thanks! Dave
-- Terry Jan Reedy
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% 40python.org
On 26 November 2017 at 09:22, Terry Reedy <tjreedy@udel.edu> wrote:
Since return and yield are often the first half of a cross-namespace assignment, requiring the () is a bit surprising. Perhaps someone else has a good reason for the difference.
These kinds of discrepancies tend to arise because there are a few different grammar nodes for "comma separated sequence of expressions", which makes it possible to miss some when enhancing the tuple syntax. Refactoring the grammar to eliminate the duplication isn't especially easy, and we don't change the syntax all that often, so it makes sense to treat cases like this one as bugs in the implementation of the original syntax change (except that the "don't change the Grammar in maintenance releases" guideline means they still need to be handled as new features when it comes to fixing them). Cheers, Nick. P.S. That said, I do wonder if it might be feasible to write a "Grammar consistency check" test that ensured the known duplicate nodes at least have consistent definitions, such that missing one in a syntax update will cause an automated test failure. Unfortunately, the nodes typically haven't been combined because they have some *intentional* differences in exactly what they allow, so I also suspect that this is easier said than done. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Henk-Jaap noted that the grammar section of the language ref for yield and return should also be updated from expression_list to starred_list with this change. As noted elsewhere, this isn't in-sync with the Grammar file (intentionally, if I understand correctly). I took a look, and I believe that every instance of expression_list (which doesn't allow the unparenthesized tuple unpacking) should be changed to starred_list. Which might really mean that starred_list should have never existed, and the changes should have been put into expression_list in the first place (though I understand the desire to be conservative with syntax changes). Here are the places where expression_list is still allowed (after fixing return and yield): subscription ::= primary "[" expression_list "]" augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression) for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite] In other words, the following all produce SyntaxErrors today (and enclosing them in parentheses avoids this): a[1, *rest] a += 1, *rest # and other augops: -= *= /= etc. for i in 1, *rest: My hunch is these cases should also be fixed to be consistent. While I can't see myself using something like "a += 1, *rest" in the immediate future, it seems weird to be inconsistent in these cases (and reinforces the oft-mistaken assumption, from Terry's earlier reply, that tuples are defined by parentheses instead of commas). Any reason I shouldn't dig in and fix this while I'm here? Dave On 11/25/17, 9:03 PM, Nick Coghlan wrote: On 26 November 2017 at 09:22, Terry Reedy <tjreedy@udel.edu> wrote: > Since return and yield are often the first half of a cross-namespace > assignment, requiring the () is a bit surprising. Perhaps someone else has > a good reason for the difference. These kinds of discrepancies tend to arise because there are a few different grammar nodes for "comma separated sequence of expressions", which makes it possible to miss some when enhancing the tuple syntax. Refactoring the grammar to eliminate the duplication isn't especially easy, and we don't change the syntax all that often, so it makes sense to treat cases like this one as bugs in the implementation of the original syntax change (except that the "don't change the Grammar in maintenance releases" guideline means they still need to be handled as new features when it comes to fixing them). Cheers, Nick. P.S. That said, I do wonder if it might be feasible to write a "Grammar consistency check" test that ensured the known duplicate nodes at least have consistent definitions, such that missing one in a syntax update will cause an automated test failure. Unfortunately, the nodes typically haven't been combined because they have some *intentional* differences in exactly what they allow, so I also suspect that this is easier said than done. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, 30 Nov 2017 at 05:01 David Cuthbert <dacut@kanga.org> wrote:
Henk-Jaap noted that the grammar section of the language ref for yield and return should also be updated from expression_list to starred_list with this change. As noted elsewhere, this isn't in-sync with the Grammar file (intentionally, if I understand correctly).
I took a look, and I believe that every instance of expression_list (which doesn't allow the unparenthesized tuple unpacking) should be changed to starred_list. Which might really mean that starred_list should have never existed, and the changes should have been put into expression_list in the first place (though I understand the desire to be conservative with syntax changes).
Here are the places where expression_list is still allowed (after fixing return and yield):
subscription ::= primary "[" expression_list "]" augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression) for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite]
In other words, the following all produce SyntaxErrors today (and enclosing them in parentheses avoids this): a[1, *rest] a += 1, *rest # and other augops: -= *= /= etc. for i in 1, *rest:
My hunch is these cases should also be fixed to be consistent. While I can't see myself using something like "a += 1, *rest" in the immediate future, it seems weird to be inconsistent in these cases (and reinforces the oft-mistaken assumption, from Terry's earlier reply, that tuples are defined by parentheses instead of commas).
Any reason I shouldn't dig in and fix this while I'm here?
It's really a question of ramifications. Do we want every place where parentheses tuples are required to allow for the non-paren version? If there was a way to get an exhaustive list of examples showing what would change in those instances then we could make a judgement call as to whether this change is desired. -Brett
Dave
On 11/25/17, 9:03 PM, Nick Coghlan wrote:
On 26 November 2017 at 09:22, Terry Reedy <tjreedy@udel.edu> wrote: > Since return and yield are often the first half of a cross-namespace > assignment, requiring the () is a bit surprising. Perhaps someone else has > a good reason for the difference.
These kinds of discrepancies tend to arise because there are a few different grammar nodes for "comma separated sequence of expressions", which makes it possible to miss some when enhancing the tuple syntax.
Refactoring the grammar to eliminate the duplication isn't especially easy, and we don't change the syntax all that often, so it makes sense to treat cases like this one as bugs in the implementation of the original syntax change (except that the "don't change the Grammar in maintenance releases" guideline means they still need to be handled as new features when it comes to fixing them).
Cheers, Nick.
P.S. That said, I do wonder if it might be feasible to write a "Grammar consistency check" test that ensured the known duplicate nodes at least have consistent definitions, such that missing one in a syntax update will cause an automated test failure. Unfortunately, the nodes typically haven't been combined because they have some *intentional* differences in exactly what they allow, so I also suspect that this is easier said than done.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
I'd be wary of going too far here. The parser (which uses a different representation for the grammar) may not support all of these -- in particular I would guess that the syntax for subscriptions is actually more complicated than shown, because of slices. Do you allow a[x:y, *rest]? On Thu, Nov 30, 2017 at 10:27 AM, Brett Cannon <brett@python.org> wrote:
On Thu, 30 Nov 2017 at 05:01 David Cuthbert <dacut@kanga.org> wrote:
Henk-Jaap noted that the grammar section of the language ref for yield and return should also be updated from expression_list to starred_list with this change. As noted elsewhere, this isn't in-sync with the Grammar file (intentionally, if I understand correctly).
I took a look, and I believe that every instance of expression_list (which doesn't allow the unparenthesized tuple unpacking) should be changed to starred_list. Which might really mean that starred_list should have never existed, and the changes should have been put into expression_list in the first place (though I understand the desire to be conservative with syntax changes).
Here are the places where expression_list is still allowed (after fixing return and yield):
subscription ::= primary "[" expression_list "]" augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression) for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite]
In other words, the following all produce SyntaxErrors today (and enclosing them in parentheses avoids this): a[1, *rest] a += 1, *rest # and other augops: -= *= /= etc. for i in 1, *rest:
My hunch is these cases should also be fixed to be consistent. While I can't see myself using something like "a += 1, *rest" in the immediate future, it seems weird to be inconsistent in these cases (and reinforces the oft-mistaken assumption, from Terry's earlier reply, that tuples are defined by parentheses instead of commas).
Any reason I shouldn't dig in and fix this while I'm here?
It's really a question of ramifications. Do we want every place where parentheses tuples are required to allow for the non-paren version? If there was a way to get an exhaustive list of examples showing what would change in those instances then we could make a judgement call as to whether this change is desired.
-Brett
Dave
On 11/25/17, 9:03 PM, Nick Coghlan wrote:
On 26 November 2017 at 09:22, Terry Reedy <tjreedy@udel.edu> wrote: > Since return and yield are often the first half of a cross-namespace > assignment, requiring the () is a bit surprising. Perhaps someone else has > a good reason for the difference.
These kinds of discrepancies tend to arise because there are a few different grammar nodes for "comma separated sequence of expressions", which makes it possible to miss some when enhancing the tuple syntax.
Refactoring the grammar to eliminate the duplication isn't especially easy, and we don't change the syntax all that often, so it makes sense to treat cases like this one as bugs in the implementation of the original syntax change (except that the "don't change the Grammar in maintenance releases" guideline means they still need to be handled as new features when it comes to fixing them).
Cheers, Nick.
P.S. That said, I do wonder if it might be feasible to write a "Grammar consistency check" test that ensured the known duplicate nodes at least have consistent definitions, such that missing one in a syntax update will cause an automated test failure. Unfortunately, the nodes typically haven't been combined because they have some *intentional* differences in exactly what they allow, so I also suspect that this is easier said than done.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ brett%40python.org
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org
-- --Guido van Rossum (python.org/~guido)
participants (6)
-
Brett Cannon -
David Cuthbert -
Guido van Rossum -
Guido van Rossum -
Nick Coghlan -
Terry Reedy