On 06/02/2020 6:30 pm, Terry Reedy wrote:
On 2/6/2020 1:28 AM, Brandt Bucher wrote:
Commits 13bc139 and 8a4cd70 introduced subtle changes in the evaluation logic of unpacking operations. Previously, all elements were evaluated prior to being collected in a container. Now, these operations are interleaved. For example, the code `[*a, *b]` used to evaluate in the order `a` -> `b` -> `a.__iter__()` -> `b.__iter__()`. Now, it evaluates as `a` -> `a.__iter__()` -> `b` -> `b.__iter__()`.
A simpler example, which sharpens the contrast, is [*a, b]. The unpacking of *b is last either way. The change is from eval(a), eval(b), extend(a.__iter__()), append b to eval(a), extend(a.__iter__()), eval(b), append b
I believe this breaking semantic change is a bug, and I've opened a PR to fix it (https://github.com/python/cpython/pull/18264).
I carefully read and considered the original issue and the discussion on the PR and agree with the intent of the PR.
The semantic change can have visible effects due to interaction of side-effects. Examples on the PR are 1. a.__iter__ raising while b prints ([*None, print('executed)]), and 2. a and b both involving the same iterator (*it, next(it))
These previously unannounced semantic changes are apparently gratuitous side-effects of an internal refactoring. They should only be made, if at all, after discussion and agreement, then announcement and a deprecation period. But I seen no reason to change the status quo semantics.
These changes were unannounced because I didn't realize the current implementation was broken. There were no tests for raising an exception in the middle of unpacking a list, and it didn't occur to me to add them.
My reasoning is that "unary *" isn't an operator; it doesn't appear on the operator precedence table in the docs, and you can't evaluate `*x`. Like the brackets and the comma, it's part of the syntax of the outer display expression, not the inner one. It specifies how the list should be built, so it should be evaluated last, as part of the list construction. And it has always been this way since PEP 448 (as far as I can tell).
I agree that '*a' is not an expression in the meaning relevant here. https://docs.python.org/3/glossary.html says "A piece of syntax which can be evaluated to some value." This is the common math/logic/CS meaning. '*a' cannot be evaluated to a Python object. It is not an 'expression statement and cannot be passed to eval().
In the python grammar, an 'expression' is a 'starred_item' but a 'starred_item' need not be an expression.
starred_item ::= expression | "*" or_expr expression ::= conditional_expression | lambda_expr conditional_expression ::= or_test ["if" or_test "else" expression]
'*a' is a 'starred_item' but not an 'expression'.
I don't know where you got that grammar from, but not GitHub https://github.com/python/cpython/blob/master/Grammar/Grammar#L142
The docs themselves seem to support this line of reasoning (https://docs.python.org/3/reference/expressions.html#evaluation-order):
In the following lines, expressions will be evaluated in the arithmetic order of their suffixes: ... expr1(expr2, expr3, *expr4, **expr5)
Note that the stars are not part of expressions 1-5, but are a part of the top-level call expression that operates on them all.
Mark Shannon disagrees with me (I'll let him reply rather than attempt to summarize his argument for him), but we figured it might be better to get more input here on exactly whether you all think the behavior should change or not. You can see the discussion on the PR itself for some additional points and context.