On Tue, Mar 08, 2016 at 10:58:16PM +0000, Paul Moore wrote:
On 8 March 2016 at 22:21, Sjoerd Job Postmus sjoerdjob@sjec.nl wrote:
As for yet another syntax suggestion (if we want to introduce something).
[y for x in numbers with abs(x) as y if y > 5]The benefit is that it reads quite natural: "with expr as name". Another benefit is that keywords get reused.
Agreed, this is a plausible suggestion. But rather than just providing an example usage, it would be helpful to see the full syntax of comprehension with the proposed addition. See https://docs.python.org/3/reference/expressions.html#grammar-token-comprehen...
It took me a while to figure this out (battling with the compiler over this), but I settled on the following grammar.
comprehension ::= expression comp_for comp_for ::= "for" target_list "in" or_test [comp_iter] comp_iter ::= comp_for | comp_if | comp_with comp_if ::= "if" expression_nocond [comp_iter] comp_with ::= "with" or_test "as" target
(For comp_with you probably want target_list instead of a single target. However, I felt like double-checking that I got my actual assumptions right by first implementing it (I've got it working now), and seeing as I'm not that experienced with modifying the Python parse/ast/symtable/compile phases, I decided to cop out and take the easy route. It should be fairly trivial to extend it to a target_list instead of a target.)
- note that a comprehension can have arbitrarily many for and if
clauses, interspersed in any order as long as the first one is a "for". I'm guessing you'd add "with <some_sort_of_expression> as <target_list>" as simply a third option. But what would the semantics be? I'm guessing that "with xxx as yyy" translates basically as a statement "yyy = xxx" in the notional expansion described in that section ("considering each of the for or if clauses a block...")
Trying to word it in such a way:
"... considering each of the `for` or `if` clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached. The `with expr as target` should be considered equivalent to `target = expr`.
(And here we already see the downside of this idea). Normally a comprehension of the form
(expr1 for target1 in expr2 for target2 in expr3 if expr4)
boils down to
for target1 in expr2: for target2 in expr3: if expr4: yield expr1
The natural extension would be for
(expr1 for target1 in expr2 with expr3 as target2 if expr4)
to reduce as follows.
for target1 in expr2: with expr3 as target2: if expr4: yield expr1
Instead, it becomes
for target1 in expr2: target2 = expr3: if expr4: yield expr1
But of course we're not going to have context managers in comprehensions, are we? So this inconsistency is somewhat forgiveable.
Assuming that is the proposed definition, a few questions arise:
- Is the behaviour this would assign to "with <foo> as x with <bar>
as x" (i.e., repeated bindings of the same name) what we'd want? Is it likely to cause confusion in practice?
You mean similar to
>>> [x for x in range(5) for x in range(x)] [0, 0, 1, 0, 1, 2, 0, 1, 2, 3]
I've never been confused by that in practice, as most people use good names for stuff. So I would see no ultra-important reason to prevent blocking it for this case and not in others.
- Do we need to restrict the <target_list>? Consider that "with
something as global_var[0]" is allowed by the definition - that would leak values out of the comprehension. This isn't new - the "for" variable works like this already - but is it something that's more likely to be abused than currently? Should we care? (Python doesn't tend to prohibit people from writing bad code).
I'd suggest not to bother with resolving naming conflicts or assignments to things other than names.
[__ for foo.bar in range(10)]
is already valid Python. No need to limit new features in ways which their 'friends' are not.
However, that already has semantics outside a comprehension for context-managers.
This is indeed a concern, albeit not a huge one - the definition has to point out that in the expansion "with" should be read as an assignment (written backwards) not as a with statement. It's not an impossible burden, but it is mildly inconsistent.
This is probably the syntax I prefer of the ones suggested so far. But I still haven't seen any *really* convincing arguments that we need anything new in the first place.
I agree on not having seen a really convincing argument yet. Especially since `for target in [expr]` works as well as `with expr as target`.
The one thing I can think of is common subexpression elimination.
[cheap_process(expensive_thingy(x)) for x in items if expensive_thingy(x) > 0]
or
[cheap_process(y) for x in items with expensive_thingy(x) as y if y > 0]
But in Python3, you could just as well write
[cheap_process(x) for x in map(expensive_thingy, items) if x > 0]
Paul