[Python-Dev] The `for y in [x]` idiom in comprehensions

[Guido van Rossum]

On Thu, Feb 22, 2018 at 11:04 AM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> Yet one discussion about reusing common subexpressions in comprehensions
> took place last week on the Python-ideas maillist (see topic "Temporary
> variables in comprehensions" [1]). The problem is that in comprehension
> like `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is
> evaluated twice. In normal loop you can introduce a temporary variable for
> `f(x)`. The OP wanted to add a special syntax for introducing temporary
> variables in comprehensions. This idea already was discussed multiple times
> in the past.
>
> There are several ways of resolving this problem with existing syntax.
>
> 1. Inner generator expression:
>
>     result = [y + g(y) for y in (f(x) for x in range(10))]
>
> 2. The same, but with extracting the inner generator expression as a
> variable:
>
>     f_samples = (f(x) for x in range(10))
>     result = [y+g(y) for y in f_samples]
>
> 3. Extracting the expression with repeated subexpressions as a function
> with local variables:
>
>     def func(x):
>         y = f(x)
>         return y + g(y)
>     result = [func(x) for x in range(10)]
>
> 4. Implementing the whole comprehension as a generator function:
>
>     def gen():
>         for x in range(10):
>             y = f(x)
>             yield y + g(y)
>     result = list(gen())
>
> 5. Using a normal loop instead of a comprehension:
>
>     result = []
>     for x in range(10):
>         y = f(x)
>         result.append(y + g(y))
>
> And maybe there are other ways.
>
> Stephan Houben proposed an idiom which looks similar to new hypothetic
> syntax:
>
>     result = [y + g(y) for x in range(10) for y in [f(x)]]
>
> `for y in [expr]` in a comprehension means just assigning expr to y. I
> never seen this idiom before, but it can be a good replacement for a
> hypothetic syntax for assignment in comprehensions. It changes the original
> comprehension less than other approaches, just adds yet one element in a
> sequence of for-s and if-s. I think that after using it more widely it will
> become pretty idiomatic.
>
> I have created a patch that optimizes this idiom, making it as fast as a
> normal assignment. [2] Yury suggested to ask Guido on the mailing list if
> he agrees that this language patten is worth optimizing/promoting.
>
> [1] https://mail.python.org/pipermail/python-ideas/2018-February
> /048971.html
> [2] https://bugs.python.org/issue32856
>

I'm not saying anything new here, but since you asked specifically for my
opinion: I don't care for the idiom; it's never occurred to me before, and
it smells of cleverness. If I saw it in a code review I would probably ask
for a regular for-loop to make the code more maintainable.

But if you say it's useful for some class of users and it would be more
useful if it was faster, I'm fine with the optimization. The optimization
is also clever, and here I appreciate cleverness!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/a54748dc/attachment.html>