[Python-ideas] Temporary variables in comprehensions

fhsxfhsx fhsxfhsx at 126.com
Sat Feb 17 11:23:28 EST 2018

Thank you Paul, what you said is enlightening and I agree on most part of it.

I'll propose two candidate syntaxs.
1. `with ... as ...`
    This syntax is more paralles as there would be `for` and `with` clause as well as `for` and `with` statement. However, the existing `with` statement is semantically different from this one, although similar.
2. `for ... is ...`
    This syntax is more uniform as the existing `for` clause make an iterator which is a special kind of variable. However, I'm afraid this syntax might be confused with `for ... in ...` as they differ only on one letter.

I like the latter one better. Other proposes are absolutely welcome.

And here is an example which appears quite often in my code where I think a new syntax can help a lot:
Suppose I have an list of goods showing by their ids in database, and I need to transform the ids into json including information from two tables, `Goods` and `GoodsCategory`, where the first table recording `id`, `name` and `category_id` indicating which category the goods belongs to, the second table recording `id`, `name` and `type` to the categories.
With the new syntax, I can write
    'id': goods.id,
    'name': goods.name,
    'category': gc.name,
    'category_type': gc.type,
  for goods_id in goods_id_list
  for goods is Goods.get_by_id(goods_id)
  for gc is GoodsCategory.get_by_id(goods.category_id)

And I cannot think of any good solutions as this one without it.
To generalize this case, for each element of the list, I need two temporary variables (`goods` and `gc` in my case), and each one was used twice.

And reply to the two past discussions you mentioned,
    This mail gave a solution to modify function `f` to keep the result. The weak point is obvious, you must modify the function `f`.
    This mail wrote
>The important thing is that you name the thing you care about before using it.
>I think this is a very natural way of writing: first you give the thing you
>care about a name, then you refer to it by name.
However, that's a problem every comprehension faces, not a problem drawn by the new syntax.

At 2018-02-15 18:08:46, "Paul Moore" <p.f.moore at gmail.com> wrote:
>On 15 February 2018 at 05:56, fhsxfhsx <fhsxfhsx at 126.com> wrote:
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so that
>> we don't need to handle `append` mentally.
>... as long as the code inside the comprehension remains relatively
>simple. It's easy to abuse comprehensions to the point where they are
>less readable than a for loop, but that's true of a lot of things, so
>isn't a specific problem with comprehensions.
>> But when it comes to something like
>> [f(x) + g(f(x)) for x in range(10)]
>> you find you have to sacrifice some readableness if you don't want two f(x)
>> which might slow down your code.
>Agreed. I hit that quite often.
>> Someone may argue that one can write
>> [y + g(y) for y in [f(x) for x in range(10)]]
>> but it's not as clear as to show what `y` is in a subsequent clause, not to
>> say there'll be another temporary list built in the process.
>> We can even replace every comprehension with map and filter, but that would
>> face the same problems.
>That is a workaround (and one I'd not thought of before) but I agree
>it's ugly, and reduces readability. Actually, factoring out the inner
>comprehension like Evpok Padding suggests:
>    f_samples = (f(x) for x in range(10))
>    [y+g(y) for y in f_samples]
>is very readable and effective, IMO, so it's not *that* obvious that
>local names are beneficial.
>> In a word, what I'm arguing is that we need a way to assign temporary
>> variables in a comprehension.
>"We need" is a bit strong here. "It would be useful to have" is
>probably true for some situations.
>> In my opinion, code like
>> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
>> is more natural than any solution we now have.
>> And that's why I pro the new syntax, it's clear, explicit and readable, and
>> is nothing beyond the functionality of the present comprehensions so it's
>> not complicated.
>The problem is that you haven't proposed an actual syntax here, just
>that one should be invented. There have been discussions on this in
>the past (a quick search found
>and https://mail.python.org/pipermail/python-ideas/2012-January/013468.html,
>for example).
>> And I hope the discussion could focus more on whether we should allow
>> assigning temporary variables in comprehensions rather than how to solve the
>> specific example I mentioned above.
>The problem isn't so much "whether we should allow it" as "can we find
>a syntax that is acceptable", and only then "does the new syntax give
>sufficient benefit to be worth adding". New syntax has a pretty high
>cost, and proposals that don't suggest explicit syntax will get stuck
>because you can't judge whether adding the capability is "worth it"
>without being clear on what the cost is - particularly when the
>benefit is relatively small (which this is).
>Agreed that it's important to focus on the general problem, but part
>of the discussion *will* include arguing as to why the existing
>workarounds and alternatives are less acceptable than new syntax. And
>that will need to include discussion of specific cases. Generally, in
>that sort of discussion, artificial examples like "y=f(x)" don't fare
>well because it's too easy to end up just debating subjective views on
>"readability". If you can provide examples from real-world code that
>clearly demonstrate the cost in terms of maintainability of the
>existing workarounds, that will help your argument a lot. Although
>you'll need to be prepared for questions like "would you be willing to
>drop support for versions of Python older than 3.8 in order to get
>this improvement?" - it's surprisingly hard to justify language (as
>opposed to library) changes when you really stop and think about it.
>Which is not to say that it can't be done, just that it's easy to
>underestimate the effort needed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180218/8c781b47/attachment-0001.html>

More information about the Python-ideas mailing list