# [Python-ideas] With clauses for generator expressions

Terry Reedy tjreedy at udel.edu
Thu Nov 15 19:36:45 CET 2012

```On 11/15/2012 10:25 AM, Andrew Barnert wrote:
> From: Nick Coghlan <ncoghlan at gmail.com>
> Sent: Thu, November 15, 2012 4:39:42 AM
>
>
>> On Thu, Nov 15, 2012 at 9:11 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>
>
>> One, and only one, clause in a comprehension or generator expression is written
>> out of sequence: the innermost clause is lifted out and written first.

This is how list comps were designed and initially defined.

> Given that there are only three clauses,
> "flatten in order, then move expression to front"

This is the simple and correct rule.

> and "flatten in reverse order, then move if clause to back"

This is more complicated and wrong.

> are identical.

> I suppose you're right that, given that the rule for nested
> expressions is to preserve the order of nesting, the first description is more
> natural.
>
> But at any rate, I don't think any such rule is what most Python programmers
> have internalized.

It *is* the rule, and a very simple one. The reference manual gives it,
though it could perhaps be clearer. The tutorial List Comprehension
section does give a clear example:
'''
For example, this listcomp combines the elements of two lists if they
are not equal:

>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
and it’s equivalent to:

>>> combs = []
>>> for x in [1,2,3]:
...     for y in [3,1,4]:
...         if x != y:
...             combs.append((x, y))
...
>>> combs
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
Note how the order of the for and if statements is the same in both
these snippets.
'''

> People obviously know how to nest clauses in general (we
> couldn't speak human languages otherwise), but they do not know how to write, or
> even read, nested comprehensions. What they know is that there are three
> clauses, and they go expression-for-if, period. And those who do learn about
> nesting seem to guess the order wrong at least half the time (hence all the
> StackOverflow posts on "why does [x for x in range(y) for y in range(5)] give me
> a NameError?").

Anyone who read and understood that snippet in the tutorial, which took
me a minute to find, would not ask such a question. There are people who
program Python without ever reading the manuals and guess as they go
and, when they stumble, prefer to post questions on forums and wait for
a customized answer rather than dig it out themselves.

--
Terry Jan Reedy

```