[Python-ideas] With clauses for generator expressions

[Terry Reedy]

On 11/15/2012 10:25 AM, Andrew Barnert wrote:
> From: Nick Coghlan <ncoghlan at gmail.com>
> Sent: Thu, November 15, 2012 4:39:42 AM
>
>
>> On Thu, Nov 15, 2012 at 9:11 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>
>
>> One, and only one, clause in a comprehension or generator expression is written
>> out of sequence: the innermost clause is lifted out and written first.

This is how list comps were designed and initially defined.

> Given that there are only three clauses,
 > "flatten in order, then move expression to front"

This is the simple and correct rule.

> and "flatten in reverse order, then move if clause to back"

This is more complicated and wrong.

> are identical.

Until one adds more clauses.

> I suppose you're right that, given that the rule for nested
> expressions is to preserve the order of nesting, the first description is more
> natural.
>
> But at any rate, I don't think any such rule is what most Python programmers
> have internalized.

It *is* the rule, and a very simple one. The reference manual gives it, 
though it could perhaps be clearer. The tutorial List Comprehension 
section does give a clear example:
'''
For example, this listcomp combines the elements of two lists if they 
are not equal:

 >>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
and it’s equivalent to:

 >>> combs = []
 >>> for x in [1,2,3]:
...     for y in [3,1,4]:
...         if x != y:
...             combs.append((x, y))
...
 >>> combs
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
Note how the order of the for and if statements is the same in both 
these snippets.
'''

> People obviously know how to nest clauses in general (we
> couldn't speak human languages otherwise), but they do not know how to write, or
> even read, nested comprehensions. What they know is that there are three
> clauses, and they go expression-for-if, period. And those who do learn about
> nesting seem to guess the order wrong at least half the time (hence all the
> StackOverflow posts on "why does [x for x in range(y) for y in range(5)] give me
> a NameError?").

Anyone who read and understood that snippet in the tutorial, which took 
me a minute to find, would not ask such a question. There are people who 
program Python without ever reading the manuals and guess as they go 
and, when they stumble, prefer to post questions on forums and wait for 
a customized answer rather than dig it out themselves.

-- 
Terry Jan Reedy