[Python-ideas] Order of loops in list comprehension

[Rob Cliffe]

On 18/10/2016 07:40, Greg Ewing wrote:
> Random832 wrote:
>> For me, it's the fact that:
>> [[a for a in b] for b in ['uvw', 'xyz']] == [['u', 'v', 'w'], ['x', 'y',
>> 'z']]
>> which makes me want to write:
>> [a for a in b for b in ['uvw', 'xyz']]
>
> You're not alone! Lately I've been becoming convinced that
> this is the way we should have done it right back at the
> beginning.
Me too.  When I first got to grips with the order of loops in a list 
comprehension, I found it counter-intuitive and jarring.  I still have 
to remind myself each time.
I guess different people have different mental models, and may feel 
differently.  The best way I can think of to explain why I feel this way is:

If the syntax were
     [ x for x in alist for alist in list-of-lists ]
there is a smooth (or rather, monotonic) gradation from the smallest 
object (x) to the next biggest object (alist) to the biggest object 
(list-of-lists), which IMHO is easier to follow.  Each object is 
conceptually zero or one steps from its neighbour.
But with the actual syntax
     [ x for alist in list-of-lists for x in alist ]
there is a conceptual hiatus after "x" ("what on earth are alist and 
list-of-lists, and what have they got do to with x?").  This would be 
even more marked with more than 2 loops: we jump from the lowest level 
object to the two highest level objects, and it all seems like a 
disorienting non-sequitur until the very last loop "joins the dots".  
You have to identify the "x" at the beginning with the "x" near (but not 
at!) the end.  Instead of (ideally, if not always in practice) reading 
the expression from left-to-right in one go, your eyes are forced to 
jump around in order for your brain to assimilate it.

A possible alternative syntax might be to follow more closely the 
for-loop syntax, e.g.
     [ for alist in list-of-lists: for x in alist: x ]
Here the "conceptual jump" between each object and the next is either 1 
or 2, which for me makes this a "second best" option.  But at least the 
"conceptual jump" is bounded (regardless of the number of loops), and 
this syntax has the advantage of familiarity.

> But it's far too late to change it now, sadly.
Indeed. :-(  But if I were ruler of the world and could have my own 
wish-list for Python 4, this (as per the first example) would be on it.

Best wishes
Rob Cliffe