[Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Sun, Oct 16, 2016 at 12:48:36PM +1300, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >Are you now supporting my argument that starring the list comprehension 
> >expression isn't meaningful?
> 
> The context it's in (a form of list display) has a clear
> meaning for a comma-separated list of values, so there
> is a reasonable interpretation that it *could* be given.

This thread is a huge, multi-day proof that people do not agree that 
this is a "reasonable" interpretation.


> >py> iterable = [(1, 'a'), (2, 'b')]
> >py> [(100, *t) for t in iterable]
> >[(100, 1, 'a'), (100, 2, 'b')]
> 
> The * there is in the context of constructing a tuple,
> not the list into which the tuple is placed.

Right: the context of the star is meaningful. We all agree that *t in a 
list display [a, b, c, ...] is meaningful; same for tuples; same for 
function calls; same for sequence unpacking for assignment.

What is not meaningful (except as a Perlish line-noise special case to 
be memorised) is *t as the list comprehension expression.

I've never disputed that we could *assert* that *t in a list comp means 
"flatten". We could assert that it means anything we like. But it 
doesn't follow from the usual meaning of sequence unpacking anywhere 
else -- that's why it is currently a SyntaxError, and that's why people 
reacted with surprise at the OP who assumed that *t would magically 
flatten his iterable. Why would you assume that? It makes no sense to me 
-- that's not how sequence unpacking works in any other context, it 
isn't how list comprehensions work.

Right from the beginning I called this "wishful thinking", and *nothing* 
since then has changed my mind. This proposal only makes even a little 
bit of sense if you imagine list comprehensions

    [*t for a in it1 for b in it2 for c in it3 ... for t in itN]

completely unrolled into a list display:

    [*t, *t, *t, *t, ... ]

but who does that? Why would you reason about your list comps like that? 
If you think about list comps as we're expected to think of them -- as 
list builders equivalent to a for-loop -- the use of *t there is 
invalid. Hence it is a SyntaxError.

You want a second way to flatten your iterables? A cryptic, mysterious, 
Perlish line-noise way? Okay, fine, but don't pretend it is sequence 
unpacking -- in the context of a list comprehension, sequence unpacking 
doesn't make sense, it is invalid. Call it something else: the new 
"flatten" operator:

    [^t for t in iterable]

for example, which magically adds an second invisible for-loop to your 
list comps:

    # expands to
    for t in iterable:
        for x in t:
            result.append(x)

Because as your own email inadvertently reinforces, if sequence 
unpacking made sense in the context of a list comprehension, it would 
already be allowed rather than a SyntaxError: it is intentionally 
prohibited because it doesn't make sense in the context of list comps.


-- 
Steve