[Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Mon, Oct 17, 2016 at 12:11:46PM -0400, Random832 wrote:

> Honestly, it goes beyond just being "wrong". The repeated refusal to
> even acknowledge any equivalence between [...x... for x in [a, b, c]]
> and [...a..., ...b..., ...c...] truly makes it difficult for me to
> accept some people's _sincerity_.

While we're talking about people being insincere, how about if you take 
a look at your own comments? This "repeated refusal" that you accuse us 
(opponents of this proposal) of is more of a rhetorical fiction than an 
actual reality. Paul, David and I have all acknowledged the point you 
are trying to make. I won't speak for Paul or David, but speaking for 
myself, it isn't that I don't understand the point you're trying to 
make, but that I do not understand why you think that point is 
meaningful or desirable.

I have acknowledged that starring the expression in a list comprehension 
makes sense if you think of the comprehension as a fully unrolled list 
display:

     [*expr, *expr *expr, *expr, ...]

What I don't believe is:

(1) that the majority of Python programmers (or even a large minority) 
regularly and consistently think of comprehensions as syntactic sugar 
for a completely unrolled list display; rather, I expect that they 
usually think of them as sugar for a for-loop;

(2) that we should encourage people to think of comprehensions as sugar 
for a completely unrolled list display rather than a for-loop;

(3) that we should accept syntax which makes no sense in the context 
of a for-loop-with-append (i.e. the thing which comprehensions are sugar 
for).

But if we *do* accept this syntax, then I believe that we should drop 
the pretense that it is a natural extension of sequence unpacking in the 
context of a for-loop-with-append (i.e. list comprehensions) and accept 
that it will be seen by people as a magical "flatten" operator.

And, in my opinion, rightly so: the semantic distance between *expr in a 
list comp and the level of explanation where it makes sense is so great 
that thinking of it as just special syntax for flattening is the 
simplest way of looking at it.

So, yet again for emphasis: I see what you mean about unrolling the list 
comprehension into a list display. But I believe that's not a helpful 
way to think about list comprehensions. The way we should be thinking 
about them is as for-loops with append, and in *that* context, sequence 
unpacking doesn't make sense.

In a list comprehension, we expect the invariant that the number of 
items produced will equal the number of loops performed. (Less if there 
are any "if" clauses.) There is one virtual append per loop. You cannot 
get the behaviour you want without breaking that invariant: either the 
append has to be replaced by extend, or you have so insert an extra loop 
into your mental picture of comprehensions.

Yet again, for emphasis: I understand that you don't believe that 
invariant is important, or at least you are willing to change it. But 
drop the pretense that this is an obvious extension to the well- 
established behaviour of list comprehensions and sequence unpacking.

If you think you can convince people (particularly Guido) that this 
flattening behaviour is important enough to give up the invariant "one 
append per loop", then by all means try. For all I know, Guido might 
agree with you and love this idea! But while you're accusing us of 
refusing to acknowledge the point you make about unrolling the loop to a 
list display (what I maintain is an unhelpful and non-obvious way of 
thinking about this), you in turn seem to be refusing to acknowledge the 
points we have made. This isn't a small change: it requires not 
insignificant changes to people's understanding of what list 
comprehension syntax means and does.



-- 
Steve