[Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Chris Angelico]

On Wed, Apr 11, 2018 at 1:41 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Personally, I still think the best approach here is a combination of
> itertools.accumulate, and the proposed name-binding as an expression
> feature:
>
>     total = 0
>     running_totals = [(total := total + x) for x in values]
>     # alternative syntax
>     running_totals = [(total + x as total) for x in values]
>
> If you don't like the dependency on an external variable (or if that
> turns out not to be practical) then we could have:
>
>     running_totals = [(total := total + x) for total in [0] for x in values]

That last one works, but it's not exactly pretty. Using an additional
'for' loop to initialize variables feels like a gross hack.
Unfortunately, the first one is equivalent to this (in a PEP 572
world):

total = 0
def <listcomp>():
    result = []
    for x in values:
        result.push(total := total + x)
    return result
running_totals = <listcomp>()

Problem: it's still happening in a function, which means this bombs
with UnboundLocalError.

Solution 1: Use the extra loop to initialize 'total' inside the
comprehension. Ugly.

Solution 2: Completely redefine comprehensions to use subscopes
instead of a nested function. I used to think this was a good thing,
but after the discussions here, I've found that this creates as many
problems as it solves.

Solution 3: Have some way for a comprehension to request that a name
be imported from the surrounding context. Effectively this:

total = 0
def <listcomp>(total=total):
    result = []
    for x in values:
        result.push(total := total + x)
    return result
running_totals = <listcomp>()

This is how, in a PEP 572 world, the oddities of class scope are
resolved. (I'll be posting a new PEP as soon as I fix up three failing
CPython tests.) It does have its own problems, though. How do you know
which names to import like that? What if 'total' wasn't assigned to
right there, but instead was being lifted from a scope further out?

Solution 4: Have *all* local variables in a comprehension get
initialized to None.

def <listcomp>():
    result = []
    total = x = None
    for x in values:
        result.push(total := (total or 0) + x)
    return result
running_totals = <listcomp>()

running_totals = [(total := (total or 0) + x) for total in [0] for x in values]

That'd add to the run-time cost of every list comp, but probably not
measurably. (Did you know, for instance, that "except Exception as e:"
will set e to None before unbinding it?) It's still not exactly
pretty, though, and having to explain why you have "or 0" in a purely
arithmetic operation may not quite work.

Solution 5: Allow an explicit initializer syntax. Could work, but
you'd have to come up with one that people are happy with.

None is truly ideal IMO.

ChrisA