[Python-ideas] Saving state in list/generator comprehension
acarter at cs.hmc.edu
Mon Jun 11 00:42:53 CEST 2012
Forgive me for any problems in this e-mail as I'm new to this mailing list.
I thought it might be nice to be able to somehow save a state in
a side effect of this (although not the intended goal) is it would make
reduce feasible in a clean manner as the final result would just be the
One mechanism I can think of is to overload the with/as keyword for use
inside of list/generator comprehensions, and using the previous result as
I believe the change to the grammar in python3k would be
comp_iter : comp_for | comp_if | comp_with
comp_with: 'with' testlist 'as' testlist
So something in the form of
[expr for i in iterable with initializer as accumulator]
would resolve to something like
result = 
accumulator = initializer
for i in iterable:
accumulator = expr
For instance reduce could be defined as (assuming all 3 arguments are
reduce = lambda function, iterable, initializer : ([initializer] +
[function(accumulator, i) for i in iterable with initializer as
Breaking this down, the "with initializer as accumulator" statement means
that when the list comprehension begins accumulator=initializer,
then after each iteration, accumulator = function(accumulator, i), so with
the function f, list [i1,i2,i3,...], and initial value i0, the resulting
"[function(accumulator, i) for i in iterable with initializer as
accumulator]" would be [f(i0,i1), f(f(i0,i1),i2),
f(f(f(i0,i1),i2),i3),...], or in left associative infix form with
the f = "+" operator, [i0+i1,i0+i1+i2,i0+i1+i2+i3,...].
Consing (effectively) initializer to the beginning of the list ensures
clean behavior for empty lists, and indexing [-1] gets the last element
which is really the only
element that matters.
Consider a slightly more complex example of a Fibonacci generator, one
might define it as follows,
a, b = 1, 0
a, b = b, a + b
Using the with statement, it would require two generator comprehensions
fibs = lambda : (b for a,b in (b, a+b for i in itertools.cycle((None,))
with a,b = 0,1))
The inner generator comprehension
(b, a+b for i in itertools.repeat(None) with a,b = 0,1)
creates an infinite generator of tuples which are
consecutive Fibonacci numbers, the outer list comprehension strips off the
Some of the pros of doing it this way is that because with/as are already
keywords in python backwards compatibility shouldn't be an issue,
but if one is just mapping with state then an extra list/generator
comprehension block is needed to strip the state from the intermediate list.
I apologize if similar ideas have already been discussed.
p.s. Is there a built-in way to get the last element from a generator
(perhaps even with a default) a quick google search did not reveal one?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas