Proposal: Use "for x = value" to assign in scope
Alright hear me out here: I've often found that it would be useful for the following type of expression to be condensed to a one-liner: def running_average(x_seq): averages = [] avg = 0 for t, x in enumerate(x_seq): avg = avg*t/(t+1) + x/(t+1) averages.append(avg) return averages Because really, there's only one line doing the heavy lifting here, the rest is kind of boilerplate. Then I learned about the beautiful and terrible "for x in [value]": def running_average(x_seq): return [avg for avg in [0] for t, x in enumerate(x_seq) for avg in [avg*t/(t+1) + x/(t+1)]] Many people find this objectionable because it looks like there are 3 for loops, but really there's only one: loops 0 and 2 are actually assignments. **My Proposal** What if we just officially bless this "using for as a temporary assignment" arrangement, and allow "for x=value" to mean "assign within the scope of this for". It would be identical to "for x in [value]", just more readable. The running average function would then be: def running_average(x_seq): return [avg for avg=0 for t, x in enumerate(x_seq) for avg = avg * t/(t+1) + x / (t+1)] ------ P.S. 1 I am aware of Python 3.8's new "walrus" operator, which would make it: def running_average(x_seq): avg = 0 return [avg := avg*t/(t+1) + x / (t+1) for t, x in enumerate(x_seq)] But it seems ugly and bug-prone to be initializing a in-comprehension variable OUTSIDE the comprehension. ------ P.S. 2 The "for x = value" syntax can achieve things that are not nicely achievable using the := walrus. Consider the following example (wherein we carry forward a "hidden" variable h but do not return it): y_seq = [y for h=0 for x in x_seq for y, h = update(x, h)] There's not really a nice way to do this with the walrus because you can't (as far as I understand) combine it with tuple-unpacking. You'd have to do something awkward like: yh = None, 0 y_seq, _ = zip(*(yh := update(x, yh[1]) for x in x_seq)) ------
On Aug 9, 2019, at 08:47, Peter O'Connor
I've often found that it would be useful for the following type of expression to be condensed to a one-liner:
def running_average(x_seq): averages = [] avg = 0 for t, x in enumerate(x_seq): avg = avg*t/(t+1) + x/(t+1) averages.append(avg) return averages
Because really, there's only one line doing the heavy lifting here, the rest is kind of boilerplate.
It seems like the only reason you can’t write this with accumulate is that accumulate doesn’t take a start value like reduce does? And I think this would be a lot clearer and more readable, especially if you’re doing this kind of thing more than once: def running(xs, func, start): yield from accumulate(enumerate(xs), lambda avg, told: func(avg, *tx), start) def running_average(xs, func): yield from running(xs, lambda avg, t, x: avg*t/(t+1) + x/(t+1), 0.0) Now the part that does the heavy lifting is all in one place and just does what it’s says, without being confusingly interleaved with the boilerplate. Plus, the parts of the boilerplate that are reusable are abstracted into functions (accumulate and running) that can be reused, while the rest of it has vanished. (This is one of those rare cases where 2.x-style decomposing def/lambda was actually useful, but if that extra lambda in running really bothers you, that’s another one-liner HOF you can abstract out trivially and reuse.) More generally, it’s a lot easier to use comprehensions and higher order functions if your algorithm can be written in terms of “generate the next immutable value” instead of “update the mutable variable”, and I don’t think that’s a limitation of the language. Comprehensions are much more readable when they’re declarative than when they’re for statements in disguise. Also, I don’t think the reason people were objecting to your four-clause comprehension was that it wasn’t easy enough to tell that the innermost clause only “loops” exactly one time, but that it’s a comprehension with four clauses in the first place. Changing the spelling of that clause to make the no-actual-looping doesn’t solve that. Finally, you can already play tricks with the walrus operator to avoid moving things like initialization outside the comprehension, just as you could with your proposed syntax. For example, “for t, x in (avg:=0) or enumerate(xs)” is a perfectly valid clause that assigns 0 to avg and then loops over the enumerate, and it doesn’t require you to turn one for clause into two. But it still adds just as much complexity for the reader to deal with, so I think you’re still better off not doing it. (As a side note, you probably want a numerical stable average like the ones in statistics or numpy, rather than one that accumulates float rounding errors indiscriminately, but that’s another issue.)
On Fri, Aug 9, 2019 at 9:03 AM Peter O'Connor
Alright hear me out here:
I've often found that it would be useful for the following type of expression to be condensed to a one-liner:
def running_average(x_seq): averages = [] avg = 0 for t, x in enumerate(x_seq): avg = avg*t/(t+1) + x/(t+1) averages.append(avg) return averages
Because really, there's only one line doing the heavy lifting here, the rest is kind of boilerplate.
But it's boilerplate that communicates the starting state of your loop which is useful to know and to have be very clearly communicated.
Then I learned about the beautiful and terrible "for x in [value]":
def running_average(x_seq): return [avg for avg in [0] for t, x in enumerate(x_seq) for avg in [avg*t/(t+1) + x/(t+1)]]
Many people find this objectionable because it looks like there are 3 for loops, but really there's only one: loops 0 and 2 are actually assignments.
**My Proposal**
What if we just officially bless this "using for as a temporary assignment" arrangement, and allow "for x=value" to mean "assign within the scope of this for". It would be identical to "for x in [value]", just more readable. The running average function would then be:
def running_average(x_seq): return [avg for avg=0 for t, x in enumerate(x_seq) for avg = avg * t/(t+1) + x / (t+1)]
I personally don't find that more readable then the unrolled version you're trying to avoid. And based on the amount of grief we got for the walrus operator I wouldn't expect much uptake on this as being considered more readable by others either. (And remember that "Readability counts"). -Brett
------ P.S. 1 I am aware of Python 3.8's new "walrus" operator, which would make it:
def running_average(x_seq): avg = 0 return [avg := avg*t/(t+1) + x / (t+1) for t, x in enumerate(x_seq)]
But it seems ugly and bug-prone to be initializing a in-comprehension variable OUTSIDE the comprehension.
------ P.S. 2 The "for x = value" syntax can achieve things that are not nicely achievable using the := walrus. Consider the following example (wherein we carry forward a "hidden" variable h but do not return it):
y_seq = [y for h=0 for x in x_seq for y, h = update(x, h)]
There's not really a nice way to do this with the walrus because you can't (as far as I understand) combine it with tuple-unpacking. You'd have to do something awkward like:
yh = None, 0 y_seq, _ = zip(*(yh := update(x, yh[1]) for x in x_seq)) ------
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RHW5AU... Code of Conduct: http://python.org/psf/codeofconduct/
On 09/08/2019 18:10, Brett Cannon wrote:
On Fri, Aug 9, 2019 at 9:03 AM Peter O'Connor
wrote: Alright hear me out here:
I've often found that it would be useful for the following type of expression to be condensed to a one-liner:
def running_average(x_seq): averages = [] avg = 0 for t, x in enumerate(x_seq): avg = avg*t/(t+1) + x/(t+1) averages.append(avg) return averages
Because really, there's only one line doing the heavy lifting here, the rest is kind of boilerplate.
But it's boilerplate that communicates the starting state of your loop which is useful to know and to have be very clearly communicated.
+1. The intent and operation of your code is clear. Win.
Then I learned about the beautiful and terrible "for x in [value]":
def running_average(x_seq): return [avg for avg in [0] for t, x in enumerate(x_seq) for avg in [avg*t/(t+1) + x/(t+1)]]
Many people find this objectionable because it looks like there are 3 for loops, but really there's only one: loops 0 and 2 are actually assignments.
I find it objectionable because it's unreadable. I would reject this in a code review as "too clever for its own good," therefore unnecessarily hard to maintain.
**My Proposal**
What if we just officially bless this "using for as a temporary assignment" arrangement, and allow "for x=value" to mean "assign within the scope of this for". It would be identical to "for x in [value]", just more readable. The running average function would then be:
def running_average(x_seq): return [avg for avg=0 for t, x in enumerate(x_seq) for avg = avg * t/(t+1) + x / (t+1)]
I personally don't find that more readable then the unrolled version you're trying to avoid. And based on the amount of grief we got for the walrus operator I wouldn't expect much uptake on this as being considered more readable by others either. (And remember that "Readability counts").
Agreed. -- Rhodri James *-* Kynesim Ltd
Peter O'Connor wrote:
Alright hear me out here: I've often found that it would be useful for the following type of expression to be condensed to a one-liner: def running_average(x_seq): averages = [] avg = 0 for t, x in enumerate(x_seq): avg = avg*t/(t+1) + x/(t+1) averages.append(avg) return averages Because really, there's only one line doing the heavy lifting here, the rest is kind of boilerplate. Then I learned about the beautiful and terrible "for x in [value]": def running_average(x_seq): return [avg for avg in [0] for t, x in enumerate(x_seq) for avg in [avg*t/(t+1) + x/(t+1)]] Many people find this objectionable because it looks like there are 3 for loops, but really there's only one: loops 0 and 2 are actually assignments.
You can solve this via `itertools.accumulate` in a concise and clear way: [x/n for n, x in enumerate(it.accumulate(x_seq), 1)]
My Proposal What if we just officially bless this "using for as a temporary assignment" arrangement, and allow "for x=value" to mean "assign within the scope of this for". It would be identical to "for x in [value]", just more readable. The running average function would then be: def running_average(x_seq): return [avg for avg=0 for t, x in enumerate(x_seq) for avg = avg * t/(t+1) + x / (t+1)] ------ P.S. 1 I am aware of Python 3.8's new "walrus" operator, which would make it: def running_average(x_seq): avg = 0 return [avg := avg*t/(t+1) + x / (t+1) for t, x in enumerate(x_seq)] But it seems ugly and bug-prone to be initializing a in-comprehension variable OUTSIDE the comprehension. ------ P.S. 2 The "for x = value" syntax can achieve things that are not nicely achievable using the := walrus. Consider the following example (wherein we carry forward a "hidden" variable h but do not return it): y_seq = [y for h=0 for x in x_seq for y, h = update(x, h)] There's not really a nice way to do this with the walrus because you can't (as far as I understand) combine it with tuple-unpacking. You'd have to do something awkward like: yh = None, 0 y_seq, _ = zip(*(yh := update(x, yh[1]) for x in x_seq))
You can't use `:=` with tuple unpacking but you can use it with tuples directly; this requires a definition of the initial tuple (preferably outside the loop) but this is i.m.o. a plus since it clearly marks the initial conditions for your algorithm: yh = (None, 2) # initial values [(yh := update(x, yh[1]))[0] for x in x_seq] If you really ever have the need to carry a variable over into a comprehension (which might be a valid thing, for example in a class body) then you can still resort to an additional `for` loop (as you've already indicated); after all it's not too bad and you can even put the different loops on different lines + add a comment if necessary: class Foo: a = 1 # this won't work: b = [x*a for x in range(5)] # we can carry `a` over to the comprehension as follows: b = [x*a for a in [a] for x in range(5)] Using just `for = ...` is not much of a difference, especially since most people will see the `for` and immediately assume it's a loop (so in that sense it's even more confusing).
participants (5)
-
Andrew Barnert
-
Brett Cannon
-
Dominik Vilsmeier
-
Peter O'Connor
-
Rhodri James