Return expressions
This idea occurred to me in context of PEP 479 and the "or stop()" hack for generator expressions. I'm not a big enough fan of the idea to pursue it myself, but if anyone is bothered by the prospect of PEP 479 taking the "or stop()" technique away, this may be worth pursuing further. As a reminder of how the hack works, the following generator:
def takewhile(iterable, pred): ... for x in iter(iterable): ... if not pred(x): ... return ... yield x ... list(takewhile(range(10), (lambda x: x < 5))) [0, 1, 2, 3, 4]
Can currently be converted to a generator expression with the aid of a helper function:
def stop(): ... raise StopIteration ... list(x for x in range(10) if x < 5 or stop()) [0, 1, 2, 3, 4]
Under PEP 479, that will raise RuntimeError instead. If return was an expression rather than a statement (ala the yield statement -> expression conversion in PEP 342), then the following would be functionally equivalent to the current "or stop()" trick:
list(x for x in range(10) if x < 5 or return) [0, 1, 2, 3, 4]
This is emphatically *not* executable pseudocode - it only makes sense in terms of the translation to the corresponding full generator function definition. However, the translation remains exact in terms of the normal semantics of the "return" keyword, unlike various other proposals to use the "while" keyword to provide comparable functionality. For comprehensions, the meaning of a bare return would likely need to be tweaked slightly to be the same as reaching the end of the frame: returning the object being constructed by the comprehension, rather than None.
[x for x in range(10) if x < 5 or return] [0, 1, 2, 3, 4] {x for x in range(10) if x < 5 or return} {0, 1, 2, 3, 4} {x:x for x in range(10) if x < 5 or return} {0:0, 1:1, 2:2, 3:3, 4:4}
Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2014-11-20 10:01, Nick Coghlan wrote:
This idea occurred to me in context of PEP 479 and the "or stop()" hack for generator expressions. I'm not a big enough fan of the idea to pursue it myself, but if anyone is bothered by the prospect of PEP 479 taking the "or stop()" technique away, this may be worth pursuing further.
As a reminder of how the hack works, the following generator:
def takewhile(iterable, pred): ... for x in iter(iterable): ... if not pred(x): ... return ... yield x ... list(takewhile(range(10), (lambda x: x < 5))) [0, 1, 2, 3, 4]
Can currently be converted to a generator expression with the aid of a helper function:
def stop(): ... raise StopIteration ... list(x for x in range(10) if x < 5 or stop()) [0, 1, 2, 3, 4]
Under PEP 479, that will raise RuntimeError instead. If return was an expression rather than a statement (ala the yield statement -> expression conversion in PEP 342), then the following would be functionally equivalent to the current "or stop()" trick:
list(x for x in range(10) if x < 5 or return) [0, 1, 2, 3, 4]
This is emphatically *not* executable pseudocode - it only makes sense in terms of the translation to the corresponding full generator function definition. However, the translation remains exact in terms of the normal semantics of the "return" keyword, unlike various other proposals to use the "while" keyword to provide comparable functionality.
For comprehensions, the meaning of a bare return would likely need to be tweaked slightly to be the same as reaching the end of the frame: returning the object being constructed by the comprehension, rather than None.
[x for x in range(10) if x < 5 or return] [0, 1, 2, 3, 4] {x for x in range(10) if x < 5 or return} {0, 1, 2, 3, 4} {x:x for x in range(10) if x < 5 or return} {0:0, 1:1, 2:2, 3:3, 4:4}
I'd prefer 'while' instead:
[x for x in range(10) while x < 5] [0, 1, 2, 3, 4] {x for x in range(10) while x < 5} {0, 1, 2, 3, 4} {x:x for x in range(10) while x < 5} {0:0, 1:1, 2:2, 3:3, 4:4}
On 20 November 2014 22:54, MRAB <python@mrabarnett.plus.com> wrote:
I'd prefer 'while' instead:
[x for x in range(10) while x < 5] [0, 1, 2, 3, 4] {x for x in range(10) while x < 5} {0, 1, 2, 3, 4} {x:x for x in range(10) while x < 5} {0:0, 1:1, 2:2, 3:3, 4:4}
That's been suggested many times, and always fails on the grounds of being completely incompatible with the expansions of comprehensions and generator expressions as nested compound statements. The return expression idea is one that reflects the actual deep structure of the nested scopes that underlie the syntactic sugar, so it's compatible with the expansion. This idea was mainly aimed at folks that might be inclined to worry about the possible loss of the "or stop()" generator expression trick in PEP 479, though. If that trick wasn't currently possible, and wasn't being proposed for removal, I never would have posted this idea. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Nov 20, 2014 at 2:36 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 20 November 2014 22:54, MRAB <python@mrabarnett.plus.com> wrote:
I'd prefer 'while' instead:
[x for x in range(10) while x < 5] [0, 1, 2, 3, 4] {x for x in range(10) while x < 5} {0, 1, 2, 3, 4} {x:x for x in range(10) while x < 5} {0:0, 1:1, 2:2, 3:3, 4:4}
That's been suggested many times, and always fails on the grounds of being completely incompatible with the expansions of comprehensions and generator expressions as nested compound statements.
The return expression idea is one that reflects the actual deep structure of the nested scopes that underlie the syntactic sugar, so it's compatible with the expansion.
This idea was mainly aimed at folks that might be inclined to worry about the possible loss of the "or stop()" generator expression trick in PEP 479, though. If that trick wasn't currently possible, and wasn't being proposed for removal, I never would have posted this idea.
The "while" expresses intent, "return" caters to low-level mechanics. The "nested compound statements" explanation is already not that good: why does the value come first, not last? For readability*. The difference between function/nested compound statements syntax and comprehension syntax is already so big, and the low-level "while x:"→"if not x: break" translation is so easy, that between the two the readability of "while" should win. Outside of comprehensions, an expression that never has a value seems quite backwards (and dangerous, in this case). * (or you might say, for similarity to math notation)
On Thu, Nov 20, 2014 at 2:36 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 20 November 2014 22:54, MRAB <python@mrabarnett.plus.com> wrote:
I'd prefer 'while' instead:
[x for x in range(10) while x < 5] [0, 1, 2, 3, 4] {x for x in range(10) while x < 5} {0, 1, 2, 3, 4} {x:x for x in range(10) while x < 5} {0:0, 1:1, 2:2, 3:3, 4:4}
That's been suggested many times, and always fails on the grounds of being completely incompatible with the expansions of comprehensions and generator expressions as nested compound statements.
The return expression idea is one that reflects the actual deep structure of the nested scopes that underlie the syntactic sugar, so it's compatible with the expansion.
This idea was mainly aimed at folks that might be inclined to worry about the possible loss of the "or stop()" generator expression trick in PEP 479, though. If that trick wasn't currently possible, and wasn't being proposed for removal, I never would have posted this idea.
The "while" expresses intent, "return" caters to low-level mechanics. The "nested compound statements" explanation is already not that good: why does the value come first, not last? For readability*. The difference between function/nested compound statements syntax and comprehension syntax is already so big, and the low-level "while x:"→"if not x: break" translation is so easy, that between the two the readability of "while" should win. I can see the problem with 'while': if there are multiple 'for' parts,
On 2014-11-20 14:43, Petr Viktorin wrote: they are equivalent to nested 'for' loops, so you might assume that a 'while' part would also be equivalent to a nested 'while' loop, whereas it would, in fact, be controlling the preceding 'for' part. In other words, it would be: for x in range(10) while x < 5: .... but could be seen as: for x in range(10): while x < 5: ....
Outside of comprehensions, an expression that never has a value seems quite backwards (and dangerous, in this case).
* (or you might say, for similarity to math notation)
On 21 November 2014 01:49, MRAB <python@mrabarnett.plus.com> wrote:
On 2014-11-20 14:43, Petr Viktorin wrote:
The "while" expresses intent, "return" caters to low-level mechanics. The "nested compound statements" explanation is already not that good: why does the value come first, not last? For readability*. The difference between function/nested compound statements syntax and comprehension syntax is already so big, and the low-level "while x:"→"if not x: break" translation is so easy, that between the two the readability of "while" should win.
I can see the problem with 'while': if there are multiple 'for' parts, they are equivalent to nested 'for' loops, so you might assume that a 'while' part would also be equivalent to a nested 'while' loop, whereas it would, in fact, be controlling the preceding 'for' part.
In other words, it would be:
for x in range(10) while x < 5: ....
but could be seen as:
for x in range(10): while x < 5:
PEP 3142 was the last attempt at asking this question - Guido isn't keen on the idea, so he rejected it the last time we did a pass through the PEPs that weren't being actively worked on. So while the odds still aren't good, a proposal that kept the statement and expression forms consistent might still have some chance, by proposing that the statement example above: for x in range(10) while x < 5: .... Be equivalent to: for x in range(10): if not x < 5: break .... That is, there'd only be one loop, rather than two. At that point, a "while" clause in a comprehension or generator expression would be part of that enhanced syntax, rather than a standalone while loop. That does also suggest another possible alternative to the "or stop()" trick - allow *break* as an expression. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Nov 21, 2014 at 3:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
That does also suggest another possible alternative to the "or stop()" trick - allow *break* as an expression.
Or just as a keyword component of a comprehension, in the same way 'for' is. Since 'if' already has meaning, [x for x in range(10) if x
= 5 break] would be confusing, so probably it'd have to be the slightly-awkward-to-read [x for x in range(10) break x >= 5], adding a termination condition to the preceding loop.
ChrisA
On Thu, Nov 20, 2014 at 4:36 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Nov 21, 2014 at 3:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
That does also suggest another possible alternative to the "or stop()" trick - allow *break* as an expression.
Or just as a keyword component of a comprehension, in the same way 'for' is. Since 'if' already has meaning, [x for x in range(10) if x
= 5 break] would be confusing, so probably it'd have to be the slightly-awkward-to-read [x for x in range(10) break x >= 5], adding a termination condition to the preceding loop.
'break' is certainly better than 'return' -- currently 'break' means "look for the nearest enclosing loop", not "look for the enclosing function", so it'd preserve that at least. [x for x in range(10) if x >= 5 else break]? with 'else break' handled in the grammar as a non-compositional phrase like 'is not'? Not sure how this interacts with multiple mixed for and if clauses -- I've never wrapped my head around those semantics. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
This is fairly similar to a recent thread suggesting an "or raise" construct ("x = f() or raise ValueError" as a shortcut for "x = f(); if not x: raise ValueError"). I suggested that "raise" be made a value-less expression, so that "x = f() or raise ..." can be intepreted normally. As a side effect, this would make lambdas slightly more powerful too. Having "return" and "break" as value-less expressions would be nice too. The interaction with nested loops in generators would be the natural one, without any generator-specific syntax: ((x, y) if stay_in_inner(x, y) else break for x in X for y in Y) is the same as for x in X: for y in Y: if stay_in_inner(x, y): yield x, y else: break or, with break-expression everywhere, as a separate generator: for x in X: for y in Y: (yield x, y) if stay_in_inner(x, y) else break whereas ((x, y) if stay_in_gen(x, y) else return for x in X for y in Y) is the same as for x in X: for y in Y: if stay_in_gen(x, y): yield x, y else: return or for x in X: for y in Y: (yield x, y) if stay_in_gen(x, y) else return i.e. the intepretation of a genexp is still to take the nested "for"s and "if"s after the leading expression, add suitable colons, newlines and indentation, and insert the leading expression at the end. Antony 2014-11-20 10:02 GMT-08:00 Nathaniel Smith <njs@pobox.com>:
On Thu, Nov 20, 2014 at 4:36 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Nov 21, 2014 at 3:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
That does also suggest another possible alternative to the "or stop()" trick - allow *break* as an expression.
Or just as a keyword component of a comprehension, in the same way 'for' is. Since 'if' already has meaning, [x for x in range(10) if x
= 5 break] would be confusing, so probably it'd have to be the slightly-awkward-to-read [x for x in range(10) break x >= 5], adding a termination condition to the preceding loop.
'break' is certainly better than 'return' -- currently 'break' means "look for the nearest enclosing loop", not "look for the enclosing function", so it'd preserve that at least.
[x for x in range(10) if x >= 5 else break]?
with 'else break' handled in the grammar as a non-compositional phrase like 'is not'? Not sure how this interacts with multiple mixed for and if clauses -- I've never wrapped my head around those semantics.
-n
-- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Sent from a random iPhone On Nov 20, 2014, at 8:21, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 21 November 2014 01:49, MRAB <python@mrabarnett.plus.com> wrote:
On 2014-11-20 14:43, Petr Viktorin wrote:
The "while" expresses intent, "return" caters to low-level mechanics. The "nested compound statements" explanation is already not that good: why does the value come first, not last? For readability*. The difference between function/nested compound statements syntax and comprehension syntax is already so big, and the low-level "while x:"→"if not x: break" translation is so easy, that between the two the readability of "while" should win.
I can see the problem with 'while': if there are multiple 'for' parts, they are equivalent to nested 'for' loops, so you might assume that a 'while' part would also be equivalent to a nested 'while' loop, whereas it would, in fact, be controlling the preceding 'for' part.
In other words, it would be:
for x in range(10) while x < 5: ....
but could be seen as:
for x in range(10): while x < 5:
PEP 3142 was the last attempt at asking this question - Guido isn't keen on the idea, so he rejected it the last time we did a pass through the PEPs that weren't being actively worked on.
The fact few times this has came up on -ideas or -dev, people suggested a variety of alternative syntaxes for this idea, just as is happening this time, and so far, they've always been subsets of the same ideas. There are syntaxes that map nicely to the nested statement translation but don't read well, syntaxes that read well but make no sense as nested statements, syntaxes that sort of meet both those criteria but are so verbose nobody would use them, syntaxes that only make sense if everyone knows that comprehensions (including genexprs) are run inside their own hidden functions, syntaxes that only make sense if you pretend they aren't, syntaxes that require a change to normal for statements that wouldn't ever realistically be used there, and syntaxes that change the basic principle behind how comprehensions work. (And maybe my suggestion to make or stop() work in listcomps the same way fits in here, even though the goal was to unify the semantics of all genexprs and other comprehensions, rather than to provide this functionality.) So far, Guido hasn't liked any of them, and none of them have gotten the kind of widespread buy-in from the community that seem likely to convince him to take another look. So it seems like a waste of time to rehash them every year or so. Maybe what we need to do is update PEP 3142, gathering all of the alternative ideas, and expanding on the rationale, so Guido can reject that, and next time this comes up, everyone will have something to read before having the same arguments about the same proposals?
On 11/21/2014 08:22 AM, Andrew Barnert wrote:
Maybe what we need to do is update PEP 3142, gathering all of the alternative ideas, and expanding on the rationale, so Guido can reject that, and next time this comes up, everyone will have something to read before having the same arguments about the same proposals?
Are we accepting nominations? I have the perfect fellow in mind. :) (And in case Chris decides to nominate me in retaliation, I decline. ;) -- ~Ethan~
On Sat, Nov 22, 2014 at 3:36 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 11/21/2014 08:22 AM, Andrew Barnert wrote:
Maybe what we need to do is update PEP 3142, gathering all of the alternative ideas, and expanding on the rationale, so Guido can reject that, and next time this comes up, everyone will have something to read before having the same arguments about the same proposals?
Are we accepting nominations? I have the perfect fellow in mind. :)
(And in case Chris decides to nominate me in retaliation, I decline. ;)
Heh! Great, I'm going to get a name for writing the PEPs that exist solely to be rejected... I was hoping PEP 479 would be accepted easily, given who was advocating it, but now it's starting to look like I doomed the proposal :) ChrisA
I would like to believe that "break"-as-a-expression solves most of these issues. It is reasonable, though, to drop the "return"-as-an-expression part of the proposal, because you need to know that the comprehension is run in a separate function to understand it (it's a bit ironic that I am dropping the original proposal to defend my own, now...). Consider ((x, y) for x in l1 if f1(x) or break for y in l2 if f2(y) or break) This maps directly to for x in l1: if f1(x) or break: for y in l2: if f2(y) or break: yield x, y which is literally a copy-paste of the second part followed by yielding the first part. I think this reads reasonably well but this is obviously a subjective issue. Antony 2014-11-21 8:49 GMT-08:00 Chris Angelico <rosuav@gmail.com>:
On 11/21/2014 08:22 AM, Andrew Barnert wrote:
Maybe what we need to do is update PEP 3142, gathering all of the
alternative ideas,
and expanding on the rationale, so Guido can reject that, and next time
On Sat, Nov 22, 2014 at 3:36 AM, Ethan Furman <ethan@stoneleaf.us> wrote: this comes
up, everyone will have something to read before having the same arguments about the same proposals?
Are we accepting nominations? I have the perfect fellow in mind. :)
(And in case Chris decides to nominate me in retaliation, I decline. ;)
Heh! Great, I'm going to get a name for writing the PEPs that exist solely to be rejected... I was hoping PEP 479 would be accepted easily, given who was advocating it, but now it's starting to look like I doomed the proposal :)
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Nov 21, 2014, at 9:41, Antony Lee <antony.lee@berkeley.edu> wrote:
I would like to believe that "break"-as-a-expression solves most of these issues.
This one has been raised every time in the past. Since we don't have the PEP, I'll try to summarize the problems. But first:
It is reasonable, though, to drop the "return"-as-an-expression part of the proposal, because you need to know that the comprehension is run in a separate function to understand it (it's a bit ironic that I am dropping the original proposal to defend my own, now...).
There's a more significant difference between the two proposals. A break expression allows you to break out of a single loop, but gives you no way to break out of the whole thing (as your example shows). A return expression allows you to break out of the whole thing, but gives you no way to break out of a single loop. That's why they're not interchangeable in explicit loop code. (Also, notice that every so often, someone proposed a numbered or labeled break, and the answer is always "Why would you need that? If your code has enough loops that you need to break out of some but not all, refactor those some into a separate function and then use return.") This also raises the question of why one of break/continue/return should be an expression but not the others. If your answer is "because you only need continue when there's code not controlled by the if, which is impossible in a comprehension" then you're pretty much admitting that the flow control expression thing is not really a general purpose thing, but a hack made for comprehensions.
Consider
((x, y) for x in l1 if f1(x) or break for y in l2 if f2(y) or break)
This maps directly to
for x in l1: if f1(x) or break: for y in l2: if f2(y) or break: yield x, y
When you write it this way, it's pretty clear that you're abusing or as an else, and that it comes in the wrong place, and that you're cramming side effects into an expression with no value. Would you put anything else with side effects here, even a call to a logging function? You're also introducing unnecessary indentation, and making the code more verbose by comparison with the obvious alternative: for x in l1: if not f1(x): break for y in l2: if not f2(y): break yield x, y Yes, that alternative can't be written as a comprehension. But that doesn't mean we should come up with a less obvious, more verbose, and harder to reason about alternative just because it can be written as a comprehension, even though we'd never write it as an explicit loop. If you want to go that route, we might as well just enshrine the "or stop()" hack instead of trying to break it. Also, can you imagine using break as an expression in any other context besides as an or operand in an if statement directly under a for statement?
2014-11-21 10:06 GMT-08:00 Andrew Barnert <abarnert@yahoo.com>:
On Nov 21, 2014, at 9:41, Antony Lee <antony.lee@berkeley.edu> wrote:
I would like to believe that "break"-as-a-expression solves most of these issues.
This one has been raised every time in the past. Since we don't have the PEP, I'll try to summarize the problems. But first:
It is reasonable, though, to drop the "return"-as-an-expression part of the proposal, because you need to know that the comprehension is run in a separate function to understand it (it's a bit ironic that I am dropping the original proposal to defend my own, now...).
There's a more significant difference between the two proposals. A break expression allows you to break out of a single loop, but gives you no way to break out of the whole thing (as your example shows). A return expression allows you to break out of the whole thing, but gives you no way to break out of a single loop. That's why they're not interchangeable in explicit loop code. (Also, notice that every so often, someone proposed a numbered or labeled break, and the answer is always "Why would you need that? If your code has enough loops that you need to break out of some but not all, refactor those some into a separate function and then use return.")
This also raises the question of why one of break/continue/return should be an expression but not the others. If your answer is "because you only need continue when there's code not controlled by the if, which is impossible in a comprehension" then you're pretty much admitting that the flow control expression thing is not really a general purpose thing, but a hack made for comprehensions.
I guess you're going to make me flip-flop and go back to defend having all three of "break", "return" and "raise" as expressions (I can't see a single use of "continue" as an expression but would be happy to be proven wrong). The issue of "return" as an expression is obviously that it exposes the internals of generators (i.e. they are in their own function) but I can live with that. ("raise" as expressions is a separate issue, the main application being to stuff it into lambdas.)
Consider
((x, y) for x in l1 if f1(x) or break for y in l2 if f2(y) or break)
This maps directly to
for x in l1: if f1(x) or break: for y in l2: if f2(y) or break: yield x, y
When you write it this way, it's pretty clear that you're abusing or as an else, and that it comes in the wrong place, and that you're cramming side effects into an expression with no value. Would you put anything else with side effects here, even a call to a logging function?
You're also introducing unnecessary indentation, and making the code more verbose by comparison with the obvious alternative:
for x in l1: if not f1(x): break for y in l2: if not f2(y): break yield x, y
Yes, that alternative can't be written as a comprehension. But that doesn't mean we should come up with a less obvious, more verbose, and harder to reason about alternative just because it can be written as a comprehension, even though we'd never write it as an explicit loop. If you want to go that route, we might as well just enshrine the "or stop()" hack instead of trying to break it.
The introduction of unnecessary indentation is honestly not there to make the non-generator example look bad, but simply to show how the proposal addresses the issue of translating nested loops. For the non-nested loop case, a simpler and arguably more readable way to write it would be (x if f(x) else break for x in X) Perhaps giving the nested case first was poor PR; yes, the nested case is slightly hackish but I am fairly sure anybody can figure out the meaning of the above.
Also, can you imagine using break as an expression in any other context besides as an or operand in an if statement directly under a for statement?
On Fri, Nov 21, 2014 at 9:41 AM, Antony Lee <antony.lee@berkeley.edu> wrote:
I would like to believe that "break"-as-a-expression solves most of these issues. It is reasonable, though, to drop the "return"-as-an-expression part of the proposal, because you need to know that the comprehension is run in a separate function to understand it (it's a bit ironic that I am dropping the original proposal to defend my own, now...).
Consider
((x, y) for x in l1 if f1(x) or break for y in l2 if f2(y) or break)
This maps directly to
for x in l1: if f1(x) or break: for y in l2: if f2(y) or break: yield x, y
which is literally a copy-paste of the second part followed by yielding the first part. I think this reads reasonably well but this is obviously a subjective issue.
I really hope "or break" doesn't become an idiom if break is turned into an expression. I find [x if x < N else break for x in ...] so much more readable than [x for x in ... if x < N or break] This is very near (but not directly equivalent) to the relatively idiomatic: for x in ...: if x < N: yield x else: break -- Devin
If someone _is_ going to do this, I have one more suggestion that I didn't raise last time around, because I don't want to argue for it, but for completeness: Briefly: add if and while clauses to the for statement; redefine comprehensions as nested for loops. The basic problem is that Python comprehensions map to arbitrarily nested for and if statements, and there's no way to add "while" semantics cleanly with another nested statement. If you look at the languages that have while clauses, this is not how their comprehensions work. Some of them, they have nested loops only, but each loop has optional modifiers that Python's don't: an if or when clause that filters, a while clause that terminates early, maybe unless and until clauses that do the same but with negated conditions (although until has a choice between doing the last iteration or not doing it). This allows for everything you can do in Python except multiple if clauses on the same for clause (which are usually better rewritten with and--or, when that's too unwieldy, you probably shouldn't have been writing it as a one-liner). In others, there's a single loop, but it takes nested iterable expressions, so you only get one set of modifiers for all of the iterables. Others just don't have any syntax for nested iteration at all; you have to turn it upside-down and use the outer loop as the iterable for the inner loop. This would have another minor arguable benefit. Every once in a while I see some code like this: for x in (x for x in xs if f(x)): ... because the writer didn't think long enough to realize he doesn't need the comprehension. This code is obviously a lot better as: for x in xs if f(x): But that example shows exactly why I don't think this is worth arguing for. The same code is even better as: for x in xs: if f(x): And that brings us back to the reason Clojure, Racket, etc. have clauses on their for loops: Because they don't have loop statements, they have a loop function. (Well, usually it's a macro that quotes the arguments so you don't have to lambda all your expressions into functions, and to allow you to avoid at least one excess repetition, but let's ignore that.) The when and while clauses are just keyword arguments to the function. More generally, they encourage or require you to write everything as a complex expression, which is the exact opposite of what Python encourages. Still, just because the idea comes from languages that are very different from Python doesn't necessarily mean it wouldn't fit; after all, that's how we got comprehensions in the first place. Sent from a random iPhone On Nov 21, 2014, at 8:36, Ethan Furman <ethan@stoneleaf.us> wrote:
On 11/21/2014 08:22 AM, Andrew Barnert wrote:
Maybe what we need to do is update PEP 3142, gathering all of the alternative ideas, and expanding on the rationale, so Guido can reject that, and next time this comes up, everyone will have something to read before having the same arguments about the same proposals?
Are we accepting nominations? I have the perfect fellow in mind. :)
(And in case Chris decides to nominate me in retaliation, I decline. ;)
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Nov 20, 2014, at 10:49, MRAB wrote:
I can see the problem with 'while': if there are multiple 'for' parts, they are equivalent to nested 'for' loops, so you might assume that a 'while' part would also be equivalent to a nested 'while' loop, whereas it would, in fact, be controlling the preceding 'for' part.
In other words, it would be:
for x in range(10) while x < 5: ....
but could be seen as:
for x in range(10): while x < 5: ....
I've always thought of comprehensions as a sort of "inside-out" loop, due to the value at the beginning: (f(a) for a in range) is roughly equivalent to for a in range: yield f(a). Due to this, I was actually somewhat surprised to find that multiple for clauses don't work as if they were nested inside-out loops
[(a, b) for a in range(3) for b in range(3)] [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)] [[(a, b) for a in range(3)] for b in range(3)] [[(0, 0), (1, 0), (2, 0)], [(0, 1), (1, 1), (2, 1)], [(0, 2), (1, 2), (2, 2)]]
I think people trying to use "while" for this are coming from an english-grammar point of view.
On 11/20/2014 10:49 AM, MRAB wrote:
I can see the problem with 'while': if there are multiple 'for' parts, they are equivalent to nested 'for' loops,
x for x in A if x=B is also equivalent to nested compound statements.
so you might assume that a 'while' part would also be equivalent to a nested 'while' loop, whereas it would, in fact, be controlling the preceding 'for' part.
-- Terry Jan Reedy
On Thu, Nov 20, 2014 at 10:01 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
list(x for x in range(10) if x < 5 or return) [0, 1, 2, 3, 4]
I'd be concerned about the confusion engendered by 'return' having totally different semantics within very small regions, e.g. we'd have def takewhile(iterable, pred): return (x for x in iterable if pred(x) or return) Esp. because normally 'return' is very powerful -- right now any mention of that token anywhere in a function body jumps out of the entire function, so you really have to scan for them actively when trying to understand a function's flow control. Not a big fan of a flow control expression in general, really; yield expressions are kinda flow-controlly, but they just pause execution, they don't disrupt the logical through-line of a block of code. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
On 11/20/2014 04:01 AM, Nick Coghlan wrote:
This idea occurred to me in context of PEP 479 and the "or stop()" hack for generator expressions. I'm not a big enough fan of the idea to pursue it myself, but if anyone is bothered by the prospect of PEP 479 taking the "or stop()" technique away, this may be worth pursuing further.
As a reminder of how the hack works, the following generator:
def takewhile(iterable, pred): ... for x in iter(iterable): ... if not pred(x): ... return ... yield x ... list(takewhile(range(10), (lambda x: x < 5))) [0, 1, 2, 3, 4]
Just a note: If this generator was used on an iterator more than once it would drop items between groups. Takewhile fits the peek-ahead pattern I mentioned in an earlier thread that groupby matches. I think groupby manages to hide the looked ahead value in the yield path, but it still does look ahead, just not in an obvious way. It's my understanding looking ahead is problomatic for genrators because they may have side effects when next() is called on them. Like reading input or writing output, or accessing some other object outside the generator. It seams to me that most (if not all) partial iterations will either want to yield the "last" value tested, or stop "before" the last value tested. The generator expression version of the above has issues with both of those. Cheers, Ron
Ron Adam wrote:
It's my understanding looking ahead is problomatic for genrators because they may have side effects when next() is called on them.
Not just generators, but any iterator. Any algorithm that requires lookahead has problems if you apply it more than once to the same iterator. Something has to store the lookahead value in between times, and the iterator protocol just doesn't allow for that. -- Greg
Nick Coghlan wrote:
list(x for x in range(10) if x < 5 or return) [0, 1, 2, 3, 4]
This is emphatically *not* executable pseudocode - it only makes sense in terms of the translation to the corresponding full generator function
-1 on messing with the semantics of 'return' (which would have implications beyond this use case) just so you can write something in one particular context that doesn't read intuitively. If early exit from comprehensions is really a desirable feature (and Guido doesn't seem to think so) it would be better addressed with an explicit language feature, e.g. list(x for x in range(10) while x < 5) [x for x in range(10) while x < 5] This makes the intent perfectly clear and works just as well in all kinds of comprehensions. It breaks the strict equivalence between comprehension syntax and the corresponding nested loop code, but you can't have everything. -- Greg
On 11/20/2014 04:01 AM, Nick Coghlan wrote:
Can currently be converted to a generator expression with the aid of a helper function:
def stop(): ... raise StopIteration ... list(x for x in range(10) if x < 5 or stop()) [0, 1, 2, 3, 4]
How about making a stop_generator() function builtin, just for this feature? It could raise a specific StopIteration instance, possibly the same subclass/instance as return does in generators, so the generator would know to catch it instead of complaining. As a builting, all the mechanics are kept out of the way as implementation details. For those who already use stop() in many places, if there are any one who does that often, they could just do stop = stop_generator at the top of their programs. I think "stop" is too general a name in this case. It's fine when the function is defined right above where it is used, but as a builtin a more specific name would be better. Cheers, Ron
participants (13)
-
Andrew Barnert -
Antony Lee -
Chris Angelico -
Devin Jeanpierre -
Ethan Furman -
Greg Ewing -
MRAB -
Nathaniel Smith -
Nick Coghlan -
Petr Viktorin -
random832＠fastmail.us -
Ron Adam -
Terry Reedy