[Python-ideas] Allowing breaks in generator expressions by overloading the while keyword

[Andrew Barnert]

On Feb 21, 2014, at 2:24, Steven D'Aprano <steve at pearwood.info> wrote:

> On Fri, Feb 21, 2014 at 02:13:13AM +0000, Carl Smith wrote:
>> Sometimes you need to build a list in a loop, but break from the loop if
>> some condition is met, keeping the list up to that point. This is so common
>> it doesn't really need an example.
>> 
>> Trying to shoehorn the break keyword in to the generator expression syntax
>> doesn't look pretty, unless you add `and break if expr` to the end, but
>> that has its own issues.
> 
> The main issue being that it gives a SyntaxError :-)
> 
> 
>> Besides, overloading `while` is much cuter...
>> 
>>    ls = [ expr for name in iterable while expr ]
> 
> I would *love* this syntax. I've read the various arguments against it, 
> and they don't convince me. It is functionality which is often needed 
> and requested, e.g.
> 
> http://stackoverflow.com/questions/5505891/using-while-in-list-comprehension-or-generator-expressions
> 
> 
> Clojure has list comprehensions that behave this way:
> 
> http://clojuredocs.org/clojure_core/clojure.core/for

As I pointed out in the thread last summer (and briefly mentioned in the blog post linked earlier in this thread), that's because Clojure comprehensions are not actually the same as Python comprehensions.

Python comprehensions, as borrowed from Haskell, have a free sequence of clauses that nest from left to right. There is no sensible way to interpret a while clause that nests under a for or if clause.

Clojure-style comprehensions, as borrowed from (I think) Racket only have nested loops, no other clauses--but then each loop has modifiers. Instead of if clauses, you have a "filter" that modifies a loop. You can't have a while clause, but you can have a "takewhile" that modifies a loop. You get a clean, simple, and easy-to-extend syntax.

This has the down side that you can't toss two if conditions into a single loop, but that almost never matters (just and them together) It has the up side that new modifiers don't have to nest under arbitrary clauses, they just have to modify a loop.

In Haskell, because it's so trivial to modify the iterable itself (e.g., with a takewhile whose predicate is composed on the fly), there really is no cost to using the Haskell-style comprehension. Sadly, that isn't true in Python, so the limitation hits us harder. 

Can we get from here to there?

Well, it might be possible to come up with a syntax for Clojure-style comprehensions that handles all currently valid Python comprehensions without two adjacent if clauses with the same meaning, despite getting there by a slightly different route. If so, just deprecate adjacent if clauses, then switch to the syntax and semantics where if is a modifier on the for clause instead of a separate clause and adjust the docs accordingly, then add while as another modifier.

Or, it would almost certainly be possible to design a hybrid, where while is a loop modifier on the for clause rather than a nested clause, but if stays the way it is today. And if you go that way, it probably makes sense to add the same modifier onto the for statement. Then all of the docs that explain comprehensions could remain unchanged. And no need for a deprecation period or removing adjacent ifs.

This may sound more radical than "just add a while clause", but given that the latter isn't actually a sensible proposal in the first place, you're not going to be able to flesh it out in a way that gets Nick Coghlan to reconsider; the former, you might have a shot at.

> but alas both Nick Coglan and (if I recall correctly) Guido have ruled 
> that Python won't get this, so until the Revolution comes, it isn't 
> going to happen.

You could always just fork Python instead of revolting. ;)