This has come up before. It will be a special case of making "if" without "else" result in a special "empty" type that is not part of the iteration. As in `[1, (2 if False) ] == [1]`. בתאריך יום א׳, 11 בספט' 2016, 13:29, מאת Bernardo Sulzbach ‏< mafagafogigante@gmail.com>:

On 09/11/2016 06:36 AM, Dominik Gresch wrote:

...

So I asked myself if a syntax as follows would be possible:

for i in range(10) if i != 5: body

Personally, I find this extremely intuitive since this kind of if-statement is already present in list comprehensions.

What is your opinion on this? Sorry if this has been discussed before -- I didn't find anything in the archives.

I find it interesting.

I thing that this will likely take up too many columns in more convoluted loops such as

for element in collection if is_pretty_enough(element) and ...: ...

However, this "problem" is already faced by list comprehensions, so it is not a strong argument against your idea.

I am still unsure about whether or not the pattern you describe is frequent enough to justify special syntax. Not to mention that the current way to do it is already **very** readable. Just notice how

for e in l: if e != 2: ...

and

for e in l if e != 2: ...

read essentially the same and take about the same number of keystrokes. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Sep 11, 2016 at 9:59 PM, Paul Moore wrote:

# Hacky use of a generator expression: for i in (x for x in range(10) if x != 5): body()

This is what I'd like to compare the proposal against. It's perfectly legal but pretty ugly - why should you nest two 'for' loops just for the sake of filtering?

# Functional style for i in filter(range(10), lambda n: n != 5): body()

And this one is very close (I'd use i instead of n in the lambda function), but still fairly verbose. That said, though, filtered iteration isn't common enough to demand its own syntax IMO. I do it fairly often, but it's usually fine to just have a condition on a separate line. (I do use ": continue" rather than making it two lines.) ChrisA

On 11.09.2016 22:15, C Anthony Risinger wrote:

On Sep 11, 2016 7:11 AM, "Chris Angelico" mailto:rosuav@gmail.com> wrote:

...

That said, though, filtered iteration isn't common enough to demand its own syntax IMO. I do it fairly often,

I do it often enough to want this.

Same here. Most of the time it's just a single condition which disturbs the coherence of the loop body.

When I first started writing Python this struck me as an inconsistency... if it's useful in comprehensions, why not regular loops? I realize comprehensions are all about construction of the list itself, but the parallel still exists.

Also feels similar to guard statements. I'd love to see Python gain more pattern matching and destructuring features because they are wonderful in Erlang/Elixir.

...

but it's usually fine to just have a condition on a separate line. (I do use ": continue" rather than making it two lines.)

FWIW, code I write or review would mandate this be two lines followed by a blank line, so 3 total. I require any abrupt change or termination in the current flow of control to be followed by a blank line so the reader clearly sees the possible jump (continue, break, and return especially).

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Ok, I guess it's time to end this thread. Thank you all for your answers and the constructive discussion. Best, Dominik On 12.09.2016 08:25, Nick Coghlan wrote:

...

Hi,

I've recently found myself writing code similar to this:

for i in range(10): if i == 5: continue "body"

which I find a bit ugly. Obviously the same could be written as

for i in range(10): if i != 5: "body"

but here you would have to look at the end of the body to see if something happens when i==5. So I asked myself if a syntax as follows would be possible:

for i in range(10) if i != 5: body

Personally, I find this extremely intuitive since this kind of if-statement is already present in list comprehensions.

What is your opinion on this? Sorry if this has been discussed before -- I didn't find anything in the archives. Generally speaking, we only add new syntax in cases where we're

On 11 September 2016 at 19:36, Dominik Gresch wrote: prepared to say "In all cases where the new syntax applies, it should be used in preference to existing alternative spellings".

Special casing a single "if-continue" in a loop body doesn't meet that (deliberately high) bar, with Paul Moore's email going into some more detail on the specifics of that.

Cheers, Nick.

On 28 September 2016 at 00:55, Erik Bray wrote:

On Sun, Sep 11, 2016 at 12:28 PM, Bernardo Sulzbach wrote:

...

On 09/11/2016 06:36 AM, Dominik Gresch wrote:

...

So I asked myself if a syntax as follows would be possible:

for i in range(10) if i != 5: body

Personally, I find this extremely intuitive since this kind of if-statement is already present in list comprehensions.

What is your opinion on this? Sorry if this has been discussed before -- I didn't find anything in the archives.

I find it interesting.

I thing that this will likely take up too many columns in more convoluted loops such as

for element in collection if is_pretty_enough(element) and ...: ...

However, this "problem" is already faced by list comprehensions, so it is not a strong argument against your idea.

Sorry to re-raise this thread--I'm inclined to agree that the case doesn't really warrant new syntax. I just wanted to add that I think the very fact that this syntax is supported by list comprehensions is an argument *in its favor*.

I could easily see a Python newbie being confused that they can write "for x in y if z" inside a list comprehension, but not in a bare for-statement. Sure they'd learn quickly enough that the filtering syntax is unique to list comprehensions. But to anyone who doesn't know the historical progression of the Python language that would seem highly arbitrary and incongruous I would think.

Just $0.02 USD from a pedagogical perspective.

This has come up before, and it's considered a teaching moment regarding how the comprehension syntax actually works: it's an *arbitrarily deep* nested chain of if statements and for statements. That is: [f(x,y,z) for x in seq1 if p1(x) for y in seq2 if p2(y) for z in seq3 if p3(z)] can be translated mechanically to the equivalent nested statements (with the only difference being that the loop variable leak due to the missing implicit scope): result = [] for x in seq1: if p1(x): for y in seq2: if p2(y): for z in seq3: if p3(z): result.append(f(x, y, z)) So while the *most common* cases are a single for loop (map equivalent), or a single for loop and a single if statement (filter equivalent), they're not only the forms folks may encounter in the wild. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 27 September 2016 at 16:54, Erik Bray wrote:

Then following my own logic it would be desirable to also allow the nested for loop syntax of list comprehensions outside them as well.

I'd say that it's a case where we should either allow arbitrary concatenation outside of comprehensions, or we allow none. And as arbitrary concatenation is obviously bad, the only sane choice is no nesting :-) Paul

On Tue, Sep 27, 2016, at 17:36, Greg Ewing wrote:

Erik Bray wrote:

...

Then following my own logic it would be desirable to also allow the nested for loop syntax of list comprehensions outside them as well.

The only use for such a syntax would be to put an inadvisable amount of stuff on one line.

The only difference between an inadvisable amount of stuff and a reasonable amount of stuff is the amount. Don't we already have a recommended column limit in PEP8? for y in range(25) for x in range(80): doesn't seem unreasonable to me; YMMV. Or maybe you want to put it on multiple lines, but not consuming indentation levels: for x in seq1 if p1(x)\ for y in seq2 if p2(y)\ for z in seq3 if p3(z): result.append(f(x, y, z))

On 1 October 2016 at 10:25, Alexander Heger wrote:

I think wasting of indentation levels for a single logical block should be avoided if possible to make the code more legible, otherwise one hits the suggested line length limit too fast - suppose this is now inside a method, you already lose at least 8 char ...

Hence generators, which allow the nested loops to be readily factored out into a named operation. def iter_interesting_triples(seq1, seq2, seq3): for x in seq1: if p1(x): for y in seq2: if p2(x, y): for z in seq3: if p3(x, y, z): yield x, y, z for x, y, z in iter_interesting_triples(seq1, seq2, seq3): f(x, y, z) As you pointed out, the simple cases with no filtering, or only filtering that depends on the value of "z", are already covered by itertools.product: for x, y, z in itertools.product(seq1, seq2, seq3): f(x, y, z) for x, y, z in itertools.product(seq1, seq2, seq3): if p(x, y, z): f(x, y, z) And that step of separating the process of generating the candidate triples from actually doing the work on them also opens up a whole new world of possibilities when it comes to your execution model, like using concurrent.futures to process them in parallel in multiple threads or processes. There's no question that modeling algorithms as purely procedural code starts breaking down beyond a certain level of complexity. The fact that *not* breaking up our Python code into more modular units gets painful as we start hitting those limits as the author of the code in question is a side effect of that - it's a warning that we have an imminent readability problem, just as written documents start needing section headings to aid reader navigation as they get longer. As a result, much of the art of developing maintainable software lies in building out the refactoring tools we have available to us when we hit those limits, and in the case of Python, that means using features like functions to factor out request/response operations, generators to factor out a result series, classes to factor out structurally related data, context managers to separate out before/after structures, couroutines to separate out interactive agents, etc. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi, On 11/09/16 10:36, Dominik Gresch wrote:

So I asked myself if a syntax as follows would be possible:

for i in range(10) if i != 5: body

I've read the thread and I understand the general issues with making the condition part of the expression. However, what if this wasn't part of changing the expression syntax but changing the declarative syntax instead to remove the need for a newline and indent after the colon? I'm fairly sure this will have been suggested and shot down in the past, but I couldn't find any obvious references so I'll say it (again?). The expression suggested could be spelled: for i in range(10): if i != 5: body So, if a colon followed by another suite is equivalent to the same construct but without the INDENT (and then the corresponding DEDENT unwinds up to the point of the first keyword) then we get something that's pretty much as succinct as Dominik suggested. Of course, we then might get: for i in myweirdobject: if i != 5: while foobar(i) > 10: while frob(i+1) < 99: body ... which is hideous. But is it actually _likely_? E.

In my experience it is exceptions and inconsistencies that consume 'working memory in the brain of humans'. By eliminating the distinction between list comprehensions and for loops we would be making the language simpler by eliminating an inconsistency. Furthermore, I do not believe it is valid to discard a potentially good idea simply because if taken to extreme it might result in ugly code. With that justification one could reject most ideas. The fact is, that in many cases this idea would result in cleaner, more compact code. We should be content to offer a language in which it is possible to express complex ideas cleanly and simply, and trust our users to use the language appropriately. For example, it was suggested that one could simplify a multi-level loop by moving the multiple levels of for loop into a separate function that acts as generator. And that is a nice idea, but when writing it, the writing of the generator function represents a speed bump. Whereas writing something like the following is simple, compact, quick, and obvious. There is no reason why it should not be allowed even though it might not always be the best approach to use: for i in range(5) for j in range(5) for k in range(5): ... And I would really like to be able to write loops of the form: for item in items if item is not None: ... It is something I do all the time, and it would be nice if it did not consume two levels on indentation. -Ken On Tue, Oct 04, 2016 at 01:31:22PM +1000, Nick Coghlan wrote:

On 4 October 2016 at 08:18, Erik wrote:

...

The expression suggested could be spelled:

for i in range(10): if i != 5: body

So, if a colon followed by another suite is equivalent to the same construct but without the INDENT (and then the corresponding DEDENT unwinds up to the point of the first keyword) then we get something that's pretty much as succinct as Dominik suggested.

What's the pay-off though? The ultimate problem with deeply nested code isn't the amount of vertical whitespace it takes up - it's the amount of working memory it requires in the brain of a human trying to read it. "This requires a lot of lines and a lot of indentation" is just an affordance at time of writing that reminds the code author of the future readability problem they're creating for themselves.

Extracting named chunks solves the underlying readability problem by reducing the working memory demand in reading the code (assuming the chunks are well named, so the reader can either make a useful guess about the purpose of the extracted piece without even looking at its documentation, or at least remember what it does after looking it up the first time they encounter it).

By contrast, eliminating the vertical whitespace without actually reducing the level of nesting is merely hiding the readability problem without actually addressing it.

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 4 October 2016 at 08:56, Stephen J. Turnbull wrote:

These are my opinions; I don't claim any authority for them. I just don't find the proposed syntax as obvious and unambiguous as you do, and would like to explain why that is so.

Ken Kundert writes:

...

In my experience it is exceptions and inconsistencies that consume 'working memory in the brain of humans'. By eliminating the distinction between list comprehensions and for loops we would be making the language simpler by eliminating an inconsistency.

I don't think of a comprehension as a for loop, I think of it as setbuilder notation (although of course I realize that since lists are sequences it has to be a for loop under the hood). So the claimed inconsistency here doesn't bother me. I realize it bothers a lot of people, but the proposed syntax is not obvious to me (ambiguous and inconsistent in its own way).

...

[T]he writing of the generator function represents a speed bump.

It used to be, for me, but it really isn't any more. Perhaps you might get used to it if you tried it. Harder to argue: the fact that Guido and Nick (inter alia) consider it good style to use named functions makes that point a hard sell (ie, you don't need to convince me, you need to convince them).

...

Whereas writing something like the following is simple, compact, quick, and obvious. There is no reason why it should not be allowed even though it might not always be the best approach to use:

for i in range(5) for j in range(5) for k in range(5): ...

To me, that is visually ambiguous with

for i in (range(5) for j in (range(5) for k in range(5))): ...

although syntactically the genexp requires the parentheses (and in fact is almost nonsensical!) I could easily see myself forgetting the parentheses (something I do frequently) when I *do* want to use a genexp (something I do frequently), with more or less hilarious results. As already mentioned:

for i, j, k in itertools.product(range(5), range(5), range(5)): ...

To me that is much clearer, because it expresses the rectangular shape of the i, j, k space. I would also stumble on

for i in range(5) for j in range(i + 1): ...

at least the first few times I saw it. Based on the English syntax of "for" (not to mention the genexp syntax), I would expect

for j in range(i + 1) for i in range(5): ...

If itertools.product is the wrong tool, then the loop bodies are presumably complex enough to deserve new indent levels. Note that simple filters like non_nil (see below) can easily be used, as long as the resulting set is still a product.

...

And I would really like to be able to write loops of the form:

for item in items if item is not None: ...

def non_nil(items): return (item for item in items if item is not None)

for item in non_nil(items): ...

I think that's very readable, so the only reason why that 2-line function needs to be syntax that I can see is your distaste for defining functions, and that of other Python programmers who think like you.

Again this is just personal opinion, but I agree 100% with everything Stephen said. It *is* a stumbling block to get used to writing generator functions like non_nil() above, but it's also a worthwhile learning experience. And yes, it's somewhat inconvenient to do so if you're working in the standard Python REPL, but designing language features around REPL usage isn't (IMO) the right choice. And if you really need a better way of handling that sort of refactoring in an interactive environment, tools like the Jupyter notebook are probably what you're looking for. (Trying to collapse multiple clauses into one line/statement is something Perl was famous for, and it's in many ways quite an attractive feature. But IMO it directly contributes to Perl's reputation for unreadability, because it *does* get used too much, whether you think it will or not - one person's "nicely compact" is another person's "obfuscated". So I'm glad that Python's design avoids encouraging that style - even though I do occasionally remember fondly my Perl one-liners :-)) Paul

On Tue, Oct 04, 2016 at 01:31:22PM +1000, Nick Coghlan wrote:

By contrast, eliminating the vertical whitespace without actually reducing the level of nesting is merely hiding the readability problem without actually addressing it.

+1 Extra newlines are cheap. Writing for x in expression: if condition: block is a simple, clean idiom that is easy to understand, avoids a number of pitfalls (where do you put the elif or else if you need one?), and only costs one extra line and one extra indent. If you have so many indents that this is a problem, that's a code smell and you ought to think more closely about what you are doing. There's another variation that saves an indent for the cost of one more line: for x in expression: if not condition: continue block In contrast, comprehensions are a single expression and are expected to usually be written in one line, although that's often hard to do without very long lines. They cannot include elif or else clauses, so avoid that particular pitfall. But the "very long line" problem shows that they are too dense: simple examples look fine: [x+1 for x in seq if cond] but in practice, they're often much longer with a much higher density of code: [default if obj is None else obj.method(arg) for (obj, count) in zip(values, counts) if count > 1] Some compromise on the optimal level of readability and code density is allowed: that's the price we pay in order to squeeze everything into a single expression. But that is not something we ought to copy when we have the luxury of a suite of statements. -- Steve

On 5 October 2016 at 05:09, Ken Kundert wrote:

On Wed, Oct 05, 2016 at 03:07:42AM +1100, Steven D'Aprano wrote:

...

Extra newlines are cheap. Writing

The cost is paid in newlines *and* extra levels of indentation.

No extra indentation if you ise "if not condition: continue" or refactor the condition into a custom iterable. Both of which have already been mentioned here as ways of achieving the desired result without a language change.

Why isn't it the programmer that is writing the code the best person to decide what is best?

Because the programmer writing the code isn't going to write and maintain the changes to the CPython/Jython/PyPy codebases, write the tests and documentation, support the questions that come from other users, etc...? More seriously, that argument could apply to *any* proposal. "Let the user decide whether to use the feature or not, and just add it". However, not all features get added precisely because someone has to make a cost/benefit judgement on any proposal and the people who do that are the CPython core devs. Discussion on this list is about thrashing out convincing arguments that will persuade the core devs - which is one of the reasons a lot of the core devs hang out here, to provide a sounding board on whether arguments are convincing or not. "Make the feature available and let the user decide if they want to use it" isn't a convincing argument. At best it could be a small part of a larger argument. It's countered by "does it make the language harder to teach having multiple ways of doing things?", "what about edge cases?" (in this case, trailing elses have been mentioned), "is there a well-known and easy workaround?", "no other languages (apart from Perl) seem to have this feature", ... and those issues need to be addressed in a full proposal. Paul

On Tue, Oct 04, 2016 at 09:09:40PM -0700, Ken Kundert wrote:

On Wed, Oct 05, 2016 at 03:07:42AM +1100, Steven D'Aprano wrote:

...

Extra newlines are cheap. Writing

The cost is paid in newlines *and* extra levels of indentation.

You've quoted me out of context -- I did also refer to extra indentation being cheap. At the point that it isn't any more, it is a code smell and you (that's generic you, not just you personally) should think hard about how the design of your code.

Why isn't it the programmer that is writing the code the best person to decide what is best?

Have you *seen* the quality of code written by the average coder? And remember, fifty percent of coders are worse than that. I jest, but only a bit. For better or worse, of course every programmer can set their own style, within the constraints of the language. If they cannot bear the language contraints, they're free to use a different language, or design their own. Anyone can be "the best person to decide" for their own private language. All languages have their own style, of what is or isn't allowed, what's encouraged and what's discouraged, and their own idiomatic way of doing things. The syntax constraints of the language depend on the language designer, not the programmers who use it. For some languages, those constraints are set by those who are appointed to sit on a standards board, usually driven by the corporations with the deepest pockets. Python, it is Guido and the core developers who set the boundaries of what coding styles can work in Python, and while the community can influence that, it doesn't control it. It isn't a wild free-for-all where every programmer is "the best person to decide". Some people might think that moving closer towards a Perl-ish one-liner culture by allowing (say): for x in seq for y in items if cond: block makes Python better ("saves some lines! saves some indents!"), but to those who like the discipline and structure of Python's existing loop syntax, this will make Python significantly worse. No decision can please everybody. -- Steve

On 5 October 2016 at 14:27, Sven R. Kunze wrote:

For small scripts this is still useful. Not everybody writes huge programs, which needs to adhere to style guides and QA.

Sure. But convenience in small scripts and the REPL typically isn't a good enough argument to justify a language change. It's (to an extent) a point in favour of the proposal, but nobody's debating that. The problem is that we aren't seeing any *other* arguments in favour. And specifically not ones that provide benefits (or at least no disadvantages - "you don't need to use it" isn't enough, someone will be bound to try to and it'll have to be dealt with, hopefully in code review but maybe in maintenance) to large-scale production code, which is probably the vast majority of Python usage[1]. Let's take "it helps interactive use and quick scripts" as a given, and move on. Any other benefits? Readability has been demonstrated as subjective, so let's skip that. Paul [1] Although these days, data analysis / interactive exploration may be a growing proportion...

On 5 October 2016 at 17:26, Nick Coghlan wrote:

Compared to those, locally modifying the token stream to inject ": INDENT" pairs when the if and for keywords are encountered between an opening "for" keyword and a closing ":" keyword would be a relatively straightforward change that only impacted folks that decided they preferred that particular flavour of Abbreviated Python to the regular version.

It's also worth noting that the obvious response "but I don't want to have to run a preprocessor against my code" is another indication that this isn't solving a significant enough problem to warrant a language change. Again, this isn't a hard and fast rule, but it is a useful rule of thumb - how much effort are you willing to go to to get this feature without it being built in? That's one of the reasons "it should be made into a module on PyPI" is a useful counter to proposals for new stdlib functions. It's also worth looking at the cases where things get added despite not going via that route - sometimes "being built in" is an important benefit of itself. But typically that's because people are encouraged to use built in facilities, so guiding beginners (or not-so-beginners) into good practice is important. In this case, it's far from clear that the feature is actually good practice. Paul