Re: [Python-Dev] For/while/if statements/comprehension/generator expressions unification
Hello Josiah, JC> Alexander, JC> The essence of what you have proposed has been proposed (multiple times) before, JC> and I seem to remember it was shot down. To increase my understanding of Python-way, can you (or someone else) explain the reasons why such proposals were rejected? JC> The below functions offer the equivalent of list comprehensions with a JC> final post-processing step. Well, what I was suggesting is not just a cross of two lists but the syntax sugar which would make statements more consistent to the generators/comprehensions and also give some new opportunities. I think that my large proposal can be splitted to several almost-independent ones, each carrying separate features, able to be implemented independently and worth independent reconsidering: 1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' statement. That's truly inconsistent that one may write list2 = [i for i in list if cond(i)] but cannot write for i in list if cond(i): 2. Bring "several for-s in a row" ability from generators/comprehensions to the statement (and expand it to "several for-s and if-s", of course). We can write list2 = [f(i, j) for i in list1 for j in list2] but cannot write for i in list1 for j in list2: yield f(i, j) That looks inconsistent as well. Yes, for this little case we could do some kind of cross, but what if there are more loops, and/or some of them require filtering by if-s? 3. Bring 'while' into the loops, both statements and iterators. Now no need to worry about "hidden-goto" 'break', especially for the cases of nested loops, while it can be easily checked right inside the looping condition? Now we can easily do for i in largelist while !found: , isn't it nice and simple? -- С уважением, Alexander mailto:maa_public@sinn.ru
Alexander Myodov wrote:
To increase my understanding of Python-way, can you (or someone else) explain the reasons why such proposals were rejected?
I wasn't around for the discussion Josiah refers to, but I would guess it is because they don't improve the power or readability of the language, and may in fact be significantly less readable than simply nesting the corresponding statements. In particular, with the current syntax, the local variables bound by the for statement are clearly visible immediately after the 'for' keyword. If multiple instances of the keyword were allowed in a single for statement, the names being bound could be hidden after an arbitrarily complex expression. This 'hidden local' problem is one of the concerns with the current behaviour where list comprehensions make their loop variables visible in the surrounding scope (this behaviour will be fixed in Py3k, and was deliberately avoided for generator expressions).
1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' statement. That's truly inconsistent that one may write
list2 = [i for i in list if cond(i)]
but cannot write
for i in list if cond(i):
No, instead you write: for i in list: if cond(i): The order of execution of loops and conditionals is entirely unambiguous, as it is shown explicitly by the nesting. Python has never been shy about requiring vertical and horizontal whitespace at the statement level in order to make the sequence of control flow more explicit. The rules are different for list comprehensions and generator expressions, because whitespace is only used to delimit tokens inside expressions, rather than being a token in its own right as it can be at the statement level. However, the order of execution of these expressions can be disambiguated by conversion to the nested statement version of the same code.
2. Bring "several for-s in a row" ability from generators/comprehensions to the statement (and expand it to "several for-s and if-s", of course). We can write
list2 = [f(i, j) for i in list1 for j in list2]
but cannot write
for i in list1 for j in list2: yield f(i, j)
That looks inconsistent as well.
If the apparent inconsistency genuinely bothers you, you can always write the statement version as: for x in (f(i, j) for i list1 for j in list2): yield x Note that this preserves the fact that only 'x' is bound in the surrounding scope, and this name is prominent in the 'for' statement.
Yes, for this little case we could do some kind of cross, but what if there are more loops, and/or some of them require filtering by if-s?
Then readers of the code could probably be helped out if some of the intermediate steps were named to explain what's going on. Alternately, just use a genexp as I show above.
3. Bring 'while' into the loops, both statements and iterators. Now no need to worry about "hidden-goto" 'break', especially for the cases of nested loops, while it can be easily checked right inside the looping condition?
'break' is no more a 'hidden-goto' than any other form of non-sequential control flow (like say, conditionals or loops) is a goto. The problem with having an actual goto is that it can go *anywhere*, leading to a high chance of writing spaghetti code - this is not the case when using break, as you can't go anywhere other than the statement immediately following the loop. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
Alexander Myodov <maa_public@sinn.ru> wrote:
Hello Josiah,
JC> Alexander, JC> The essence of what you have proposed has been proposed (multiple times) before, JC> and I seem to remember it was shot down.
To increase my understanding of Python-way, can you (or someone else) explain the reasons why such proposals were rejected?
JC> The below functions offer the equivalent of list comprehensions with a JC> final post-processing step.
Well, what I was suggesting is not just a cross of two lists but the syntax sugar which would make statements more consistent to the generators/comprehensions and also give some new opportunities. I think that my large proposal can be splitted to several almost-independent ones, each carrying separate features, able to be implemented independently and worth independent reconsidering:
Try using the code I offered. It allows the cross of an aribitrary number of restartable iterables, in the same order as an equivalent list comprehension or generator expression.
list(cross([1,2], [3,4], [5,6])) [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, 5), (2, 4, 6)]
There were a few hoops I had to jump through in cross in order to be able to hande single iterables as well as tuples embedded in the passed iterables, but they work as they should.
list(cross([(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)])) [((1, 1), (3, 3), (5, 5)), ((1, 1), (3, 3), (6, 6)), ((1, 1), (4, 4), (5, 5)), ((1, 1), (4, 4), (6, 6)), ((2, 2), (3, 3), (5, 5)), ((2, 2), (3, 3), (6, 6)), ((2, 2), (4, 4), (5, 5)), ((2, 2), (4, 4), (6, 6))]
1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' statement. That's truly inconsistent that one may write
list2 = [i for i in list if cond(i)]
but cannot write
for i in list if cond(i):
Note: list comprehensions and generator expressions have not been in Python since the beginning. List comprehensions were added in Python 2.0 as syntactic sugar for the common case of list construction with simple predicates. x = [] for i in y: for j in z: if f(i,j): x.append((i,j)) x = [(i,j) for i in y for j in z if f(i,j)] If you are good, you can add predicates anywhere you want inside of for loops using ifilter. def f(i): return i%2==0 def g(i): return i**2%3 < 2 for i in ifilter(f, x): for j in ifilter(g, y): ... No need to have syntax for something that is simple enough to use... If you want to filter on multiple items (updated in each loop), there is always the curry decorator and/or class offered in PEP 309 (minor modification would offer a rightcurry). For all of the reasons you offer, Nick has a great point that there is no need to complicate the for loop with generator/list comprehension semantics when you can embed either of them in the right side of the for loop. - Josiah
On Tue, Sep 20, 2005 at 10:20:44AM -0700, Josiah Carlson wrote:
Try using the code I offered. It allows the cross of an aribitrary number of restartable iterables, in the same order as an equivalent list comprehension or generator expression.
list(cross([1,2], [3,4], [5,6])) [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, 5), (2, 4, 6)]
There were a few hoops I had to jump through in cross in order to be able to hande single iterables as well as tuples embedded in the passed iterables, but they work as they should.
list(cross([(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)])) [((1, 1), (3, 3), (5, 5)), ((1, 1), (3, 3), (6, 6)), ((1, 1), (4, 4), (5, 5)), ((1, 1), (4, 4), (6, 6)), ((2, 2), (3, 3), (5, 5)), ((2, 2), (3, 3), (6, 6)), ((2, 2), (4, 4), (5, 5)), ((2, 2), (4, 4), (6, 6))]
This comes up on c.l.p every month or two. Folks offer their own solutions optimized for speed, memory, or golfed for char length. I'll throw in my same two bits as always, sprat:~# python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import probstat c = probstat.Cartesian([[1,2], [3,4], [5,6]]) list(c) [[1, 3, 5], [2, 3, 5], [1, 4, 5], [2, 4, 5], [1, 3, 6], [2, 3, 6], [1, 4, 6], [2, 4, 6]] c = probstat.Cartesian([[(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)]]) list(c) [[(1, 1), (3, 3), (5, 5)], [(2, 2), (3, 3), (5, 5)], [(1, 1), (4, 4), (5, 5)], [(2, 2), (4, 4), (5, 5)], [(1, 1), (3, 3), (6, 6)], [(2, 2), (3, 3), (6, 6)], [(1, 1), (4, 4), (6, 6)], [(2, 2), (4, 4), (6, 6)]]
The signature is slightly different (list of lists) but otherwise does what you want. Unchanged since 2002! http://probstat.sourceforge.net/ -jackdied
Alexander Myodov wrote:
for i in largelist while !found:
This doesn't cover enough use cases to be worth it, IMO. The exit condition is often buried in the middle of other statements in the loop, not right at the beginning. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
participants (5)
-
Alexander Myodov -
Greg Ewing -
Jack Diederich -
Josiah Carlson -
Nick Coghlan