Object: Creation of a ForLoopIterationObject object (name open to suggestions) using a "For/in/as" syntax. Disclaimer: I've read PEP-3136 which was refused. The present proposal includes a solution to what it tried to solve (multi-level breaking out of loops), but is a more general proposal and it offers many unrelated advantages and simplifications. It also gives another clean way of making a kind of for/except/else syntax of another active conversation. What it tries to solve: During the iteration process, we often need to create temporary variables (like counters or states) or tests that don't correspond to the logic of the algo, but are required to make it work. This proposal aims at removing these unnecessary and hard to understand parts and help write simpler, cleaner, more maintainable and more logical code. So, more PEP-8 compliant code. How does it solve it: By instanciating a new object when starting a loop, which contains some data about the iteration process and some methods to act on it. The list of available properties and methods are yet to be defined, but here are a subset of what could be really helpful. A word about compatibility and understandability before: "as" is already a keyword, so it is already reserved and easy to parse. It couldn't be taken for its other uses (context managers, import statements and exceptions) as they all start with a specific and different keyword. It would have a really similar meaning, so be easy to understand. It doesn't imply any incompatibility with previous code, as current syntax would of course still be supported, and it doesn't change anything anywhere else (no change in indentation levels for example, no deprecation). Syntax: for element in elements as elements_loop: assert type(elements_loop) is ForLoopIterationObject Properties and methods (non exhaustive list, really open to discussion and suggestions): for element in elements as elements_loop: elements_loop.length: int # len ? count ? elements_loop.counter: int # Add a counter0, as in Django? a counter1 ? elements_loop.completed: bool elements_loop.break() elements_loop.continue() elements_loop.skip(count=1) Examples of every presented element (I didn't execute the code, it's just to explain the proposal): ################################################## # forloop.counter and forloop.length for num, dog in enumerate(dogs): print(num, dog) print("That's a total of {} dogs !".format(len(dogs))) # would be equivalent to for dog in dogs as dogs_loop: print(dogs_loop.counter, dog) print("That's a total of {} dogs !".format(dogs_loop.length)) # -> cleaner, and probably faster as it won't have to call len() or enumerate() #(but I could easily be wrong on that) ################################################## # forloop.length when we broke out of the list small_and_medium_dogs_count = 0 for dog in generate_dogs_list_by_size(): if dog.size >= medium_dog_size: break small_and_medium_dogs_count += 1 print("That's a total of {} small to medium dogs !".format( small_and_medium_dogs_count)) # would be equivalent to for dog in generate_dogs_list_by_size() as dogs_loop: if dog.size >= medium_dog_size: break print("That's a total of {} small to medium dogs !".format( dogs_loop.length - 1)) # -> easier to read, less temporary variables ################################################## # forloop.break(), to break out of nested loops (or explicitly out of current #loop) - a little like pep-3136's first proposal has_dog_named_rex = False for owner in owners: for dog in dogs: if dog.name == "Rex": has_dog_named_rex = True break if has_dog_named_rex: break # would be equivalent to for owner in owners as owners_loop: for dog in dogs: # syntax without "as" is off course still supported if dog.name == "Rex": owners_loop.break() # -> way easier to read and understand, less temporary variables ################################################## # forloop.continue(), to call "continue" from any nested loops (or explicitly #in active loop) has_dog_named_rex = False for owner in owners: for dog in owner.dogs: if dog.name == "Rex": has_dog_named_rex = True break if has_dog_named_rex: continue # do something # would be equivalent to for owner in owners as owners_loop: for dog in owner.dogs: if dog.name == "Rex": owners_loop.continue() # do something # -> way easier to read and understand, less temporary variables ################################################## # forloop.completed, to know if the list was entirely consumed or "break" was #called (or exception caught) - might help with the for/except/elseproposal broken_out = False for dog in dogs: if dog.name == "Rex": broken_out = True break if broken_out: print("We didn't consume all the dogs list") # would be equivalent to for dog in dogs as dogs_loop: if dog.name == "Rex": break if not dogs_loop.completed: print("We didn't consume all the dogs list") # -> less temporary variables, every line of code is relevant, easy to #understand ################################################## # forloop.skip # In the example, we want to skip 2 elements starting from item #2 skip = 0 for num, dog in enumerate(dogs): if skip: skip -= 1 continue if num == 2: skip = 2 # would be equivalent to for dog in dogs as dogs_loop: if dogs_loop.counter == 2: dogs_loop.skip(2) # -> way easier to read and understand, less temporary variables # Notes : # - Does a call to forloop.skip() implies a forloop.continue() call or does # the code continue its execution until the end of the loop, which will # then be skipped? Implying the .continue() call seems less ambiguous to # me. Or the method should be called skip_next_iteration, or something # like that. # - Does a call to forloop.skip(2) adds 2 to forloop.length or not? # -> kwargs may be added to allow both behaviours for both questions. # We could allow the argument to be a function that accepts a single argument #and return a boolean, like dogs_loop.skip(lambda k: k % 3 == 0) # Execute the code on multiples of 3 only ################################################## Thoughts : - It would allow to pass forloop.break and forloop.continue as callback to other functions. Not sure yet if it's a good or a bad thing (readability against what it could offer). - I haven't yet used much the asynchronous functionalities, so I couldn't yet think about the implications of such a new syntax to this (and what about a lazy keyword in here?) - I suppose it's a huge work to create such a syntax. And I have no idea how complicated it could be to create methods (like break() and continue()) doing what keywords were doing until now. - I'm not sure if that would make sense in list comprehensions, despite being confusing. - Would enable to support callback events like forloop.on_break. But would there be a need for that? - Would this have a major impact on the loops execution times? - would a "while condition as condition_loop:" be of any use too? Sorry for the very long message, I hope it will get your interest. And I also hope my English was clear enough. Brice Parent
Hi Brice, On Fri, Mar 3, 2017 at 1:14 AM, Brice PARENT <contact@brice.xyz> wrote:
A word about compatibility and understandability before: "as" is already a keyword, so it is already reserved and easy to parse. It couldn't be taken for its other uses (context managers, import statements and exceptions) as they all start with a specific and different keyword. It would have a really similar meaning, so be easy to understand. It doesn't imply any incompatibility with previous code, as current syntax would of course still be supported, and it doesn't change anything anywhere else (no change in indentation levels for example, no deprecation).
That still would make it usable only on Python 3.7+ It seem to me you ca already do that now with helper object/function: In [1]: class Loop: ...: ...: def __init__(self, iterable): ...: self._it = iter(iterable) ...: self._break = False ...: ...: def __next__(self): ...: if self._break: ...: raise StopIteration ...: return (self, next(self._it)) ...: ...: def __iter__(self): ...: return self ...: ...: def brk(self): ...: self._break = True ...: ...: In [2]: for loop,i in Loop(range(10)): ...: if i==5: ...: loop.brk() ...: print(i) 1 2 3 4 5<break>
Syntax: for element in elements as elements_loop: assert type(elements_loop) is ForLoopIterationObject
Properties and methods (non exhaustive list, really open to discussion and suggestions): for element in elements as elements_loop: elements_loop.length: int # len ? count ? elements_loop.counter: int # Add a counter0, as in Django? a counter1 ? elements_loop.completed: bool elements_loop.break() elements_loop.continue() elements_loop.skip(count=1)
I did not implement these, but they are as feasible. (break is keyword though, so you need to rename it)
Examples of every presented element (I didn't execute the code, it's just to explain the proposal):
################################################## # forloop.counter and forloop.length
for num, dog in enumerate(dogs): print(num, dog)
print("That's a total of {} dogs !".format(len(dogs)))
# would be equivalent to
for dog in dogs as dogs_loop: print(dogs_loop.counter, dog)
I find it less readable than enumerate.
print("That's a total of {} dogs !".format(dogs_loop.length))
# -> cleaner, and probably faster as it won't have to call len() or enumerate() #(but I could easily be wrong on that)
Though I can see how accumulating the length while iterating may make sens.
################################################## # forloop.length when we broke out of the list
small_and_medium_dogs_count = 0 for dog in generate_dogs_list_by_size(): if dog.size >= medium_dog_size: break
small_and_medium_dogs_count += 1
print("That's a total of {} small to medium dogs !".format( small_and_medium_dogs_count))
# would be equivalent to
for dog in generate_dogs_list_by_size() as dogs_loop: if dog.size >= medium_dog_size: break
print("That's a total of {} small to medium dogs !".format( dogs_loop.length - 1))
# -> easier to read, less temporary variables
################################################## # forloop.break(), to break out of nested loops (or explicitly out of current #loop) - a little like pep-3136's first proposal
has_dog_named_rex = False for owner in owners: for dog in dogs: if dog.name == "Rex": has_dog_named_rex = True break
if has_dog_named_rex: break
# would be equivalent to
for owner in owners as owners_loop: for dog in dogs: # syntax without "as" is off course still supported if dog.name == "Rex": owners_loop.break()
See my above proposal, you would still need to break the inner loop here as well. So i'm guessing you miss a `break` afrer owner_loop.break() ?
# -> way easier to read and understand, less temporary variables
################################################## # forloop.continue(), to call "continue" from any nested loops (or explicitly #in active loop)
has_dog_named_rex = False for owner in owners: for dog in owner.dogs: if dog.name == "Rex": has_dog_named_rex = True break
if has_dog_named_rex: continue
# do something
# would be equivalent to
for owner in owners as owners_loop: for dog in owner.dogs: if dog.name == "Rex": owners_loop.continue()
you are missing a break as well here you need to break out of owner.dogs, but fair. I think though that having to `.continue()` the outer loop before breaking the inner (you have to right ?) can make things complicated.
# do something
# -> way easier to read and understand, less temporary variables
################################################## # forloop.completed, to know if the list was entirely consumed or "break" was #called (or exception caught) - might help with the for/except/elseproposal
broken_out = False for dog in dogs: if dog.name == "Rex": broken_out = True break
if broken_out: print("We didn't consume all the dogs list")
# would be equivalent to
for dog in dogs as dogs_loop: if dog.name == "Rex": break
if not dogs_loop.completed: print("We didn't consume all the dogs list")
# -> less temporary variables, every line of code is relevant, easy to #understand
################################################## # forloop.skip # In the example, we want to skip 2 elements starting from item #2
skip = 0 for num, dog in enumerate(dogs): if skip: skip -= 1 continue
if num == 2: skip = 2
# would be equivalent to
for dog in dogs as dogs_loop: if dogs_loop.counter == 2: dogs_loop.skip(2)
# -> way easier to read and understand, less temporary variables
# Notes : # - Does a call to forloop.skip() implies a forloop.continue() call or does # the code continue its execution until the end of the loop, which will # then be skipped? Implying the .continue() call seems less ambiguous to # me. Or the method should be called skip_next_iteration, or something # like that. # - Does a call to forloop.skip(2) adds 2 to forloop.length or not? # -> kwargs may be added to allow both behaviours for both questions.
# We could allow the argument to be a function that accepts a single argument #and return a boolean, like dogs_loop.skip(lambda k: k % 3 == 0) # Execute the code on multiples of 3 only
##################################################
Thoughts : - It would allow to pass forloop.break and forloop.continue as callback to other functions. Not sure yet if it's a good or a bad thing (readability against what it could offer). - I haven't yet used much the asynchronous functionalities, so I couldn't yet think about the implications of such a new syntax to this (and what about a lazy keyword in here?) - I suppose it's a huge work to create such a syntax. And I have no idea how complicated it could be to create methods (like break() and continue()) doing what keywords were doing until now. - I'm not sure if that would make sense in list comprehensions, despite being confusing. - Would enable to support callback events like forloop.on_break. But would there be a need for that? - Would this have a major impact on the loops execution times? - would a "while condition as condition_loop:" be of any use too?
I think most of what you ask can be achieve by a Generator with some custom function, and does not require new syntax. It's good to be able to do that as an external library then you can experiment evolve it to get real world usage. And you can use it now ! Cheers, -- M
On 03/03/2017 08:21 AM, Matthias Bussonnier wrote:
################################################## # forloop.break(), to break out of nested loops (or explicitly out of current #loop) - a little like pep-3136's first proposal
has_dog_named_rex = False for owner in owners: for dog in dogs: if dog.name == "Rex": has_dog_named_rex = True break
if has_dog_named_rex: break
# would be equivalent to
for owner in owners as owners_loop: for dog in dogs: # syntax without "as" is off course still supported if dog.name == "Rex": owners_loop.break()
See my above proposal, you would still need to break the inner loop here as well. So i'm guessing you miss a `break` afrer owner_loop.break() ?
No, he's not -- part of implementing this change includes not needing to specify the inner breaks. -- ~Ethan~
Thanks Matthias for taking the time to give your opinion about it. Just to set the focus where I may have failed to point it: the main purpose of this proposal is the creation of the object itself, an object representing the loop. What we can do with it is still a sub-level of this proposal, as the presence of this object is what allows all these simplifications, and offers a lot of possibilities. Le 03/03/17 à 17:21, Matthias Bussonnier a écrit :
A word about compatibility and understandability before: "as" is already a keyword, so it is already reserved and easy to parse. It couldn't be taken for its other uses (context managers, import statements and exceptions) as they all start with a specific and different keyword. It would have a really similar meaning, so be easy to understand. It doesn't imply any incompatibility with previous code, as current syntax would of course still be supported, and it doesn't change anything anywhere else (no change in indentation levels for example, no deprecation). That still would make it usable only on Python 3.7+ It seem to me you ca already do that now with helper object/function:
Yes, what I meant is that it wouldn't break any code from previous versions. But as with any new keyword or syntax evolution, newer code wouldn't get well in older versions.
In [1]: class Loop: ...: ...: def __init__(self, iterable): ...: self._it = iter(iterable) ...: self._break = False ...: ...: def __next__(self): ...: if self._break: ...: raise StopIteration ...: return (self, next(self._it)) ...: ...: def __iter__(self): ...: return self ...: ...: def brk(self): ...: self._break = True ...: ...:
In [2]: for loop,i in Loop(range(10)): ...: if i==5: ...: loop.brk() ...: print(i) 1 2 3 4 5<break> A 2 level breaking would have to be used this way :
for outerloop, i in Loop(range(4)): for innerloop, j in Loop(range(3)): if i==2 and j==1: outerloop.brk() break # this print(i, j) if outerloop.broken: break # or continue. For the code following this line not to be executed break() and continue() methods (which I named this way to reflect the behaviour of the statements, but I'm not sure whether it could or should be kept this way) are only an improvement on the keywords in nested loops or complex situations where explicitly exiting a certain loop helps readability by removing a bunch of conditions and assignments. It's also allowing to pass this function as a callback to something else, but I'm not sure it would be that useful.
Syntax: for element in elements as elements_loop: assert type(elements_loop) is ForLoopIterationObject
Properties and methods (non exhaustive list, really open to discussion and suggestions): for element in elements as elements_loop: elements_loop.length: int # len ? count ? elements_loop.counter: int # Add a counter0, as in Django? a counter1 ? elements_loop.completed: bool elements_loop.break() elements_loop.continue() elements_loop.skip(count=1) I did not implement these, but they are as feasible. (break is keyword though, so you need to rename it)
Length, counter and completed (and .skip() in some ways) are just helping the readability while allowing a more concise code, as they don't force to create temporary variables. But they belong here as a part of the whole thing. If the object had to be integrated, for .break(), .continue(), .skip() and any other useful method, those little improvements would be nice to have and help to keep your code look more like the design in your head. .break() and .continue() help as they modify the flow. They roughly do this : LOOP1 LOOP2 if something: LOOP1.break() # similar to calling break repeatedly once for every loop until LOOP1 included, so going straight to "something_after_every_loop" something_else yet_another_thing something_after_every_loop The same goes for .continue(), calling break in every inner loop, and continue in the one the method is called from.
Examples of every presented element (I didn't execute the code, it's just to explain the proposal):
################################################## # forloop.counter and forloop.length
for num, dog in enumerate(dogs): print(num, dog)
print("That's a total of {} dogs !".format(len(dogs)))
# would be equivalent to
for dog in dogs as dogs_loop: print(dogs_loop.counter, dog) I find it less readable than enumerate.
Well, I find both readable, but the second only requires to focus on the loop itself, not on how enumerate and unpacking work. The second syntax however allows the exact same syntax when you work with dicts, lists or tuples. Here, dogs may be of any type of iterable, while in first version, you'd need to implement a counter to have a unified loop syntax (I'm not saying we should always be able to replace those types, just that in that case it makes sense to me). Brice
Hi Brice, On Fri, Mar 3, 2017 at 10:00 AM, Brice PARENT <contact@brice.xyz> wrote:
Thanks Matthias for taking the time to give your opinion about it.
Just to set the focus where I may have failed to point it: the main purpose of this proposal is the creation of the object itself, an object representing the loop. What we can do with it is still a sub-level of this proposal, as the presence of this object is what allows all these simplifications, and offers a lot of possibilities. A 2 level breaking would have to be used this way :
for outerloop, i in Loop(range(4)): for innerloop, j in Loop(range(3)): if i==2 and j==1: outerloop.brk() break # this
Thanks, I think it does make sens, I'm going to guess, outerloop.brk(inners=True) might also be helpful if you have more inners loops. I think that implicitely breaking inner ones might not always be the right thing to do so having a way to not break inner ones does make sens. The other possibility would be to allow the Loop object to catch raised exceptions and potentially continue running loops. Then you could "just" use raise to break multiple loops, but that might be a weird Loop/ContextManager hybrid. -- M
On Sat, Mar 4, 2017 at 5:50 AM, Matthias Bussonnier <bussonniermatthias@gmail.com> wrote:
Thanks, I think it does make sens, I'm going to guess, outerloop.brk(inners=True) might also be helpful if you have more inners loops. I think that implicitely breaking inner ones might not always be the right thing to do so having a way to not break inner ones does make sens.
*scratches head* How do you break an outer loop without breaking the inner loop? What happens? ChrisA
*scratches head* How do you break an outer loop without breaking the inner loop? What happens?
Finish running the inner, then breaking the outer. Instead of breaking inner and outer. for i in outer: bk = False for j in inner: if cond: bk = True if bk: break vs for i in outer: bk = False for j in inner: if cond: bk = True break # this. if bk: break Sorry if I'm mis expressing myself. -- M On Fri, Mar 3, 2017 at 10:52 AM, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, Mar 4, 2017 at 5:50 AM, Matthias Bussonnier <bussonniermatthias@gmail.com> wrote:
Thanks, I think it does make sens, I'm going to guess, outerloop.brk(inners=True) might also be helpful if you have more inners loops. I think that implicitely breaking inner ones might not always be the right thing to do so having a way to not break inner ones does make sens.
*scratches head* How do you break an outer loop without breaking the inner loop? What happens?
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, Mar 4, 2017 at 5:56 AM, Matthias Bussonnier <bussonniermatthias@gmail.com> wrote:
*scratches head* How do you break an outer loop without breaking the inner loop? What happens?
Finish running the inner, then breaking the outer. Instead of breaking inner and outer.
for i in outer: bk = False for j in inner: if cond: bk = True if bk: break
vs
for i in outer: bk = False for j in inner: if cond: bk = True break # this. if bk: break
Sorry if I'm mis expressing myself.
Oh, I see what you mean. So "breaking the outer loop" really means "flag the outer loop such that, on next iteration, it will terminate". The trouble is that that's not exactly what "break" means. Consider: for i in outer: print("Top of outer") for j in inner: print("Top of inner") if cond1: break if cond2: break_outer(inner=False) if cond3: break_outer(inner=True) print("Bottom of inner") print("Bottom of outer") cond1 would print "bottom of outer" and keep loping. cond2 and cond3 should presumably _not_ print "bottom of outer". Should they print "bottom of inner", though? Seems like an odd use, though. If I want a multi-step break, I want to break every step, not just the last one. ChrisA
On 2017-03-03 01:52 PM, Chris Angelico wrote:
On Sat, Mar 4, 2017 at 5:50 AM, Matthias Bussonnier <bussonniermatthias@gmail.com> wrote:
Thanks, I think it does make sens, I'm going to guess, outerloop.brk(inners=True) might also be helpful if you have more inners loops. I think that implicitely breaking inner ones might not always be the right thing to do so having a way to not break inner ones does make sens.
*scratches head* How do you break an outer loop without breaking the inner loop? What happens?
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ I believe what Matthias is hoping for is an equivalent of Java's named break feature. Breaking out of an outer loop implicitly breaks out of all inner loops Alex
On 03/03/17 19:02, Alexandre Brault wrote:
I believe what Matthias is hoping for is an equivalent of Java's named break feature. Breaking out of an outer loop implicitly breaks out of all inner loops
Yes, and although I think making this a runtime object is an interesting thought (in terms of perhaps allowing other funky stuff to be implemented by a custom object, in line with Python's general dynamic ethos), I think that it should perhaps be considered a lexer/parser level thing only. * Creating a real object at runtime for each loop which needs to be the target of a non-inner break or continue is quite a large overhead. How would this affect Python variants other than CPython? * For anything "funky" (my words, not yours ;)), there needs to be a way of creating a custom loop object - what would the syntax for that be? A callable needs to be invoked as well as the name bound (the current suggestion just binds a name to some magical object that appears from somewhere). * If nothing "funky" needs to be done then why not just make the whole thing syntax-only and have no real object, by making the 'as' name a parser-only token which is only valid as the optional subject of a break or continue statement: for foo in bar as LABEL: . # (a) . for spam in ham: . . if eggs(spam): continue LABEL . . if not frob(spam): break LABEL # (b) (a) is the code generator's implied 'continue' target for the LABEL loop. (b) is the code generator's implied 'break' target for the LABEL loop. I'm not saying that's a great solution either. It's probably not an option as there is now something that looks like a bound name but is not actually available at runtime - the following would not work: for foo in bar as LABEL: print(dir(LABEL)) (and presumably that is part of the reason why the proposal is the way it is). I'm generally +0 on the concept (it might be nice, but I'm not sure either the original proposal or what I mention above are particularly problem-free ;)). E.
* Creating a real object at runtime for each loop which needs to be the target of a non-inner break or continue is quite a large overhead. How would this affect Python variants other than CPython?
I don't know much about the specifics on how it works behind the scenes, and even less about other implementations of the language, so I just give my opinions on the usage and the benefits of such a syntax. However, I'm not sure the object should be constructed and fed for every loop usage. It should probably only be instanciated if explicitly asked by the coder (by the use of "as loop_name"). I don't know what happens to the caught exceptions when they are not set to a variable (using except ExceptionType as e), but I think we would be in a similar case, only, if the object is instanciated, its states would evolve each time the loop starts another iteration.
* For anything "funky" (my words, not yours ;)), there needs to be a way of creating a custom loop object - what would the syntax for that be? A callable needs to be invoked as well as the name bound (the current suggestion just binds a name to some magical object that appears from somewhere).
I don't really understand what this means, as I'm not aware of how those things work in the background. But it would create an object and feed it to the variable, yes. I guess it would be magical in the sense it's not the habitual way of constructing an object. But it's what we're already used to with "as". When we use a context manager, like "with MyPersonalStream() as my_stream:", my_stream is not an object of type "MyPersonalStream" that has been built using the constructor, but the return of __enter__() (at least, it's what I understood), and the MyPersonalStream instance is somewhere else waiting for its closing fate. And when catching an exception, we feed a new variable with an object created somewhere-else-in-the-code, not with a classic instanciation. So this behaviour, obtaining an object created somewhere else and not explicitly, seems logical with the other uses of "as", IMHO.
* If nothing "funky" needs to be done then why not just make the whole thing syntax-only and have no real object, by making the 'as' name a parser-only token which is only valid as the optional subject of a break or continue statement:
for foo in bar as LABEL: . # (a) . for spam in ham: . . if eggs(spam): continue LABEL . . if not frob(spam): break LABEL # (b)
(a) is the code generator's implied 'continue' target for the LABEL loop. (b) is the code generator's implied 'break' target for the LABEL loop.
I'm not saying that's a great solution either. It's probably not an option as there is now something that looks like a bound name but is not actually available at runtime - the following would not work:
for foo in bar as LABEL: print(dir(LABEL))
(and presumably that is part of the reason why the proposal is the way it is).
This solution, besides having been explicitly rejected by Guido himself, brings two functionalities that are part of the proposal, but are not its main purpose, which is having the object itself. Allowing to break and continue from it are just things that it could bring to us, but there are countless things it could also bring (not all of them being good ideas, of course), like the .skip() and the properties I mentioned, but we could discuss about some methods like forloop.reset(), forloop.is_first_iteration() which is just of shortcut to (forloop.count == 0), forloop.is_last_iteration() (which would probably only be available on fixed-length iterables), and probably many other things to play with the sequencing of the loop.
I'm generally +0 on the concept (it might be nice, but I'm not sure either the original proposal or what I mention above are particularly problem-free ;)).
I have no clue on the implementation side, and I'm pretty sure such a big change couldn't ever be problem-free, I'm just convinced it could bring a lot of control and readability on the user side. Brice
Hi Brice, On 04/03/17 08:45, Brice PARENT wrote:
* Creating a real object at runtime for each loop which needs to be the target of a non-inner break or continue
However, I'm not sure the object should be constructed and fed for every loop usage. It should probably only be instanciated if explicitly asked by the coder (by the use of "as loop_name").
That's what I meant by "needs to be the target of a non-inner break or continue" (OK, you are proposing something more than just a referenced break/continue target, but we are talking about the same thing). Only loops which use the syntax get a loop manager object.
* For anything "funky" (my words, not yours ;)), there needs to be a way of creating a custom loop object - what would the syntax for that be? A callable needs to be invoked as well as the name bound (the current suggestion just binds a name to some magical object that appears from somewhere).
I don't really understand what this means, as I'm not aware of how those things work in the background.
What I mean is, in the syntax "for spam in ham as eggs:" the name "eggs" is bound to your loop manager object. Where is the constructor call for this object? what class is it? That's what I meant by "magical". If you are proposing the ability to create user-defined loop managers then there must be somewhere where your custom class's constructor is called. Otherwise how does Python know what type of object to create? Something like (this is not a proposal, just something plucked out of the air to hopefully illustrate what I mean): for spam in ham with MyLoop() as eggs: eggs.continue()
I guess it would be magical in the sense it's not the habitual way of constructing an object. But it's what we're already used to with "as". When we use a context manager, like "with MyPersonalStream() as my_stream:", my_stream is not an object of type "MyPersonalStream" that has been built using the constructor, but the return of __enter__()
By you have to spell the constructor (MyPersonalStream()) to see what type of object is being created (whether or not the eventual name bound in your context is to the result of a method call on that object, the constructor of your custom context manager is explicitly called. If you are saying that the syntax always implicitly creates an instance of a builtin class which can not be subclassed by a custom class then that's a bit different.
This solution, besides having been explicitly rejected by Guido himself,
I didn't realise that. Dead in the water then probably, which is fine, I wasn't pushing it.
brings two functionalities that are part of the proposal, but are not its main purpose, which is having the object itself. Allowing to break and continue from it are just things that it could bring to us, but there are countless things it could also bring (not all of them being good ideas, of course), like the .skip() and the properties I mentioned,
I understand that, but I concentrated on those because they were easily converted into syntax (and would probably be the only things I'd find useful - all the other stuff is mostly doable using a custom iterator, I think). I would agree that considering syntax for all of the extra things you mention would be a bad idea - which your loop manager object idea gets around.
but we could discuss about some methods like forloop.reset(), forloop.is_first_iteration() which is just of shortcut to (forloop.count == 0), forloop.is_last_iteration()
Also, FWIW, if I knew that in addition to the overhead of creating a loop manager object I was also incurring the overhead of a loop counter being maintained (usually, one is not required - if it is, use enumerate()) I would probably not use this construct and instead find ways of restructuring my code to avoid it using regular for loops. I'm not beating up on you - like I said, I think the idea is interesting. E.
Hi Erik,
I don't really understand what this means, as I'm not aware of how those things work in the background.
What I mean is, in the syntax "for spam in ham as eggs:" the name "eggs" is bound to your loop manager object. Where is the constructor call for this object? what class is it? That's what I meant by "magical". It's what I thought, thanks for the clarification.
If you are proposing the ability to create user-defined loop managers then there must be somewhere where your custom class's constructor is called. Otherwise how does Python know what type of object to create? I wasn't thinking about a custom object, although this syntax wouldn't ease the process of customizing the looping behaviour itself.
Something like (this is not a proposal, just something plucked out of the air to hopefully illustrate what I mean):
for spam in ham with MyLoop() as eggs: eggs.continue() I get it. If there are use cases where we'd like to use a custom loop type, it surely gets a bit more complicated, but we could find some alternative syntaxes like yours or one of those :
with LoopIterator.use(MyLoop): for spam in ham as eggs: eggs.continue() or for spam in ham as (MyLoop, eggs): # When it's a 2-tuple, first element is the class, second the instance. But I'm not sure about a tuple where one value is to be read while the other is assigned to... eggs.continue() or eggs = MyLoop() for spam in ham using eggs: # I particularly dislike this one... eggs.continue() But anyway, I'm not sure we're yet at this point of the thinking!
brings two functionalities that are part of the proposal, but are not its main purpose, which is having the object itself. Allowing to break and continue from it are just things that it could bring to us, but there are countless things it could also bring (not all of them being good ideas, of course), like the .skip() and the properties I mentioned,
I understand that, but I concentrated on those because they were easily converted into syntax (and would probably be the only things I'd find useful - all the other stuff is mostly doable using a custom iterator, I think).
Probably, i'll probably try to implement those as I've needed one or the other some times already.
I would agree that considering syntax for all of the extra things you mention would be a bad idea - which your loop manager object idea gets around.
but we could discuss about some methods like forloop.reset(), forloop.is_first_iteration() which is just of shortcut to (forloop.count == 0), forloop.is_last_iteration()
Also, FWIW, if I knew that in addition to the overhead of creating a loop manager object I was also incurring the overhead of a loop counter being maintained (usually, one is not required - if it is, use enumerate()) I would probably not use this construct and instead find ways of restructuring my code to avoid it using regular for loops.
I would certainly not enforce the users to use this syntax, nor the system to maintain the object if it is not instanciated explicitly by the user. Also, I don't know how it works behind the doors, so I have no idea whether having such an object and a counter (which is probably the only things to maintain, as everything else just depend on the counter) would change a lot the cost of the loop in term of speed and memory (and anything else).
I'm not beating up on you - like I said, I think the idea is interesting.
Don't worry, I didn't think you were. I like this idea as I think it could be of a good help to many people, but it doesn't get much traction here, so I probably overestimate it! Brice
Thanks for the idea and prior research. I'm not convinced that this warrants new syntax. Most of what you propose (skipping, counting, exposing a length if available, tracking if completed) could be solved already by creating your own wrapper around an iterable: elements_loop = ForLoopIterationObject(elements) for element in elements_loop: ... Some of your proposal simply can't be done perfectly (e.g. length) since iterables can have infinite length. There is `__length_hint__` IIRC which might (or might not) give a rough estimate. You also couldn't do `elements_loop.continue()` or `elements_loop.break()` in a python-based wrapper without special interpreter magic (at least now as far as I can imagine...). I realize that is the point of this proposal as it would allow breaking/continuing nested loops, however, that proposal was already rejected because Guido didn't think that the use-cases were compelling enough to add new syntax/complexity to the python language and most of the time you can address that case by adding a function here or there. In this proposal, you're adding significantly _more_ complexity than was proposed in PEP-3136. Also, you're creating an object for every loop -- and that object needs to do stuff when iterating (e.g. increment a counter). It's very unlikely that this could be anything but a performance drain compared to existing solutions (only use `enumerate` when you need it). I suppose that the interpreter could be smart enough to only create a ForLoopIterationObject when it actually needs it (e.g. when there is an `as` statement), but that shifts a lot more complexity onto the language implementors -- again for perhaps little benefit. On Fri, Mar 3, 2017 at 1:14 AM, Brice PARENT <contact@brice.xyz> wrote:
Object: Creation of a ForLoopIterationObject object (name open to suggestions) using a "For/in/as" syntax.
Disclaimer: I've read PEP-3136 which was refused. The present proposal includes a solution to what it tried to solve (multi-level breaking out of loops), but is a more general proposal and it offers many unrelated advantages and simplifications. It also gives another clean way of making a kind of for/except/else syntax of another active conversation.
What it tries to solve: During the iteration process, we often need to create temporary variables (like counters or states) or tests that don't correspond to the logic of the algo, but are required to make it work. This proposal aims at removing these unnecessary and hard to understand parts and help write simpler, cleaner, more maintainable and more logical code. So, more PEP-8 compliant code.
How does it solve it: By instanciating a new object when starting a loop, which contains some data about the iteration process and some methods to act on it. The list of available properties and methods are yet to be defined, but here are a subset of what could be really helpful.
A word about compatibility and understandability before: "as" is already a keyword, so it is already reserved and easy to parse. It couldn't be taken for its other uses (context managers, import statements and exceptions) as they all start with a specific and different keyword. It would have a really similar meaning, so be easy to understand. It doesn't imply any incompatibility with previous code, as current syntax would of course still be supported, and it doesn't change anything anywhere else (no change in indentation levels for example, no deprecation).
Syntax: for element in elements as elements_loop: assert type(elements_loop) is ForLoopIterationObject
Properties and methods (non exhaustive list, really open to discussion and suggestions): for element in elements as elements_loop: elements_loop.length: int # len ? count ? elements_loop.counter: int # Add a counter0, as in Django? a counter1 ? elements_loop.completed: bool elements_loop.break() elements_loop.continue() elements_loop.skip(count=1)
Examples of every presented element (I didn't execute the code, it's just to explain the proposal):
################################################## # forloop.counter and forloop.length
for num, dog in enumerate(dogs): print(num, dog)
print("That's a total of {} dogs !".format(len(dogs)))
# would be equivalent to
for dog in dogs as dogs_loop: print(dogs_loop.counter, dog)
print("That's a total of {} dogs !".format(dogs_loop.length))
# -> cleaner, and probably faster as it won't have to call len() or enumerate() #(but I could easily be wrong on that)
################################################## # forloop.length when we broke out of the list
small_and_medium_dogs_count = 0 for dog in generate_dogs_list_by_size(): if dog.size >= medium_dog_size: break
small_and_medium_dogs_count += 1
print("That's a total of {} small to medium dogs !".format( small_and_medium_dogs_count))
# would be equivalent to
for dog in generate_dogs_list_by_size() as dogs_loop: if dog.size >= medium_dog_size: break
print("That's a total of {} small to medium dogs !".format( dogs_loop.length - 1))
# -> easier to read, less temporary variables
################################################## # forloop.break(), to break out of nested loops (or explicitly out of current #loop) - a little like pep-3136's first proposal
has_dog_named_rex = False for owner in owners: for dog in dogs: if dog.name == "Rex": has_dog_named_rex = True break
if has_dog_named_rex: break
# would be equivalent to
for owner in owners as owners_loop: for dog in dogs: # syntax without "as" is off course still supported if dog.name == "Rex": owners_loop.break()
# -> way easier to read and understand, less temporary variables
################################################## # forloop.continue(), to call "continue" from any nested loops (or explicitly #in active loop)
has_dog_named_rex = False for owner in owners: for dog in owner.dogs: if dog.name == "Rex": has_dog_named_rex = True break
if has_dog_named_rex: continue
# do something
# would be equivalent to
for owner in owners as owners_loop: for dog in owner.dogs: if dog.name == "Rex": owners_loop.continue()
# do something
# -> way easier to read and understand, less temporary variables
################################################## # forloop.completed, to know if the list was entirely consumed or "break" was #called (or exception caught) - might help with the for/except/elseproposal
broken_out = False for dog in dogs: if dog.name == "Rex": broken_out = True break
if broken_out: print("We didn't consume all the dogs list")
# would be equivalent to
for dog in dogs as dogs_loop: if dog.name == "Rex": break
if not dogs_loop.completed: print("We didn't consume all the dogs list")
# -> less temporary variables, every line of code is relevant, easy to #understand
################################################## # forloop.skip # In the example, we want to skip 2 elements starting from item #2
skip = 0 for num, dog in enumerate(dogs): if skip: skip -= 1 continue
if num == 2: skip = 2
# would be equivalent to
for dog in dogs as dogs_loop: if dogs_loop.counter == 2: dogs_loop.skip(2)
# -> way easier to read and understand, less temporary variables
# Notes : # - Does a call to forloop.skip() implies a forloop.continue() call or does # the code continue its execution until the end of the loop, which will # then be skipped? Implying the .continue() call seems less ambiguous to # me. Or the method should be called skip_next_iteration, or something # like that. # - Does a call to forloop.skip(2) adds 2 to forloop.length or not? # -> kwargs may be added to allow both behaviours for both questions.
# We could allow the argument to be a function that accepts a single argument #and return a boolean, like dogs_loop.skip(lambda k: k % 3 == 0) # Execute the code on multiples of 3 only
##################################################
Thoughts : - It would allow to pass forloop.break and forloop.continue as callback to other functions. Not sure yet if it's a good or a bad thing (readability against what it could offer). - I haven't yet used much the asynchronous functionalities, so I couldn't yet think about the implications of such a new syntax to this (and what about a lazy keyword in here?) - I suppose it's a huge work to create such a syntax. And I have no idea how complicated it could be to create methods (like break() and continue()) doing what keywords were doing until now. - I'm not sure if that would make sense in list comprehensions, despite being confusing. - Would enable to support callback events like forloop.on_break. But would there be a need for that? - Would this have a major impact on the loops execution times? - would a "while condition as condition_loop:" be of any use too?
Sorry for the very long message, I hope it will get your interest. And I also hope my English was clear enough.
Brice Parent _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt@getpattern.com // P: 603.892.7736 We’re looking for beta testers. Go here <https://www.getpattern.com/meetpattern> to sign up!
participants (7)
-
Alexandre Brault -
Brice PARENT -
Chris Angelico -
Erik -
Ethan Furman -
Matt Gilson -
Matthias Bussonnier