PEP 340 - possible new name for block-statement

How about, instead of trying to emphasize how different a block-statement is from a for-loop, we emphasize their similarity? A regular old loop over a sequence or iterable is written as: for VAR in EXPR: BLOCK A variation on this with somewhat different semantics swaps the keywords: in EXPR for VAR: BLOCK If you don't need the variable, you can leave the "for VAR" part out: in EXPR: BLOCK Too cute? :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, Apr 28, 2005 at 03:55:03PM -0700, Guido van Rossum wrote:
Looks weird to my eyes. On a related note, I was thinking about the extra cleanup 'block' provides. If the 'file' object would provide a suitable iterator, you could write: block open(filename) as line: ... and have the file closed at the end of the block. It does not read so well though. In a way, it seems to make more sense if 'block' called iter() on the expression and 'for' did not. block would guarantee to cleanup iterators that it created. 'for' does not but implictly creates them. Neil

Neil Schemenauer wrote:
Probably makes perfect sense if you're Dutch, though. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

Guido van Rossum wrote:
I don't think it reads well. I would prefer something that would be understandable for a newbie's eyes, even if it fits more with common usage than with the real semantics behind it. For example a Boost-like keyword like: scoped EXPR as VAR: BLOCK scoped EXPR: BLOCK We may argue that it doesn't mean a lot, but at least if a newbie sees the following code, he would easily guess what it does: scoped synchronized(mutex): scoped opening(filename) as file: ... When compared with: in synchronized(mutex): in opening(filename) for file: ... As a C++ programmer, I still dream I could also do: scoped synchronized(mutex) scoped opening(filename) as file ... which would define a block until the end of the current block... Regards, Nicolas

[Nicolas Fleury]
Definitely not. In too many languages, a "scope" is a new namespace, and that's exactly what a block (by whichever name) is *not*. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <gvanrossum@gmail.com> wrote:
scopeless, unscoped, Scope(tm) (we would be required to use the unicode trademark symbol, of course)... It's way too long, and is too close to a pre-existing keyword, but I think 'finalized' is descriptive. But... finalize EXPR as VAR: BLOCK That reads nice... Maybe even 'cleanup', or 'finalize_after_iteration_without_iter_call' (abbreviated to 'faiwic', of course). <1.0 wink> All right, it's late enough. Enough 'ideas' from me tonight. - Josiah

Guido van Rossum wrote:
Humm... what about "context"? context EXPR as VAR: BLOCK I may answer the question myself, but is an alternative syntax without an indentation conceivable? (yes, even since the implicit block could be run multiple times). Because in that case, a keyword like "block" would not look right. It seems to me that in most RAII cases, the block could end at the end of the current block and that's fine, and over-indentation can be avoided. However, I realize that the indentation makes more sense in the context of Python and removes some magic that would be natural for a C++ programmer used to presence of stack... Ok, I answer my question, but "context" still sounds nicer to me than "block";) Regards, Nicolas

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: | How about, instead of trying to emphasize how different a | block-statement is from a for-loop, we emphasize their similarity? | | A regular old loop over a sequence or iterable is written as: | | for VAR in EXPR: | BLOCK | | A variation on this with somewhat different semantics swaps the keywords: | | in EXPR for VAR: | BLOCK | | If you don't need the variable, you can leave the "for VAR" part out: | | in EXPR: | BLOCK | | Too cute? :-) Far too close to the "for" loop, IMHO. I read that, I'd have to remind myself every time, "now, which one is it that can receive values passed back in: for ... in, or in ... for?" I'm definitely -1 on that one: too confusing. Another possibility just occurred to me. How about "using"? ~ using EXPR as VAR: ~ BLOCK Reads similarly to "with", but leaves the "with" keyword open for possible use later. Since it seems traditional for one to introduce oneself upon first posting to python-dev, my name is Robin Munn. Yes, my name is just one letter different from Robin Dunn's. It's not like I *intended* to cause confusion... :-) Anyway, I was introduced to Python a few years ago, around version 2.1 or so, and fell in love with the fact that I could read my own code six months later and understand it. I try to help out where I can, but I don't know the guts of the interpreter, so on python-dev I mostly lurk. Robin Munn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCcbf16OLMk9ZJcBQRAuYpAJ4n24AgsX3SrW0g7jlWJM+HfzHXMwCfTbTq eJ2mLzg1uLZv09KDUemM+WU= =SXux -----END PGP SIGNATURE-----

Whoa! Read the PEP closely. Passing a value back to the iterator (using "continue EXPR") is supported both in the for-loop and in the block-statement; it's new syntax so there's no backwards compatibility issue. The real difference is that when a for-loop is exited through a break, return or exception, the iterator is left untouched; but when the same happens in a block-statement, the iterator's __exit__ or __error__ method is called (I haven't decided what to call it).
Another possibility just occurred to me. How about "using"?
Blah. I'm beginning to like block just fine. With using, the choice of word for the generator name becomes iffy IMO; and it almost sounds like it's a simple renaming: "using X as Y" could mean "Y = X". -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Robin Munn wrote: [snip]
Examples from PEP 340: ========== def synchronized(lock): ... using synchronized(myLock): ... ===== (+0) def opening(filename, mode="r"): ... using opening("/etc/passwd") as f: ... ===== (+1) def auto_retry(n=3, exc=Exception): ... using auto_retry(3, IOError): ... ===== (+1) def synchronized_opening(lock, filename, mode="r"): ... using synchronized_opening("/etc/passwd", myLock) as f: ... ===== (+1) A.R.

Guido van Rossum wrote:
How about, instead of trying to emphasize how different a block-statement is from a for-loop, we emphasize their similarity?
If you want to emphasise the similarity, the following syntax and explanation is something that occurred to me during lunch today: Python offers two variants on the basic iterative loop. "for NAME from EXPR:" enforces finalisation of the iterator. At loop completion, a well-behaved iterator is always completely exhausted. This form supports block management operations, that ensure timely release of resources such as locks or file handles. If the values being iterated over are not required, then the statement may be simplified to "for EXPR:". "for NAME in EXPR:" skips the finalisation step. At loop completion, a well-behaved iterator may still contain additional values. This form allows an iterator to be consumed in stages. Regardless of whether you like the above or not, I think the PEP's proposed use of 'as' is incorrect - it looks like the variable should be referring to the expression being iterated over, rather than the values returned from the iterator. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan a écrit :
Well, I would go a step further and keep only the for-loop syntax, mainly because I don't understand why there is two syntax for things that's so close we can merge them ! You can simply states that the for-loop call the "__error__" method of the object if available without invalidating any other property of the new for-loop (ie. as defined in the PEP 340). One main reason is a common error could be (using the synchronised iterator introduced in the PEP): for l in synchronised(mylock): do_something() It will compile, run, never raise any error but the lock will be acquired and never released ! Then, I think there is no use case of a generator with __error__ in the for-loop as it is now. So, IMO, it is error-prone and useless to have two different syntaxes for such things. Pierre -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68

Pierre Barbier de Reuille wrote:
It's better than that. With the code above, CPython is actually likely to release the lock when the loop exits. Change the code to the below to ensure the lock doesn't get released: sync = synchronised(mylock): for l in sync: do_something()
Hmm. This does make PJE's suggestion of requiring a decorator in order to flag generators for finalisation a little more appealing. Existing generators (without the flag) would not be cleaned up, preserving backwards compatibility. Generators with the flag would allow resource clean up. In this case of no new statement syntax, it would probably make more sense to refer to iterators that get cleaned up as finalised iterators, and a builtin with the obvious name would be: def finalised(obj): obj.__finalise__ = True # The all important flag! return obj The syntax below would still be horrible: for f in opening(filename): for line in f: # process line But such ugliness could be fixed by pushing the inner loop inside the block iterator: for line in opened(filename): # process line @finalised def opened(filename): f = open(filename) try: for line in f: yield line finally: f.close() Then, in Py3K, finalisation could simply become the default for loop behaviour. However, the '__finalise__' flag would result in some impressive code bloat, as any for loop would need to expand to: itr = iter(EXPR1) if getattr(itr, "__finalise__", False): # Finalised semantics # I'm trying to channel Guido here. # This would really look like whatever the PEP 340 block statement # semantics end up being val = arg = None ret = broke = False while True: try: VAR1 = next(itr, arg) except StopIteration: BLOCK2 break try: val = arg = None ret = False BLOCK1 except Exception, val: itr.__error__(val) if ret: try: itr.__error__(StopIteration()) except StopIteration: pass return val else: # Non-finalised semantics arg = None while True: try: VAR1 = next(itr, arg) except StopIteration: BLOCK2 break arg = None BLOCK1 The major danger I see is that you could then write a generator containing a yield inside a try/finally, _without_ applying the finalisation decorator. Leading to exactly the problem described above - the lock (or whatever) is never cleaned up, because the generator is not flagged for finalisation. In this scenario, even destruction of the generator object won't help. Cheers, Nick. P.S. I think PEP 340's proposed for loop semantics are currently incorrect, as BLOCK2 is unreachable. It should look more like the non-finalised semantics above (with BLOCK2 before the break in the except clause) -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan a écrit :
Well indeed, but this will be an implementation-dependant behaviour ...
[...]
Mmmmh ... why introduce a new flag ? Can't you just test the presence of the "__error__" method ? This would lift your problem wouldn't it ?
-- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68

Pierre Barbier de Reuille wrote:
Mmmmh ... why introduce a new flag ? Can't you just test the presence of the "__error__" method ? This would lift your problem wouldn't it ?
Perhaps - it would require doing something a little tricky with generators to allow the programmer to specify whether the generator should be finalised or not. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan <ncoghlan@gmail.com> wrote:
The problem is that BLOCK2 is executed within the while loop (the same problem I had with a fix I offered), which may contain a break for breaking out of a higher-level loop construct. Here's one that works as you intended (though perhaps I'm being a bit to paranoid about the __error__ attribute)... val = arg = None ret = ex_block_2 = False while True: try: VAR1 = next(itr, arg) except StopIteration: ex_block_2 = True break try: val = arg = None ret = False BLOCK1 except Exception, val: if hasattr(itr, '__error__): itr.__error__(val) if ret: try: if hasattr(itr, '__error__'): itr.__error__(StopIteration()) except StopIteration: pass return val if ex_block_2: BLOCK2
Indeed, I also mentioned this on Wednesday. - Josiah

Nick Coghlan <ncoghlan@gmail.com> wrote:
And that bad boy should be... # Non-finalised semantics ex_block_2 = False arg = None while True: try: VAR1 = next(itr, arg) except StopIteration: ex_block_2 = True break arg = None BLOCK1 if ex_block_2: BLOCK2 Josiah Carlson wrote:
Indeed, I also mentioned this on Wednesday.
Though I was somewhat incorrect as code examples I offered express the actual intent. - Josiah

On Fri, Apr 29, 2005, Nick Coghlan wrote:
If you want to emphasise the similarity, the following syntax and explanation is something that occurred to me during lunch today:
We don't want to emphasize the similarity.
-1 -- the Zen of Python implies that we should be able to tell which construct we're using at the beginning of the line. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 09:38 AM 4/29/05 -0700, Aahz wrote:
-1 -- the Zen of Python implies that we should be able to tell which construct we're using at the beginning of the line.
Hm, maybe we should just use "@", then. :) e.g. @synchronized(self): @with_file("foo") as f: # etc. Although I'd personally prefer a no-keyword approach: synchronized(self): with_file("foo") as f: # etc.

[Phillip J. Eby]
I'd like that too, but it was shot down at least once. Maybe we can resurrect it? opening("foo") as f: # etc. is just a beauty! -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, Apr 29, 2005, Guido van Rossum wrote:
I'm still -1 for the same reason I mentioned earlier: function calls spanning multiple lines are moderately common in Python code, and it's hard to distinguish these cases because multi-line calls usually get indented like blocks. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 10:42 AM 4/29/05 -0700, Aahz wrote:
But the indentation of a multi-line call doesn't start with a colon. Or are you saying you're concerned about things like: opening( blah, blah, foo, wah=flah ) as fidgety, widgety, foo: sping() Which is quite ugly, to be sure, but then I don't see where adding an extra keyword helps.

On Fri, Apr 29, 2005, Phillip J. Eby wrote:
Neither does the un-keyworded block. It starts with a colon on the end of the previous line. I thought part of the point of Python was to minimize reliance on punctuation, especially where it's not clearly visible? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 12:05 PM 4/29/05 -0700, Aahz wrote:
Actually, I've just realized that I was misled by your argument into thinking that the possibility of confusing a multi-line call and a block of this sort is a problem. It's not, because template blocks can be viewed as multi-line calls that just happen to include a block of code as one of the arguments. So, mistaking one for the other when you're just skimming the code and not looking at things like "as" or the ":", is really not important. In the second place, the most important cue to understanding the behavior of a template block is the template function itself; the bare syntax gives it the most prominence. Blocks like 'synchronized(self):' should be instantly comprehensible to Java programmers, for example, and 'retry(3):' is also pretty self-explanatory. And so far, template function names and signatures have been quite brief as well.

On Fri, Apr 29, 2005, Phillip J. Eby wrote:
Maybe. I'm not persuaded, but this inclines me toward agreeing with your position.
This works IMO IFF Python is regarded as a language with user-defined syntactical structures. Guido has historically disagreed strongly with that philosophy; until and unless he reverses his opinion, this is precisely why the non-keyword version will continue to receive -1 from me. (As it happens, I agree with Guido, so if Guido wants to change, I'll probably argue until I see good reason. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

[Phillip]
[Aahz]
Actually, I think this is a nice way to have my cake and eat it too: on the one hand, there still isn't any user-defined syntax, because the keyword-less block syntax is still fixed by the compiler. On the other hand, people are free to *think* of it as introducing syntax if it helps them understand the code better. Just as you can think of each distinct @decorator as a separate piece of syntax that modifies a function/method definition. And just as you can think of a function call as a user-defined language extension. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

At 04:02 PM 4/29/05 -0700, Guido van Rossum wrote:
And, amusingly enough, those folks who wanted a decorator suite can now have their wish, e.g.: decorate(classmethod): def something(cls, blah): ... Given a suitable frame-sniffing implementation of 'decorate'. :) By the way, I notice PEP 340 has two outstanding items with my name on them; let me see if I can help eliminate one real quick. Tracebacks: it occurs to me that I may have unintentionally given the impression that I need to pass in an arbitrary traceback, when in fact I only need to pass in the current sys.exc_info(). So, if the error call-in doesn't pass in anything but an error flag, and the template iterator is supposed to just read sys.exc_info(), maybe that would be less of an issue? For one thing, it would make handling arbitrary errors in the template block cleaner, because the traceback for unhandled errors in something like this: synchronized(foo): raise Bar would look something like this: File .... line ... of __main__: synchronized(foo): File .... line ... of synchronized: yield File .... line ... of __main__: raise Bar Which, IMO, is the "correct" traceback for this circumstance, although since the first and last frame would actually be the same, you'd probably only get the lower two entries (the yield and the raise), which is OK too I think. Anyway, I mainly just wanted to note that I'd be fine with having a way to say, "Hey, there's an error, handle it" that doesn't allow passing in the exception or traceback, but is just a flag that means "look at Python's error state" instead of passing a value back in. I can do this because when I need to pass in a traceback, it's because I'm trying to pass a terminated coroutine's error into another coroutine. So, the traceback I want to pass in is Python's existing "last error" state anyway.

[Phillip]
I've updated the PEP (tying a couple of loose ends and making the promised change to the new API); I've decided to change the signature of __exit__() to be a triple matching the return value of sys.exc_info(), IOW the same as the "signature" of the raise-statement. There are still a few loose ends left, including the alternative API that you've proposed (which I'm not super keen on, to be sure, but which is still open for consideration). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
+1 Certainly my favorite because it's direct and easy on the eyes. Second would be:: in opening("foo") as f: # etc. because I can see Aahz's point about introducing the block with a keyword instead of relying on the ":" punctuation and subsequent indentation of the block for skimming code. -Shane Holloway

On Thu, Apr 28, 2005 at 03:55:03PM -0700, Guido van Rossum wrote:
Looks weird to my eyes. On a related note, I was thinking about the extra cleanup 'block' provides. If the 'file' object would provide a suitable iterator, you could write: block open(filename) as line: ... and have the file closed at the end of the block. It does not read so well though. In a way, it seems to make more sense if 'block' called iter() on the expression and 'for' did not. block would guarantee to cleanup iterators that it created. 'for' does not but implictly creates them. Neil

Neil Schemenauer wrote:
Probably makes perfect sense if you're Dutch, though. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

Guido van Rossum wrote:
I don't think it reads well. I would prefer something that would be understandable for a newbie's eyes, even if it fits more with common usage than with the real semantics behind it. For example a Boost-like keyword like: scoped EXPR as VAR: BLOCK scoped EXPR: BLOCK We may argue that it doesn't mean a lot, but at least if a newbie sees the following code, he would easily guess what it does: scoped synchronized(mutex): scoped opening(filename) as file: ... When compared with: in synchronized(mutex): in opening(filename) for file: ... As a C++ programmer, I still dream I could also do: scoped synchronized(mutex) scoped opening(filename) as file ... which would define a block until the end of the current block... Regards, Nicolas

[Nicolas Fleury]
Definitely not. In too many languages, a "scope" is a new namespace, and that's exactly what a block (by whichever name) is *not*. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <gvanrossum@gmail.com> wrote:
scopeless, unscoped, Scope(tm) (we would be required to use the unicode trademark symbol, of course)... It's way too long, and is too close to a pre-existing keyword, but I think 'finalized' is descriptive. But... finalize EXPR as VAR: BLOCK That reads nice... Maybe even 'cleanup', or 'finalize_after_iteration_without_iter_call' (abbreviated to 'faiwic', of course). <1.0 wink> All right, it's late enough. Enough 'ideas' from me tonight. - Josiah

Guido van Rossum wrote:
Humm... what about "context"? context EXPR as VAR: BLOCK I may answer the question myself, but is an alternative syntax without an indentation conceivable? (yes, even since the implicit block could be run multiple times). Because in that case, a keyword like "block" would not look right. It seems to me that in most RAII cases, the block could end at the end of the current block and that's fine, and over-indentation can be avoided. However, I realize that the indentation makes more sense in the context of Python and removes some magic that would be natural for a C++ programmer used to presence of stack... Ok, I answer my question, but "context" still sounds nicer to me than "block";) Regards, Nicolas

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: | How about, instead of trying to emphasize how different a | block-statement is from a for-loop, we emphasize their similarity? | | A regular old loop over a sequence or iterable is written as: | | for VAR in EXPR: | BLOCK | | A variation on this with somewhat different semantics swaps the keywords: | | in EXPR for VAR: | BLOCK | | If you don't need the variable, you can leave the "for VAR" part out: | | in EXPR: | BLOCK | | Too cute? :-) Far too close to the "for" loop, IMHO. I read that, I'd have to remind myself every time, "now, which one is it that can receive values passed back in: for ... in, or in ... for?" I'm definitely -1 on that one: too confusing. Another possibility just occurred to me. How about "using"? ~ using EXPR as VAR: ~ BLOCK Reads similarly to "with", but leaves the "with" keyword open for possible use later. Since it seems traditional for one to introduce oneself upon first posting to python-dev, my name is Robin Munn. Yes, my name is just one letter different from Robin Dunn's. It's not like I *intended* to cause confusion... :-) Anyway, I was introduced to Python a few years ago, around version 2.1 or so, and fell in love with the fact that I could read my own code six months later and understand it. I try to help out where I can, but I don't know the guts of the interpreter, so on python-dev I mostly lurk. Robin Munn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCcbf16OLMk9ZJcBQRAuYpAJ4n24AgsX3SrW0g7jlWJM+HfzHXMwCfTbTq eJ2mLzg1uLZv09KDUemM+WU= =SXux -----END PGP SIGNATURE-----

Whoa! Read the PEP closely. Passing a value back to the iterator (using "continue EXPR") is supported both in the for-loop and in the block-statement; it's new syntax so there's no backwards compatibility issue. The real difference is that when a for-loop is exited through a break, return or exception, the iterator is left untouched; but when the same happens in a block-statement, the iterator's __exit__ or __error__ method is called (I haven't decided what to call it).
Another possibility just occurred to me. How about "using"?
Blah. I'm beginning to like block just fine. With using, the choice of word for the generator name becomes iffy IMO; and it almost sounds like it's a simple renaming: "using X as Y" could mean "Y = X". -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Robin Munn wrote: [snip]
Examples from PEP 340: ========== def synchronized(lock): ... using synchronized(myLock): ... ===== (+0) def opening(filename, mode="r"): ... using opening("/etc/passwd") as f: ... ===== (+1) def auto_retry(n=3, exc=Exception): ... using auto_retry(3, IOError): ... ===== (+1) def synchronized_opening(lock, filename, mode="r"): ... using synchronized_opening("/etc/passwd", myLock) as f: ... ===== (+1) A.R.

Guido van Rossum wrote:
How about, instead of trying to emphasize how different a block-statement is from a for-loop, we emphasize their similarity?
If you want to emphasise the similarity, the following syntax and explanation is something that occurred to me during lunch today: Python offers two variants on the basic iterative loop. "for NAME from EXPR:" enforces finalisation of the iterator. At loop completion, a well-behaved iterator is always completely exhausted. This form supports block management operations, that ensure timely release of resources such as locks or file handles. If the values being iterated over are not required, then the statement may be simplified to "for EXPR:". "for NAME in EXPR:" skips the finalisation step. At loop completion, a well-behaved iterator may still contain additional values. This form allows an iterator to be consumed in stages. Regardless of whether you like the above or not, I think the PEP's proposed use of 'as' is incorrect - it looks like the variable should be referring to the expression being iterated over, rather than the values returned from the iterator. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan a écrit :
Well, I would go a step further and keep only the for-loop syntax, mainly because I don't understand why there is two syntax for things that's so close we can merge them ! You can simply states that the for-loop call the "__error__" method of the object if available without invalidating any other property of the new for-loop (ie. as defined in the PEP 340). One main reason is a common error could be (using the synchronised iterator introduced in the PEP): for l in synchronised(mylock): do_something() It will compile, run, never raise any error but the lock will be acquired and never released ! Then, I think there is no use case of a generator with __error__ in the for-loop as it is now. So, IMO, it is error-prone and useless to have two different syntaxes for such things. Pierre -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68

Pierre Barbier de Reuille wrote:
It's better than that. With the code above, CPython is actually likely to release the lock when the loop exits. Change the code to the below to ensure the lock doesn't get released: sync = synchronised(mylock): for l in sync: do_something()
Hmm. This does make PJE's suggestion of requiring a decorator in order to flag generators for finalisation a little more appealing. Existing generators (without the flag) would not be cleaned up, preserving backwards compatibility. Generators with the flag would allow resource clean up. In this case of no new statement syntax, it would probably make more sense to refer to iterators that get cleaned up as finalised iterators, and a builtin with the obvious name would be: def finalised(obj): obj.__finalise__ = True # The all important flag! return obj The syntax below would still be horrible: for f in opening(filename): for line in f: # process line But such ugliness could be fixed by pushing the inner loop inside the block iterator: for line in opened(filename): # process line @finalised def opened(filename): f = open(filename) try: for line in f: yield line finally: f.close() Then, in Py3K, finalisation could simply become the default for loop behaviour. However, the '__finalise__' flag would result in some impressive code bloat, as any for loop would need to expand to: itr = iter(EXPR1) if getattr(itr, "__finalise__", False): # Finalised semantics # I'm trying to channel Guido here. # This would really look like whatever the PEP 340 block statement # semantics end up being val = arg = None ret = broke = False while True: try: VAR1 = next(itr, arg) except StopIteration: BLOCK2 break try: val = arg = None ret = False BLOCK1 except Exception, val: itr.__error__(val) if ret: try: itr.__error__(StopIteration()) except StopIteration: pass return val else: # Non-finalised semantics arg = None while True: try: VAR1 = next(itr, arg) except StopIteration: BLOCK2 break arg = None BLOCK1 The major danger I see is that you could then write a generator containing a yield inside a try/finally, _without_ applying the finalisation decorator. Leading to exactly the problem described above - the lock (or whatever) is never cleaned up, because the generator is not flagged for finalisation. In this scenario, even destruction of the generator object won't help. Cheers, Nick. P.S. I think PEP 340's proposed for loop semantics are currently incorrect, as BLOCK2 is unreachable. It should look more like the non-finalised semantics above (with BLOCK2 before the break in the except clause) -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan a écrit :
Well indeed, but this will be an implementation-dependant behaviour ...
[...]
Mmmmh ... why introduce a new flag ? Can't you just test the presence of the "__error__" method ? This would lift your problem wouldn't it ?
-- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68

Pierre Barbier de Reuille wrote:
Mmmmh ... why introduce a new flag ? Can't you just test the presence of the "__error__" method ? This would lift your problem wouldn't it ?
Perhaps - it would require doing something a little tricky with generators to allow the programmer to specify whether the generator should be finalised or not. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Nick Coghlan <ncoghlan@gmail.com> wrote:
The problem is that BLOCK2 is executed within the while loop (the same problem I had with a fix I offered), which may contain a break for breaking out of a higher-level loop construct. Here's one that works as you intended (though perhaps I'm being a bit to paranoid about the __error__ attribute)... val = arg = None ret = ex_block_2 = False while True: try: VAR1 = next(itr, arg) except StopIteration: ex_block_2 = True break try: val = arg = None ret = False BLOCK1 except Exception, val: if hasattr(itr, '__error__): itr.__error__(val) if ret: try: if hasattr(itr, '__error__'): itr.__error__(StopIteration()) except StopIteration: pass return val if ex_block_2: BLOCK2
Indeed, I also mentioned this on Wednesday. - Josiah

Nick Coghlan <ncoghlan@gmail.com> wrote:
And that bad boy should be... # Non-finalised semantics ex_block_2 = False arg = None while True: try: VAR1 = next(itr, arg) except StopIteration: ex_block_2 = True break arg = None BLOCK1 if ex_block_2: BLOCK2 Josiah Carlson wrote:
Indeed, I also mentioned this on Wednesday.
Though I was somewhat incorrect as code examples I offered express the actual intent. - Josiah

On Fri, Apr 29, 2005, Nick Coghlan wrote:
If you want to emphasise the similarity, the following syntax and explanation is something that occurred to me during lunch today:
We don't want to emphasize the similarity.
-1 -- the Zen of Python implies that we should be able to tell which construct we're using at the beginning of the line. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 09:38 AM 4/29/05 -0700, Aahz wrote:
-1 -- the Zen of Python implies that we should be able to tell which construct we're using at the beginning of the line.
Hm, maybe we should just use "@", then. :) e.g. @synchronized(self): @with_file("foo") as f: # etc. Although I'd personally prefer a no-keyword approach: synchronized(self): with_file("foo") as f: # etc.

[Phillip J. Eby]
I'd like that too, but it was shot down at least once. Maybe we can resurrect it? opening("foo") as f: # etc. is just a beauty! -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, Apr 29, 2005, Guido van Rossum wrote:
I'm still -1 for the same reason I mentioned earlier: function calls spanning multiple lines are moderately common in Python code, and it's hard to distinguish these cases because multi-line calls usually get indented like blocks. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 10:42 AM 4/29/05 -0700, Aahz wrote:
But the indentation of a multi-line call doesn't start with a colon. Or are you saying you're concerned about things like: opening( blah, blah, foo, wah=flah ) as fidgety, widgety, foo: sping() Which is quite ugly, to be sure, but then I don't see where adding an extra keyword helps.

On Fri, Apr 29, 2005, Phillip J. Eby wrote:
Neither does the un-keyworded block. It starts with a colon on the end of the previous line. I thought part of the point of Python was to minimize reliance on punctuation, especially where it's not clearly visible? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

At 12:05 PM 4/29/05 -0700, Aahz wrote:
Actually, I've just realized that I was misled by your argument into thinking that the possibility of confusing a multi-line call and a block of this sort is a problem. It's not, because template blocks can be viewed as multi-line calls that just happen to include a block of code as one of the arguments. So, mistaking one for the other when you're just skimming the code and not looking at things like "as" or the ":", is really not important. In the second place, the most important cue to understanding the behavior of a template block is the template function itself; the bare syntax gives it the most prominence. Blocks like 'synchronized(self):' should be instantly comprehensible to Java programmers, for example, and 'retry(3):' is also pretty self-explanatory. And so far, template function names and signatures have been quite brief as well.

On Fri, Apr 29, 2005, Phillip J. Eby wrote:
Maybe. I'm not persuaded, but this inclines me toward agreeing with your position.
This works IMO IFF Python is regarded as a language with user-defined syntactical structures. Guido has historically disagreed strongly with that philosophy; until and unless he reverses his opinion, this is precisely why the non-keyword version will continue to receive -1 from me. (As it happens, I agree with Guido, so if Guido wants to change, I'll probably argue until I see good reason. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it."

[Phillip]
[Aahz]
Actually, I think this is a nice way to have my cake and eat it too: on the one hand, there still isn't any user-defined syntax, because the keyword-less block syntax is still fixed by the compiler. On the other hand, people are free to *think* of it as introducing syntax if it helps them understand the code better. Just as you can think of each distinct @decorator as a separate piece of syntax that modifies a function/method definition. And just as you can think of a function call as a user-defined language extension. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

At 04:02 PM 4/29/05 -0700, Guido van Rossum wrote:
And, amusingly enough, those folks who wanted a decorator suite can now have their wish, e.g.: decorate(classmethod): def something(cls, blah): ... Given a suitable frame-sniffing implementation of 'decorate'. :) By the way, I notice PEP 340 has two outstanding items with my name on them; let me see if I can help eliminate one real quick. Tracebacks: it occurs to me that I may have unintentionally given the impression that I need to pass in an arbitrary traceback, when in fact I only need to pass in the current sys.exc_info(). So, if the error call-in doesn't pass in anything but an error flag, and the template iterator is supposed to just read sys.exc_info(), maybe that would be less of an issue? For one thing, it would make handling arbitrary errors in the template block cleaner, because the traceback for unhandled errors in something like this: synchronized(foo): raise Bar would look something like this: File .... line ... of __main__: synchronized(foo): File .... line ... of synchronized: yield File .... line ... of __main__: raise Bar Which, IMO, is the "correct" traceback for this circumstance, although since the first and last frame would actually be the same, you'd probably only get the lower two entries (the yield and the raise), which is OK too I think. Anyway, I mainly just wanted to note that I'd be fine with having a way to say, "Hey, there's an error, handle it" that doesn't allow passing in the exception or traceback, but is just a flag that means "look at Python's error state" instead of passing a value back in. I can do this because when I need to pass in a traceback, it's because I'm trying to pass a terminated coroutine's error into another coroutine. So, the traceback I want to pass in is Python's existing "last error" state anyway.

[Phillip]
I've updated the PEP (tying a couple of loose ends and making the promised change to the new API); I've decided to change the signature of __exit__() to be a triple matching the return value of sys.exc_info(), IOW the same as the "signature" of the raise-statement. There are still a few loose ends left, including the alternative API that you've proposed (which I'm not super keen on, to be sure, but which is still open for consideration). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
+1 Certainly my favorite because it's direct and easy on the eyes. Second would be:: in opening("foo") as f: # etc. because I can see Aahz's point about introducing the block with a keyword instead of relying on the ":" punctuation and subsequent indentation of the block for skimming code. -Shane Holloway
participants (15)
-
Aahz
-
André Roberge
-
David Ascher
-
Greg Ewing
-
Guido van Rossum
-
Josiah Carlson
-
Neil Schemenauer
-
Nick Coghlan
-
Nicolas Fleury
-
Phillip J. Eby
-
Pierre Barbier de Reuille
-
Reinhold Birkenfeld
-
Robin Munn
-
Rodrigo B. de Oliveira
-
Shane Holloway (IEEE)