Mailman 3 May 2005 - Python-checkins

python/nondist/peps pep-0340.txt,1.15,1.16
by gvanrossum＠users.sourceforge.net May 3, 2005

May 3, 2005

Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16785 Modified Files: pep-0340.txt Log Message: Solidify loose ends. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- pep-0340.txt 2 May 2005 23:35:52 -0000 1.15 +++ pep-0340.txt 3 May 2005 16:32:24 -0000 1.16 @@ -22,10 +22,6 @@ (Reliable Acquisition/Release Pairs), and PEP 325 (Resource-Release Support for Generators). - This proposal is just a strawman; we've had a heated debate about - this on python-dev recently [1], and I figured it would be time to - write up a precise spec in PEP form. - Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) @@ -70,7 +66,7 @@ Use Cases - TBD. For now, see the Examples section near the end. + See the Examples section near the end. Specification: the __next__() Method @@ -159,14 +155,15 @@ This is also the case in the body of the block-statement proposed below. + EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to + "continue (1, 2, 3)". + Specification: the Anonymous Block Statement A new statement is proposed with the syntax block EXPR1 as VAR1: BLOCK1 - else: - BLOCK2 Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary expression (but not an expression-list) and VAR1 is an arbitrary @@ -268,14 +265,14 @@ the limitations of all finalization semantics) that the block will be resumed eventually. - I haven't decided yet whether the block-statement should also - allow an optional else-clause, like the for-loop, but I'm leaning - against it. I think it would be confusing, and emphasize the + Unlike the for-loop, the block-statement does not have an + else-clause. I think it would be confusing, and emphasize the "loopiness" of the block-statement, while I want to emphasize its *difference* from a for-loop. In addition, there are several - possible semantics for an else-clause. + possible semantics for an else-clause, and only a very weak use + case. -Specification: Generator Exception Handling +Specification: Generator Exit Handling Generators will implement the new __next__() method API, as well as the old argument-less next() method which becomes an alias for @@ -330,11 +327,10 @@ When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the value attribute of the argument. If it resumes a yield-statement, the value is - ignored (or should this be considered an error?). When the - *initial* call to __next__() receives an argument that is not - None, the generator's execution is started normally; the - argument's value attribute is ignored (or should this be - considered an error?). When __next__() is called without an + ignored (this is similar to ignoring the value returned by a + function call). When the *initial* call to __next__() receives an + argument that is not None, TypeError is raised; this is likely + caused by some logic error. When __next__() is called without an argument or with None as argument, and a yield-expression is resumed, the yield-expression returns None. @@ -360,22 +356,20 @@ cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Loose Ends - - These are things that need to be resolved before accepting the - PEP. +Alternatives Considered - - Fill in the remaining TBD sections. + - Many alternatives have been proposed for 'block', including '@' + and no keyword at all. I haven't seen a proposal for another + keyword that I like better than 'block' yet, and not using a + keyword at all makes many folks (including me) uncomfortable. - - Address Phillip Eby's proposal to have the block-statement use + - Phillip Eby has proposed to have the block-statement use an entirely different API than the for-loop, to differentiate - between the two (a generator would have to be wrapped in a - decorator to make it support the block API). - - - Decide on the keyword ('block', 'with', '@', nothing, or - something else?). - - - Whether a block-statement should allow an else-clause. + between the two. A generator would have to be wrapped in a + decorator to make it support the block API. IMO this adds more + complexity with very little benefit; and we can't relly deny + that the block-statement is conceptually a loop -- it supports + break and continue, after all. Comparison to Thunks @@ -418,7 +412,7 @@ and I'd be bummed if I couldn't write this as: def findSomething(self, key, default=None): - block synchronized(self.lock): + block locking(self.lock): for item in self.elements: if item.matches(key): return item @@ -427,7 +421,7 @@ This particular example can be rewritten using a break: def findSomething(self, key, default=None): - block synchronized(self.lock): + block locking(self.lock): for item in self.elements: if item.matches(key): break @@ -478,16 +472,17 @@ However, the use cases for multiple blocks seem elusive. -Alternatives Considered - - TBD. + (Proposals have since been made to change the implementation of + thunks to remove most of these objections, but the resulting + semantics are fairly complex to explain and to implement, so IMO + that defeats the purpose of using thunks in the first place.) Examples 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: - def synchronized(lock): + def locking(lock): lock.acquire() try: yield @@ -496,7 +491,7 @@ Used as follows: - block synchronized(myLock): + block locking(myLock): # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even # if by an uncaught exception). @@ -549,18 +544,40 @@ 5. It is possible to nest blocks and combine templates: - def synchronized_opening(lock, filename, mode="r"): - block synchronized(lock): + def locking_opening(lock, filename, mode="r"): + block locking(lock): block opening(filename) as f: yield f Used as follows: - block synchronized_opening("/etc/passwd", myLock) as f: + block locking_opening("/etc/passwd", myLock) as f: for line in f: print line.rstrip() - 6. Coroutine example TBD. + 6. It is possible to write a regular iterator with the + semantics of example 1: + + class locking: + def __init__(self, lock): + self.lock = lock + self.state = 0 + def __next__(self, arg=None): + # ignores arg + if self.state: + assert self.state == 1 + self.lock.release() + self.state += 1 + raise StopIteration + else: + self.lock.acquire() + self.state += 1 + return None + def __exit__(self, type, value=None, traceback=None): + assert self.state in (0, 1, 2) + if self.state == 1: + self.lock.release() + raise type, value, traceback Acknowledgements

1 0

python/nondist/peps pep-0340.txt,1.14,1.15
by gvanrossum＠users.sourceforge.net May 2, 2005

May 2, 2005

Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16732 Modified Files: pep-0340.txt Log Message: Fix typo in example 4 found by Chris Ryland. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- pep-0340.txt 2 May 2005 15:33:47 -0000 1.14 +++ pep-0340.txt 2 May 2005 23:35:52 -0000 1.15 @@ -536,7 +536,7 @@ try: yield return - except Exception, err: + except exc, err: # perhaps log exception here continue raise # re-raise the exception we caught earlier @@ -565,13 +565,13 @@ Acknowledgements In no useful order: Alex Martelli, Barry Warsaw, Bob Ippolito, - Brett Cannon, Brian Sabbey, Doug Landauer, Duncan Booth, Fredrik - Lundh, Greg Ewing, Holger Krekel, Jason Diamond, Jim Jewett, - Josiah Carlson, Ka-Ping Yee, Michael Chermside, Michael Hudson, - Neil Schemenauer, Nick Coghlan, Paul Moore, Phillip Eby, Raymond - Hettinger, Samuele Pedroni, Shannon Behrens, Skip Montanaro, - Steven Bethard, Terry Reedy, Tim Delaney, Aahz, and others. - Thanks all for the valuable contributions! + Brett Cannon, Brian Sabbey, Chris Ryland, Doug Landauer, Duncan + Booth, Fredrik Lundh, Greg Ewing, Holger Krekel, Jason Diamond, + Jim Jewett, Josiah Carlson, Ka-Ping Yee, Michael Chermside, + Michael Hudson, Neil Schemenauer, Nick Coghlan, Paul Moore, + Phillip Eby, Raymond Hettinger, Samuele Pedroni, Shannon Behrens, + Skip Montanaro, Steven Bethard, Terry Reedy, Tim Delaney, Aahz, + and others. Thanks all for the valuable contributions! References

1 0

python/nondist/peps pep-0340.txt,1.13,1.14
by gvanrossum＠users.sourceforge.net May 2, 2005

May 2, 2005

Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24412 Modified Files: pep-0340.txt Log Message: Delete the last reference to ContinuationIteration. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- pep-0340.txt 2 May 2005 03:30:07 -0000 1.13 +++ pep-0340.txt 2 May 2005 15:33:47 -0000 1.14 @@ -291,8 +291,7 @@ The yield-statement will be allowed to be used on the right-hand side of an assignment; in that case it is referred to as yield-expression. The value of this yield-expression is None - unless __next__() was called with a ContinueIteration argument; - see below. + unless __next__() was called with an argument; see below. A yield-expression must always be parenthesized except when it occurs at the top-level expression on the right-hand side of an

1 0

python/nondist/peps pep-0340.txt,1.12,1.13
by gvanrossum＠users.sourceforge.net May 2, 2005

May 2, 2005

Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28163 Modified Files: pep-0340.txt Log Message: Establish the "alternative" version. The exception API is called __exit__(), and its signature is the same as that of the raise-statement. Still some loose ends. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pep-0340.txt 29 Apr 2005 18:51:03 -0000 1.12 +++ pep-0340.txt 2 May 2005 03:30:07 -0000 1.13 @@ -26,30 +26,6 @@ this on python-dev recently [1], and I figured it would be time to write up a precise spec in PEP form. -Proposal Evolution - - The discussion on python-dev has changed my mind slightly on how - exceptions should be handled, but I don't have the time to do a - full update of the PEP right now. Basically, I'm now in favor of - a variation on the exception handling proposed in the section - "Alternative __next__() and Generator Exception Handling" below. - - The added twist is that instead of adding a flag argument to - next() and __next__() to indicate whether the previous argument is - a value or an exception, we use a separate API (an __exit__() - method taking an exception and perhaps a traceback) for the - exception. If an iterator doesn't implement __exit__(), the - exception is just re-raised. It is expected that, apart from - generators, very few iterators will implement __exit__(); one use - case would be a fast implementation of synchronized() written in - C. - - The built-in next() function only interfaces to the next() and - __next__() methods; there is no user-friendly API to call - __exit__(). (Or perhaps calling next(itr, exc, traceback) would - call itr.__exit__(exc, traceback) if itr has an __exit__ method - and otherwise raise exc.__class__, exc, traceback?) - Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) @@ -96,40 +72,25 @@ TBD. For now, see the Examples section near the end. -Specification: the Iteration Exception Hierarchy - - Two new built-in exceptions are defined, and StopIteration is - moved in the exception hierarchy: - - class Iteration(Exception): - pass - - class StopIteration(Iteration): - pass - - class ContinueIteration(Iteration): - def __init__(self, value=None): - self.value = None - Specification: the __next__() Method A new method for iterators is proposed, called __next__(). It - takes one optional argument, which defaults to None. If not None, - the argument must be an Iteration instance. Calling the + takes one optional argument, which defaults to None. Calling the __next__() method without argument or with None is equivalent to using the old iterator API, next(). For backwards compatibility, it is recommended that iterators also implement a next() method as an alias for calling the __next__() method without an argument. - Calling the __next__() method with a StopIteration instance - signals the iterator that the caller wants to abort the iteration - sequence; the iterator should respond by doing any necessary - cleanup and raising StopIteration. Calling it with a - ContinueIteration instance signals the iterator that the caller - wants to continue the iteration; the ContinueIteration exception - has a 'value' attribute which may be used by the iterator as a - hint on what to do next. Calling it with a (base class) Iteration - instance is the same as calling it with None. + The argument to the __next__() method may be used by the iterator + as a hint on what to do next. + +Specification: the __exit__() Method + + An optional new method for iterators is proposed, called + __exit__(). It takes up to three arguments which correspond to + the three "arguments" to the raise-statement: type, value, and + traceback. If all three arguments are None, sys.exc_info() may be + consulted to provide suitable default values. Specification: the next() Built-in Function @@ -143,34 +104,43 @@ return itr.next() raise TypeError("next() with arg for old-style iterator") -Specification: the 'for' Loop + This function is proposed because there is often a need to call + the next() method outside a for-loop; the new API, and the + backwards compatibility code, is too ugly to have to repeat in + user code. + + Note that I'm not proposing a built-in function to call the + __exit__() method of an iterator. I don't expect that this will + be called much outside the block-statement. + +Specification: a Change to the 'for' Loop A small change in the translation of the for-loop is proposed. The statement for VAR1 in EXPR1: BLOCK1 + else: + BLOCK2 will be translated as follows: itr = iter(EXPR1) arg = None + brk = False while True: try: VAR1 = next(itr, arg) except StopIteration: + brk = True break arg = None BLOCK1 + if brk: + BLOCK2 - (However, 'itr' and 'arg' are hidden from the user, their scope - ends when the while-loop is exited, and they are not shared with - nested or outer for-loops, and the user cannot override the - built-ins referenced.) - - I'm leaving the translation of an else-clause up to the reader; - note that you can't simply affix the else-clause to the while-loop - since it is always broken out. + (However, the variables 'itr' etc. are not user-visible and the + built-in names used cannot be overridden by the user.) Specification: the Extended 'continue' Statement @@ -180,7 +150,7 @@ is legal and is translated into - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue (Where 'arg' references the corresponding hidden variable from the @@ -195,21 +165,28 @@ block EXPR1 as VAR1: BLOCK1 + else: + BLOCK2 Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary expression (but not an expression-list) and VAR1 is an arbitrary assignment target (which may be a comma-separated list). - The "as VAR1" part is optional; if omitted, the assignment to VAR1 - in the translation below is omitted (but the next() call is not!). + The "as VAR1" part is optional; if omitted, the assignments to + VAR1 in the translation below are omitted (but the expressions + assigned are still evaluated!). - The choice of the 'block' keyword is contentious; it has even been - proposed not to use a keyword at all. PEP 310 uses 'with' for - similar semantics, but I would like to reserve that for a - with-statement similar to the one found in Pascal and VB. To - sidestep this issue momentarily I'm using 'block' until we can - agree on a keyword. (I just found that the C# designers don't - like 'with' [2].) + The choice of the 'block' keyword is contentious; many + alternatives have been proposed, including not to use a keyword at + all (which I actually like). PEP 310 uses 'with' for similar + semantics, but I would like to reserve that for a with-statement + similar to the one found in Pascal and VB. (Though I just found + that the C# designers don't like 'with' [2], and I have to agree + with their reasoning.) To sidestep this issue momentarily I'm + using 'block' until we can agree on the right keyword, if any. + + Note that the 'as' keyword is not contentious (it will finally be + elevated to proper keyword status). Note that it is left in the middle whether a block-statement represents a loop or not; this is up to the iterator, but in the @@ -218,61 +195,66 @@ The translation is subtly different from the translation of a for-loop: iter() is not called, so EXPR1 should already be an iterator (not just an iterable); and the iterator is guaranteed to - be exhausted when the block-statement is left: + be notified when the block-statement is left, regardless if this + is due to a break, return or exception: - itr = EXPR1 - val = arg = None - ret = False + itr = EXPR1 # The iterator + ret = False # True if a return statement is active + val = None # Return value, if ret == True + arg = None # Argument to __next__() (value from continue) + exc = None # sys.exc_info() tuple if an exception is active while True: try: - VAR1 = next(itr, arg) + if exc: + ext = getattr(itr, "__exit__", None) + if ext is not None: + VAR1 = ext(*exc) # May re-raise *exc + else: + raise *exc # Well, the moral equivalent :-) + else: + VAR1 = next(itr, arg) # May raise StopIteration except StopIteration: if ret: return val - if val is not None: - raise val break try: - val = arg = None ret = False + val = arg = exc = None BLOCK1 - except Exception, val: - arg = StopIteration() - - (Again, 'itr' etc. are hidden, and the user cannot override the - built-ins.) + except: + exc = sys.exc_info() - The "raise val" translation is inexact; this is supposed to - re-raise the exact exception that was raised inside BLOCK1, with - the same traceback. We can't use a bare raise-statement because - we've just caught StopIteration. + (Again, the variables and built-ins are hidden from the user.) Inside BLOCK1, the following special translations apply: - "continue" and "continue EXPR2" are always legal; the latter is translated as shown earlier: - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue - "break" is always legal; it is translated into: - arg = StopIteration() + exc = (StopIteration,) continue - "return EXPR3" is only legal when the block-statement is contained in a function definition; it is translated into: - val = EXPR3 + exc = (StopIteration,) ret = True - arg = StopIteration() + val = EXPR3 continue The net effect is that break, continue and return behave much the same as if the block-statement were a for-loop, except that the iterator gets a chance at resource cleanup before the - block-statement is left. The iterator also gets a chance if the - block-statement is left through raising an exception. + block-statement is left, through the optional __exit__() method. + The iterator also gets a chance if the block-statement is left + through raising an exception. If the iterator doesn't have an + __exit__() method, there is no difference with a for-loop (except + that a for-loop calls iter() on EXPR1). Note that a yield-statement (or a yield-expression, see below) in a block-statement is not treated differently. It suspends the @@ -287,15 +269,18 @@ be resumed eventually. I haven't decided yet whether the block-statement should also - allow an optional else-clause, like the for-loop. I think it - would be confusing, and emphasize the "loopiness" of the - block-statement, while I want to emphasize its *difference* from a - for-loop. + allow an optional else-clause, like the for-loop, but I'm leaning + against it. I think it would be confusing, and emphasize the + "loopiness" of the block-statement, while I want to emphasize its + *difference* from a for-loop. In addition, there are several + possible semantics for an else-clause. Specification: Generator Exception Handling Generators will implement the new __next__() method API, as well - as the old argument-less next() method. + as the old argument-less next() method which becomes an alias for + calling __next__() without an argument. They will also implement + the new __exit__() method API. Generators will be allowed to have a yield statement inside a try-finally statement. @@ -330,32 +315,41 @@ are all illegal. (Some of the edge cases are motivated by the current legality of "yield 12, 42".) - When __next__() is called with a StopIteration instance argument, - the yield statement that is resumed by the __next__() call will - raise this StopIteration exception. The generator should re-raise - this exception; it should not yield another value. When the - *initial* call to __next__() receives a StopIteration instance - argument, the generator's execution is aborted and the exception - is re-raised without passing control to the generator's body. + When __exit__() is called, the generator is resumed but at the + point of the yield-statement or -expression the exception + represented by the __exit__ argument(s) is raised. The generator + may re-raise this exception, raise another exception, or yield + another value, execpt that if the exception passed in to + __exit__() was StopIteration, it ought to raise StopIteration + (otherwise the effect would be that a break is turned into + continue, which is unexpected at least). When the *initial* call + resuming the generator is an __exit__() call instead of a + __next__() call, the generator's execution is aborted and the + exception is re-raised without passing control to the generator's + body. - When __next__() is called with a ContinueIteration instance - argument, the yield-expression that it resumes will return the - value attribute of the argument. If it resumes a yield-statement, - the value is ignored. When the *initial* call to __next__() - receives a ContinueIteration instance argument, the generator's - execution is started normally; the argument's value attribute is - ignored. + When __next__() is called with an argument that is not None, the + yield-expression that it resumes will return the value attribute + of the argument. If it resumes a yield-statement, the value is + ignored (or should this be considered an error?). When the + *initial* call to __next__() receives an argument that is not + None, the generator's execution is started normally; the + argument's value attribute is ignored (or should this be + considered an error?). When __next__() is called without an + argument or with None as argument, and a yield-expression is + resumed, the yield-expression returns None. When a generator that has not yet terminated is garbage-collected (either through reference counting or by the cyclical garbage - collector), its __next__() method is called once with a - StopIteration instance argument. Together with the requirement - that __next__() should always re-raise a StopIteration argument, - this guarantees the eventual activation of any finally-clauses - that were active when the generator was last suspended. Of - course, under certain circumstances the generator may never be - garbage-collected. This is no different than the guarantees that - are made about finalizers (__del__() methods) of other objects. + collector), its __exit__() method is called once with + StopIteration as its first argument. Together with the + requirement that a generator ought to raise StopIteration when + __exit__() is called with StopIteration, this guarantees the + eventual activation of any finally-clauses that were active when + the generator was last suspended. Of course, under certain + circumstances the generator may never be garbage-collected. This + is no different than the guarantees that are made about finalizers + (__del__() methods) of other objects. Note: the syntactic extensions to yield make its use very similar to that in Ruby. This is intentional. Do note that in Python the @@ -367,71 +361,6 @@ cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Specification: Alternative __next__() and Generator Exception Handling - - The above specification doesn't let the generator handle general - exceptions. If we want that, we could modify the __next__() API - to take either a value or an exception argument, with an - additional flag argument to distinguish between the two. When the - second argument is True, the first must be an Exception instance, - which raised at the point of the resuming yield; otherwise the - first argument is the value that is returned from the - yield-expression (or ignored by a yield-statement). Wrapping a - regular value in a ContinueIteration is then no longer necessary. - - The next() built-in would be modified likewise: - - def next(itr, arg=None, exc=False): - nxt = getattr(itr, "__next__", None) - if nxt is not None: - return nxt(arg, exc) - if arg is None and not exc: - return itr.next() - raise TypeError("next() with args for old-style iterator") - - The translation of a block-statement would become: - - itr = EXPR1 - arg = val = None - ret = exc = False - while True: - try: - VAR1 = next(itr, arg, exc) - except StopIteration: - if ret: - return val - break - try: - arg = val = None - ret = exc = False - BLOCK1 - except Exception, arg: - exc = True - - The translation of "continue EXPR2" would become: - - arg = EXPR2 - continue - - The translation of "break" inside a block-statement would become: - - arg = StopIteration() - exc = True - continue - - The translation of "return EXPR3" inside a block-statement would - become: - - val = EXPR3 - arg = StopIteration() - ret = exc = True - continue - - The translation of a for-loop would be the same as indicated - earlier (inside a for-loop only the translation of "continue - EXPR2" is changed; break and return translate to themselves in - that case). - Loose Ends These are things that need to be resolved before accepting the @@ -447,17 +376,8 @@ - Decide on the keyword ('block', 'with', '@', nothing, or something else?). - - Phillip Eby wants a way to pass tracebacks along with - exceptions. - - - The translation for the for-loop's else-clause. - - Whether a block-statement should allow an else-clause. - - Which API to use to pass in an exception: itr.__next__(exc), - itr.__next__(exc, True) or itr.__exit__(exc[, traceback]). - Hmm..., perhaps itr.__next__(exc, traceback)? - Comparison to Thunks Alternative semantics proposed for the block-statement turn the @@ -626,7 +546,7 @@ block auto_retry(3, IOError): f = urllib.urlopen("http://python.org/peps/pep-0340.html") - print f.read() + print f.read() 5. It is possible to nest blocks and combine templates:

1 0