Make return inside a finally a SyntaxError
Here are two examples of why allowing return inside a finally block is a bad idea: def f(): try: return 3 finally: return 4 def f(): try: raise Exception() finally: return 4 Michael Foord -- http://www.ironpythoninaction.com
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Do you have real code that suffers from this problem? Is this a common mistake for Python beginners? Collin Winter
-- http://www.ironpythoninaction.com On 18 Jul 2009, at 00:44, Collin Winter <collinw@gmail.com> wrote:
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Do you have real code that suffers from this problem? Is this a common mistake for Python beginners?
Collin Winter
Not specifically but I think it is an unfortunate design decision that should be corrected. Michael
On Fri, Jul 17, 2009 at 5:00 PM, Michael<fuzzyman@gmail.com> wrote:
On 18 Jul 2009, at 00:44, Collin Winter <collinw@gmail.com> wrote:
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
Do you have real code that suffers from this problem? Is this a common mistake for Python beginners?
Not specifically but I think it is an unfortunate design decision that should be corrected.
Do you believe def f(): try: raise OSError finally: raise KeyError should be a SyntaxError, too? Collin Winter
-- http://www.ironpythoninaction.com On 18 Jul 2009, at 01:13, Collin Winter <collinw@gmail.com> wrote:
On Fri, Jul 17, 2009 at 5:00 PM, Michael<fuzzyman@gmail.com> wrote:
On 18 Jul 2009, at 00:44, Collin Winter <collinw@gmail.com> wrote:
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
Do you have real code that suffers from this problem? Is this a common mistake for Python beginners?
Not specifically but I think it is an unfortunate design decision that should be corrected.
Do you believe
def f(): try: raise OSError finally: raise KeyError
should be a SyntaxError, too?
No, the semantics of exceptions raised in finally blocks are pretty clear. Michael
Collin Winter
Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
I wouldn't call that a syntax error. Wouldn't you also have to forbid break, continue and, perhaps, yield? Isn't Python for consenting adults, or something like that? :-)
MRAB wrote:
Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
I wouldn't call that a syntax error. Wouldn't you also have to forbid break, continue and, perhaps, yield?
I shouldn't checked before posting... :-( Test 1: break swallows the exception.
for n in range(3): try: print n raise IndexError finally: break
0 Test 2: continue raises a SyntaxError.
for n in range(2): try: print n raise IndexError finally: continue
SyntaxError: 'continue' not supported inside 'finally' clause Test 3: yield propagates the exception on resuming.
def test(): for n in range(3): try: print n raise IndexError finally: print "yielding", n yield n
t = test() t.next() 0 yielding 0 0 t.next()
Traceback (most recent call last): File "<pyshell#24>", line 1, in <module> t.next() File "<pyshell#21>", line 5, in test raise IndexError IndexError
On Jul 17, 8:18 pm, MRAB <pyt...@mrabarnett.plus.com> wrote:
Isn't Python for consenting adults, or something like that? :-)
I agree. And the language reference does shed some light on this behavior: http://docs.python.org/reference/compound_stmts.html#the-try-statement If you don't want return used inside finally blocks then tools like pylint or pychecker can be made to detect this.
Michael schrieb:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Woot! That's totally. It never even occurred to me to use return in the finally block. +1 for the syntax error. Nice catch, Michael! Christian PS: I wasn't sure so I tested the functions. Both functions return 4.
On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
Given that the finally block is guaranteed to run after exiting the try block, I don't see anything objectionable about that. Surprising, perhaps, possibly even an artifact of the CPython implementation, but why is it a "bad idea" that needs to be protected against?
def f(): try: raise Exception() finally: return 4
That's no weirder than: def f(): try: raise Exception() except Exception: return 4 or: def f(): try: raise Exception() except Exception: pass return 4 -- Steven D'Aprano
18-07-2009, 02:59 Steven D'Aprano <steve@pearwood.info> wrote:
On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
Given that the finally block is guaranteed to run after exiting the try block, I don't see anything objectionable about that. Surprising, perhaps, possibly even an artifact of the CPython implementation, but why is it a "bad idea" that needs to be protected against?
def f(): try: raise Exception() finally: return 4
That's no weirder than:
def f(): try: raise Exception() except Exception: return 4
or:
def f(): try: raise Exception() except Exception: pass return 4
The problem is that finally clause is not for catching exceptions but for doing clean-up actions. And here finally-clause *stops* propagation of any exception! (also from lower levels!) def f() try: raise KeyError except KeyError: raise Exception finally: return 4 ...returns 4 without obstacles! It's weird and missleading. I thing it's definitely a bug. -- Jan Kaliszewski <zuo@chopin.edu.pl>
2009/7/18 Steven D'Aprano <steve@pearwood.info>
On Sat, 18 Jul 2009 09:41:10 am Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
Given that the finally block is guaranteed to run after exiting the try block, I don't see anything objectionable about that. Surprising, perhaps, possibly even an artifact of the CPython implementation, but why is it a "bad idea" that needs to be protected against?
It's arbitrary - and constructs with arbitrary interpretations are generally not a *good* thing at the very least. It also makes it harder to read code, where a return in a try:... finally block may not do anything.
def f(): try: raise Exception() finally: return 4
That's no weirder than:
def f(): try: raise Exception() except Exception: return 4
or:
def f(): try: raise Exception() except Exception: pass return 4
It is much weirder. In the case of explicitly using an except you are catching the error and choosing not to propagate it. In a finally where the normal semantics are that the exception is re-raised the return silently circumvents that. Michael
-- Steven D'Aprano _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Michael Foord writes:
It's arbitrary - and constructs with arbitrary interpretations are generally not a *good* thing at the very least. It also makes it harder to read code, where a return in a try:... finally block may not do anything.
But that's true of *all* code in a try block: def foo(): try: x = 42 finally: x = "gotcha!" return x or def bar(): try: x = some_function_normally_returning_42() return x except Exception: return "didn't see that coming!"
It is much weirder. In the case of explicitly using an except you are catching the error and choosing not to propagate it. In a finally where the normal semantics are that the exception is re-raised the return silently circumvents that.
AFAICS "finally" is a restriction of call-with-current-continuation. Ie, the block is "spliced into" the control flow at the exit, and if control "flows off the end" of the finally block, then we go back to doing what we were about to do (return, propagate an exception). But if the block exits otherwise, then we don't. What's "weird" about that? Eg, the finally block could do something that itself raises an exception. If so, I guess you would say that instead of dealing with the "new" exception, we should blow it off and reraise the original exception?
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Well, too bad. As others have (gently) tried to point out, there will always remain a gray area for what should be accepted and what shouldn't. Feel free to put in your company's style guide that this is a bad idea. In most cases it probably is. But I don't think the parser should police this particular issue. (And were you surprised anyway? Since you agree that an exception raised in a finally block has well-defined semantics, why wouldn't a return statement?) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Fri, Jul 17, 2009 at 7:13 PM, Guido van Rossum <guido@python.org> wrote:
Since you agree that an exception raised in a finally block has well-defined semantics, why wouldn't a return statement?)
I think that to some extent exception > return. That is in both of these cases try: return finally: raise Exception() try: raise Exception() finally: return it makes sense to me that the caller would see an exception. I disagree that it should be a syntax error. But it could be a runtime error when an exception would get silently eaten. --- Bruce
On Fri, Jul 17, 2009 at 7:13 PM, Guido van Rossum <guido@python.org> wrote:
Since you agree that an exception raised in a finally block has well-defined semantics, why wouldn't a return statement?
Because I'm not always fortunate enough to be programming in Python, and I don't use any other language where "return" can swallow exceptions. I see how this behavior is internally consistent and even potentially useful. But as a reader of code, I've got internal models of the semantics of both "finally" and "return", and because I associate "finally" with exceptional circumstances, my model for "finally" activates more strongly in this case than my model for "return". Ultimately, it has an adverse effect on my ability to reason about the code. On the other hand, even if I was certain this was a design error, I wouldn't feel it was worth losing backward compatibility over. -- Curt Hagenlocher curt@hagenlocher.org
2009/7/18 Guido van Rossum <guido@python.org>
On Fri, Jul 17, 2009 at 4:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Well, too bad. As others have (gently) tried to point out, there will always remain a gray area for what should be accepted and what shouldn't. Feel free to put in your company's style guide that this is a bad idea. In most cases it probably is. But I don't think the parser should police this particular issue. (And were you surprised anyway? Since you agree that an exception raised in a finally block has well-defined semantics, why wouldn't a return statement?)
I was surprised - I *thought* the semantics of finally were well defined; that exceptions inside the try block will be propagated unless a new exception is raised inside the finally. That return (and break it transpires) silently and implicitly swallow those exceptions is surprising. It is also *odd*. If you *want* behavior then it is trivial to use an except instead of a finally, so I don't see a use case for it. All the best, Michael
-- --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/> )
Michael Foord <fuzzyman@gmail.com> wrote:
I was surprised - I *thought* the semantics of finally were well defined; that exceptions inside the try block will be propagated unless a new exception is raised inside the finally. That return (and break it transpires) silently and implicitly swallow those exceptions is surprising.
It is also *odd*. If you *want* behavior then it is trivial to use an except instead of a finally, so I don't see a use case for it.
+1 and even more. -- Jan Kaliszewski <zuo@chopin.edu.pl>
Michael Foord wrote:
I was surprised - I *thought* the semantics of finally were well defined; that exceptions inside the try block will be propagated unless a new exception is raised inside the finally. That return (and break it transpires) silently and implicitly swallow those exceptions is surprising.
It is also *odd*. If you *want* behavior then it is trivial to use an except instead of a finally, so I don't see a use case for it.
I think the perception of whether this behaviour is odd or not depends on your perception of how a finally block works. If you just see it as a normal sequence of instructions with an implicit "re-raise current exception" at the end of the normal flow of control, then the fact that return, break and raise can all bypass that re-raise by exiting the block differently isn't particularly surprising (the only surprising thing is that this doesn't also hold for continue). However, if your mental model is that finally blocks are special sections of code that promise to reraise the exception *even if* the finally block exits by a means other than control flowing off the end, then I can see why the current behaviour would be surprising. This sounds like a documentation problem to me - the explanation of the finally block in the tutorial and language reference should be pushing people towards the first interpretation, since it is the one actually used by the language. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Michael wrote:
def f(): try: raise Exception() finally: return 4
I actually did this on purpose recently. I felt somewhat evil while I was doing it, but it was the most concise way of getting the behaviour I wanted. I'll probably go back and change it at some though, since exploiting this trick will likely confuse someone in the future. That said, I think this is in the realm of style guides and lint tools rather than the compiler. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan <ncoghlan@gmail.com> writes:
That said, I think this is in the realm of style guides and lint tools rather than the compiler.
That's a good idea. The pylint folks seem quite receptive to patches; perhaps you (Michael) could present a new test to catch this class of likely-error constructs. -- \ “[W]e are still the first generation of users, and for all that | `\ we may have invented the net, we still don't really get it.” | _o__) —Douglas Adams | Ben Finney
Greg Ewing writes:
Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
Don't do those things, then!
Nobody is *forcing* you to write returns inside finally blocks.
+1 Why not just emphasize in the documentation that return (etc) in a finally: suite *will* get executed *after* return or an exception is raised in the try: suite? "These stunts were performed by professionals. Don't try this at home, kids." More specifically, maybe there should be an explicit warning that in a finally: suite if exit_condition: return do_work() # end of suite has (perhaps surprisingly, YMMV) different semantics from if not exit_condition: do_work() # end of suite WDOT?
2009/7/18 Stephen J. Turnbull <stephen@xemacs.org>
Greg Ewing writes:
Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
Don't do those things, then!
Nobody is *forcing* you to write returns inside finally blocks.
+1
Why not just emphasize in the documentation that return (etc) in a finally: suite *will* get executed *after* return or an exception is raised in the try: suite?
That exceptions can be silently swallowed in a finally block (which is 'expected' and usually intended to propagate exceptions) in the presence of a return (or a break apparently) is worrying. Another solution would be to have the exception raised instead of swallowed. There is a harder migration regarding backwards compatibility though - you can only warn when the exception is swallowed which may never be seen by the programmer. Michael
"These stunts were performed by professionals. Don't try this at home, kids."
More specifically, maybe there should be an explicit warning that in a finally: suite
if exit_condition: return do_work() # end of suite
has (perhaps surprisingly, YMMV) different semantics from
if not exit_condition: do_work() # end of suite
WDOT? _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Michael Foord writes:
That exceptions can be silently swallowed in a finally block (which is 'expected' and usually intended to propagate exceptions)
I would say "intended to be executed regardless of control transfer out of the try block".
in the presence of a return (or a break apparently) is worrying.
Another solution would be to have the exception raised instead of swallowed.
That would really piss off anybody who deliberately put a return in a finally block, I think.
2009/7/18 Stephen J. Turnbull <stephen@xemacs.org>
Michael Foord writes:
That exceptions can be silently swallowed in a finally block (which is 'expected' and usually intended to propagate exceptions)
I would say "intended to be executed regardless of control transfer out of the try block".
in the presence of a return (or a break apparently) is worrying.
Another solution would be to have the exception raised instead of swallowed.
That would really piss off anybody who deliberately put a return in a finally block, I think.
I think the current behavior (swallowing an exception that most people would expect to be re-raised) surprises and pisses people off. Guido has already made a pronouncement so not much point continuing this discussion. Michael -- http://www.ironpythoninaction.com/
Michael Foord <fuzzyman@gmail.com> wrote:
2009/7/18 Stephen J. Turnbull <stephen@xemacs.org>
That would really piss off anybody who deliberately put a return in a finally block, I think.
I think the current behavior (swallowing an exception that most people would expect to be re-raised) surprises and pisses people off. Guido has already made a pronouncement so not much point continuing this discussion.
But maybe we can convince BDFL that it should be -- not SyntaxError but -- RuntimeError? *j
Michael Foord writes:
I think the current behavior (swallowing an exception that most people would expect to be re-raised) surprises and pisses people off. Guido has already made a pronouncement so not much point continuing this discussion.
If you say so. I think that there's an education and documentation issue here, though. You write elsewhere: Finally is normally dead simple but the explanation usually includes what it does with exceptions: Finally is *always* run last. Exceptions that occur in the try block are re-raised once the finally has completed. Exceptions inside the finally block will be re-raised [Isn't that wrong? An exception in the finally block will terminate the finally block, no? There's no "re-"raise about it.] The bit that has to be *added* in Python (and usually isn't added because it isn't at all obvious) is: Exceptions are not reraised if the finally returns or breaks. Well, yes, except that I don't think that your explanation accurately reflects what finally is supposed to do. I would write: On *all* exits from the try block, control is first transferred to the finally block. If control flows off its end, the exit proceeds as if there had been no finally block: values are returned by return, exceptions are propagated to the innermost containing handler, and otherwise control flows to the containing suite. An exception raised (either by called code or an explicit raise) in a finally block will supersede a return or exception generated in the try block. Control transfer constructs (break, return) in a finally block also supersede return statements or exceptions generated in the try block. I suppose the difference from what you wrote is a reflection of finally's heritage as a construct completely independent of exceptions.
Michael schrieb:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
I've been bitten by this as well, putting all my cleanup code including the return unthinkingly into the finally clause. It took a while of debugging to find this instance, and realize that the behavior is correct if unexpected. On the other hand, there may be cases where it is actually the most concise way to express what you want. I guess it's like the parameter defaults "wart": once you've fallen over it once, you'll not make the same mistake again, and use it when it's correct to do so. However, why break and continue behave inconsistently, I can't see right now. Maybe continue needs some more trickery implementation-wise? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
I honestly can't read those examples and be surprised. Do I have finally-colored glasses? Maybe "What does f do? It returns 3, but then in finally returns 4" reads understandably, if silly, to me. I will admit that a finally with a non-conditional return or raise will effectively be a blank except: but in that case, as well, I say "well, don't do that" On Fri, Jul 17, 2009 at 7:41 PM, Michael<fuzzyman@gmail.com> wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Michael Foord
-- http://www.ironpythoninaction.com _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Michael wrote:
Here are two examples of why allowing return inside a finally block is a bad idea:
def f(): try: return 3 finally: return 4
def f(): try: raise Exception() finally: return 4
Michael Foord
I just remembered the case where I found it convenient to exploit this behaviour for a quick and dirty script where I wanted as much of a list that worked as I could get before one of the operations threw an exception or I ran out of items: def f(iter): s = [] try: for x in iter: s.append(op(x)) finally: return s The fact that that would be better rewritten using a narrower except statement (see postscript below) doesn't change the fact that it got the job done (which was what was important at the time). Cheers, Nick. P.S. The "better" approach to the above situation: def f(iter): s = [] try: for x in iter: s.append(op(x)) except RuntimeError: pass return s Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Jul 17, 8:18 pm, MRAB <pyt...@mrabarnett.plus.com> wrote:
Isn't Python for consenting adults, or something like that? :-)
I agree. And the language reference does shed some light on this: http://docs.python.org/reference/compound_stmts.html#the-try-statement There are a lot of other things which are bad ideas as well. If you personally don't want to permit return inside finally blocks, tools like pylint or pychecker can be made to verify this.
participants (17)
-
Ben Finney -
Bruce Leban -
Calvin Spealman -
Christian Heimes -
Collin Winter -
Curt Hagenlocher -
Georg Brandl -
Greg Ewing -
Guido van Rossum -
Jan Kaliszewski -
Michael -
Michael Foord
-
MRAB -
Nick Coghlan -
ryles -
Stephen J. Turnbull -
Steven D'Aprano