Pre-PEP: Exception Reorganization for Python 3.0

Well, it has been discussed at multiple times in the past and I have promised to write this PEP several times, so I finally found enough time to write a PEP on reorganizing exceptions for Python 3.0 . Key points in this PEP is the reworking the hierarchy, requiring anything raised to inherit from a certain superclass, and to change bare 'except' clauses to catch a specific superclass. The first and last points I expect some contention, but the middle point I expect people are okay with (Guido liked the idea when Paul Prescod brought it up and the only person who didn't like it, Holger, ended up being okay with it when the superclass had a reasonable name). One thing people might not notice is that I have some minor ideas listed in the tree in parentheses. If people have an opinion on the ideas please speak up. Otherwise the other major points of contention are covered in the Open Issues section or will be brought up in the usual trashing of PEPs that cover contraversial changes. And please realize this is for Python 3.0! None of this is being proposed for any version before then (they could be, but that is another PEP entirely). Hopefully this PEP along with Ping's PEP 344 will cover the major ideas for exceptions for Python 3.0 . -Brett -------------------------------------------------------------- PEP: XXX Title: Exception Reorganization for Python 3.0 Version: $Revision: 1.5 $ Last-Modified: $Date: 2005/06/07 13:17:37 $ Author: Brett Cannon <brett@python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Jul-2005 Post-History: XX-XXX-XXX Abstract ======== Python, as of version 2.4, has 38 exceptions (including warnings) in the built-in namespace in a rather shallow hierarchy. This list of classes has grown over the years without a chance to learn from mistakes and cleaning up the hierarchy. This PEP proposes doing a reorganization for Python 3000 when backwards-compatibility is not an issue. It also proposes changing bare ``except`` clauses to catch only exceptions inheriting from a specific superclass. Lastly, this PEP proposes, in Python 3000, that all objects to be passed to a ``raise`` statement must inherit from a specific superclass. Rationale ========= Exceptions are a critical part of Python. While exceptions are traditionally used to signal errors in a program, they have also grown to be used for flow control for things such as iterators. There importance is great. But the organization of the exception hierarchy has not been maintained. Mostly for backwards-compatibility reasons, the hierarchy has stayed very flat and old exceptions who usefulness have not been proven have been left in. Making exceptions more hierarchical would help facilitate exception handling by making catching exceptions much more logical with use of superclasses. This should also help lead to less errors from being too broad in what exceptions are caught in an ``except`` clause. A required superclass for all exceptions is also being proposed [Summary2004-08-01]_. By requiring any object that is used in a ``raise`` statement to inherit from a specific superclass, certain attributes (such as those laid out in PEP 344 [PEP344]_) can be guaranteed to exist. This also will lead to the planned removal of string exceptions. Lastly, bare ``except`` clauses can be made less error-prone [Summary2004-09-01]_. Often people use a bare ``except`` when what they really wanted were non-critical exceptions to be caught while more system-specific ones, such as MemoryError, to pass through and to halt the interpreter. With a hierarchy reorganization, bare ``except`` clauses can be changed to only catch exceptions that subclass a non-critical exception superclass, allowing more critical exceptions to propagate to the top of the execution stack as was most likely intended. Philosophy of Reorganization ============================ There are several goals in this reorganization that defined the philosophy used to guide the work. One goal was to prune out unneeded exceptions. Extraneous exceptions should not be left in since it just serves to clutter the built-in namespace. Unneeded exceptions also dilute the importance of other exceptions by splitting uses between several exceptions when all uses should have been under a single exception. Another goal was to introduce any exceptions that were deemed needed to fill any holes in the hierarchy. Most new exceptions were done to flesh out the inheritance hierarchy to make it easier to catch a category of exceptions with a simpler ``except`` clause. Changing inheritance to make it more reasonable was a goal. As stated above, having proper inheritance allows for more accurate ``except`` statements when catching exceptions based on the inheritance tree. Lastly, any renaming to make an exception's use more obvious from its name was done. Having to look up what an exception is meant to be used for because the name does not proper reflect its usage is annoying and slows down debugging. Having a proper name also makes debugging easier on new programmers. New Hierarchy ============= Raisable (new; rename BaseException?) +-- CriticalException (new) +-- KeyboardInterrupt +-- MemoryError +-- SystemExit +-- SystemError (subclass SystemExit?) +-- AssertionError +-- SyntaxError +-- IndentationError +-- TabError +-- Exception (replaces StandardError) +-- ArithmeticError +-- FloatingPointError +-- DivideByZeroError +-- OverflowError +-- ControlFlowException (new) +-- StopIteration +-- GeneratorExit (introduced by PEP 342 [PEP342]_; subclass StopIteration?) +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError (subclass UnicodeEncodeError and UnicodeDecodeError?) +-- LookupError (better name?) +-- IndexError +-- KeyError +-- TypeError +-- AttributeError (subclassing new) +-- OSError (does not inherit from EnvironmentError) +-- WindowsError +-- MacError (new) +-- UnixError (new) +-- IOError (does not inherit from EnvironmentError) +-- EOFError (subclassing new) +-- ImportError +-- NotImplementedError +-- NamespaceException (new) +-- UnboundGlobalError (new name for NameError) +-- UnboundLocalError (no longer subclasses UnboundGlobalError which replaces NameError) +-- WeakReferenceError (rename for ReferenceError) +-- ValueError +-- Warning +-- UserWarning +-- PendingDeprecationWarning +-- DeprecationWarning (subclassing new) +-- SyntaxWarning +-- SemanticsWarning (new name for RuntimeWarning) +-- FutureWarning Differences Compared to Python 2.4 ================================== Changes to exceptions from Python 2.4 can take shape in three forms: removal, renaming, or change in their superclass. There are also new exceptions introduced in the proposed hierarchy. New Exceptions -------------- Raisable '''''''' The base exception **all** exceptions inherit from. The name "Raisable" has been chosen to reflect that it is not meant to be treated as an exception directly, but as the common object that all things to base passed to ``raise`` must inherit from. CriticalException ''''''''''''''''' The superclass for exceptions for which a severe error has occurred that one would not want to catch accidentally. The name is meant to reflect the point that these exceptions are usually raised only when the interpreter should most likely be terminated. MacError '''''''' Introduced for symmertry with WindowsError. UnixError ''''''''' Introduced for symmetry with WindowsError. NamespaceException '''''''''''''''''' To provide a common superclass for exceptions dealing with namespace issues, this exception is introduced. Both UnboundLocalError and UnboundGlobalError (the new name for NameError) inherit from this class. Removed Exceptions ------------------ EnvironmentError '''''''''''''''' Originally meant as an exception for when an event outside of the interpreter needed to raise an exception, its use has been deemed unneeded. While both IOError and OSError inherited this class, the distinction between OS and I/O is significant enough to not warrant having a common subclass that one might base an ``except`` clause on. StandardError ''''''''''''' Originally meant to act as a superclass for exceptions not considered critical, its subclassing was rampant and partially negated its usefulness. The need for StandardError is negated thanks to the CriticalException/Exception dichotomy where Exception takes its place. RuntimeError '''''''''''' Meant to be used when an existing exception does not fit, its usefulness is consider meager in Python 2.4 [exceptionsmodule]_. Also, with the CriticalException/Exception dichotomy, Exception or CriticalException can be raised directly for the same use. Renamed Exceptions ------------------ ReferenceError '''''''''''''' Renamed WeakReferenceError. ReferenceError was added to the built-in exception hierarchy in Python 2.2 [exceptionsmodule]_. Taken directly from the ``weakref`` module, its name comes directly from its original name when it resided in the module. Unfortunately its name does not suggest its connection to weak references and thus deserves a renaming. NameError ''''''''' Renamed UnboundGlobalError. While NameError suggests its use, the name does not properly restrict its scope. With UnboundLocalError already in existence, it seems reasonable to change NameError to UnboundGlobalError to brings its name more in line. RuntimeWarning '''''''''''''' Renamed SemanticsWarning. RuntimeWarning is to represent semantic changes coming in the future. But while saying that affects "runtime" is true, flat-out stating it is a semantic change is much clearer, eliminating any possible association of "runtime" with the virtual machine specifically. Changed Inheritance ------------------- StopIteration and GeneratorExit ''''''''''''''''''''''''''''''' Inherit from ControlFlowException. Collecting all control flow-related exceptions under a common superclass continues with the theme of maintaining a hierarchy. AttributeError '''''''''''''' Inherits TypeError. Since most attribute access errors can be attributed to an object not being the type that one expects, it makes sense for AttributeError to be considered a type error. IOError ''''''' No longer subclasses EnvironmentError. While IOError does fall under the umbrella of EnvironmentError, the need for EnvironmentError has been deemed wanting, and thus has been removed. Thus IOError now subclasses Exception directly. EOFError '''''''' Subclasses IOError. Since an EOF comes from I/O it only makes sense that it be considered an I/O error. UnboundGlobalError and UnboundLocalError '''''''''''''''''''''''''''''''''''''''' Both subclass NamespaceException. Originally UnboundLocalError subclassed NameError (original name of UnboundGlobalError), but with both exceptions specifying a certain namespace, that subclassing has been deemed inappropriate. Instead, a common class between the two of them has been introduced. DeprecationWarning ''''''''''''''''''' Subclasses PendingDeprecationWarning. Since a DeprecationWarning is a PendingDeprecationWarning that is happening sooner, if you care about PendingDeprecationWarnings you are defintely going to care about DeprecationWarnings. Required Base Class for ``raise`` ================================= By requiring all objects passed to a ``raise`` statement, one is guaranteed that all exceptions will have certain attributes. If PEP 342 [PEP344]_ is accepted, the attributes outlined there will be guaranteed to be on all exceptions raised. This should help facilitate debugging by making the querying of information from exceptions much easier. Technically speaking, this can be easily implemented by modifying ``RAISE_VARARGS`` to do an inheritance check and raise TypeError if it does not inherit from Raisable. Bare ``except`` Clauses ======================= Bare ``except`` clauses typically are not meant to catch all exceptions. Often times catching KeyboardInterrupt is not intended. Unfortunately the exception hierarchy in Python 2.4 does not support a simple way to control what exceptions are caught except to be very explicit with a list of exceptions. The reorganization changes this by adding more of a hierarchy. If bare ``except`` statements only catch classes that inherit from Exception, its use will be more in line with what people expect. An implementation can be handled at the bytecode level by modifying the compiler to emit bytecode to do a check for Exception when a bare ``except`` is used. Open Issues =========== Change the Name of Raisable? ---------------------------- It has been argued that Raisable is a bad name to use since it is not an actual word [python-dev1]_. At issue is choosing a name that clearly explains the class' usage. "BaseException" might be acceptable although the name does not quite make it as explicit as Raisable that the class is not meant to be raised directly. Should Bare ``except`` Clauses be Removed? ------------------------------------------ It has been argued that bare ``except`` clauses should be removed entirely [python-dev2]_. The train of thought is that it will force people to specify what they want to catch and not be too broad. One issue with this, though, is whether people will make the proper choices of what to catch or go too broadly. Would a new Python programmer make the right decision and not catch CriticalExceptions or would they go overboard and catch Raisable? It would seem the issue boils down to whether we think people will make proper decisions or make a reasonable solution available for the simplest case assuming they might make a bad one. Change the name of Exception? ----------------------------- Some have suggested names other than "Exception" for the superclass to inherit from for bare ``except`` clauses to match against. The issue with going with a name that is not simplistic is that it raises the chance of people subclassing the wrong exception for what they want. By using the most generic name for the common case it raises the chances that people will use this class for subclassing their own exceptions rather than Raisable or CriticalException. Acknowledgements ================ XXX References ========== .. [PEP342] PEP 342 (Coroutines via Enhanced Generators) (http://www.python.org/peps/pep-0342.html) .. [PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks) (http://www.python.org/peps/pep-0344.html) .. [exceptionsmodules] 'exceptions' module (http://docs.python.org/lib/module-exceptions.html) .. [Summary2004-08-01] python-dev Summary (An exception is an exception, unless it doesn't inherit from Exception) (http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is...) .. [Summary2004-09-01] python-dev Summary (Cleaning the Exception House) (http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-ex...) .. [python-dev1] python-dev email (Exception hierarchy) (http://mail.python.org/pipermail/python-dev/2004-August/047908.html) .. [python-dev2] python-dev email (Dangerous exceptions) (http://mail.python.org/pipermail/python-dev/2004-September/048681.html) Copyright ========= This document has been placed in the public domain.

On 7/29/05, Brett Cannon <bcannon@gmail.com> wrote:
Thanks for getting this ball rolling! (I wonder what happened to Ping's PEP 344 -- he just dropped out, it seems.) Below is some feedback.
I appreciate the attempt to make bare except: less dangerous by not catching certain exceptions, but I worry that these changes might actually make bare except: *more* likely to be used, which is contrary to the intention. Maybe we should just get rid of it, and encourage people to write except Exception: or except Raisable: depending on what they want.
MacError UnixError
Do we really need these? Let's not add things that won't be used.
OTOH there's something to say to unify NameError and AttributeError, isn't there?
-1000. Depending on whether you use open() or os.open() you'll get one or the other for pretty much the same thing. Also, I think that socket.error and select.error should inherit from EnvironmentError (or maybe even from OSError).
RuntimeError
-0.5. I use it all the time as the "application" exception in apps (scripts) that are too small to define their own exception hierarchy.
Yes, but how useful it this? I don't expect to ever come across a situation where I'd want to catch both, so this is more for organization of the docs than for anything else. IMO a good principle for determining the ideal exception hierarchy is whether there's a use case for catching the common base class.
IOError
No longer subclasses EnvironmentError.
-1000, see above.
Then we'll make it a word. Is Java's equivalent, Throwable, any more a word? In this case I like the parallel with Java.
Should Bare ``except`` Clauses be Removed?
Yes, see above. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 7/29/05, Guido van Rossum <gvanrossum@gmail.com> wrote:
No problem. Had this bouncing around in my head for a while. I just needed time to finally type it out.
(I wonder what happened to Ping's PEP 344 -- he just dropped out, it seems.)
Don't know. If he doesn't come back I will pick up the attribute part and roll it into this PEP or a separate PEP.
Fine by me. I would just make sure that it is suggested people typically catch Exception most of the time and discourage direct catching of Raisable unless they know what they are doing.
Is WindowsError used enough to warrant its existence?
Somewhat. You could say that an object is just its own namespace. But I don't see a strong enough correlation to warrant the merging. Or you just saying we should rename AttributeError to NameError?
But it doesn't have to stay that way. This is Python 3.0 we are talking about, so we can change how both work. But if you want to keep EnvironmentError that's fine.
Also, I think that socket.error and select.error should inherit from EnvironmentError (or maybe even from OSError).
Shouldn't the former inherit from IOError?
How about GenericException? I just don't think RuntimeError is quite the right name for this.
Well, Robert says there is a use case in CherryPy.
Fine by me.
Should Bare ``except`` Clauses be Removed?
Yes, see above.
OK. -Brett

Brett Cannon wrote:
+-- ControlFlowException (new)
While I like the idea of ControlFlowException as the "one obvious way" to break out of multiple nested loops, I'm not convinced StopIteration and GeneratorExit should be inheriting from it. However, I'm even less sure StopIteration and GeneratorExit should be in the Exception portion of the hierarchy at all. This is particularly true of GeneratorExit - I would be very concerned if "except Exception:" were to suppress GeneratorExit. If ControlFlowException was moved up to be a peer of Exception, I'd be much happier with the idea - we would then have four major categories of raisable classes: CriticalException - something is seriously wrong ControlFlowException - an exception is being used to affect control flow in a particular way, and should be intercepted only if you know what that control flow is doing Exception - your every day, run of the mill, exceptions Warning - not really an exception.
+-- GeneratorExit (introduced by PEP 342 [PEP342]_; subclass StopIteration?)
Definitely not. GeneratorExit was added because the following idiom prevents the use of StopIteration or a subclass to reliably terminate a generator: try: yield itr.next() catch StopIteration: pass
I don't think there is a particularly strong argument to change the name of LookupError. While Py3K *allows* backwards incompatibility, we shouldn't be gratutitous about it.
I'd actually be inclined to make AttributeError a subclass of NameError: +-- NameError +-- AttributeError (subclassing new) +-- UnboundGlobalError (new) +-- UnboundLocalError Current uses of NameError are replaced with UnboundGlobalError. The re-use of NameError reduces the backwards incompatibility, and also concisely summarises the common feature of these three exceptions.
Like Guido, I believe RuntimeError is very useful for quick-and-dirty scripts. Also, gratuitous incompatibility should be avoided, even for Py3k.
I like Raisable, because it states exactly what the base class indicates: something which can be raised (i.e., used with the "raise" statement). I'm also unclear on why anyone would say it isn't an actual word: http://dictionary.reference.com/search?q=raisable
So long as the tutorial says that anything broader then "except Exception" is going to be wrong 99.9% of the time, I don't think there will be a problem.
Change the name of Exception? -----------------------------
Again, there is a strong backwards compatibility argument for keeping the name Exception for the "this is what you should normally catch" category. Mainly, there's a lot of code out there that already uses it this way. Without a compelling argument in favour of a different name, stick with Exception. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
I can live with this. I was waivering on putting ControlFlowException under CriticalException, but obviously keeping that dichotomy was more important back when bare 'except' clauses were going to be allowed, but now that it looks like they are being removed I am much more open to putting stuff to inherit directly from Raisable.
Good enough for me.
It looks like the backwards-compatilibity gods are starting to stomp on me; I can sacrifice a renaming to appease them.
Hmm. Well, I think it was either Tim or Barry (or both) who once suggested AttributeError inherit from TypeError, but they have not spoken up nor has anyone else so far supporting the idea. But then again people have argued for LookupError instead. I am going to have to think about this one before I attempt to reply on this topic.
OK, but is the name so great? Yes, it is a runtime error, but it just doesn't hit the point of it being a dirty little exception to use when you don't want to create a new one. How about GenericException or SimpleException? Or are the backwards-compatibility gods going to smack me again for this suggestion and spout something off containing the words "not" and "Perl 6"? =)
I think the point was it was not in someone's spell checker. But I consider this point settled since Guido said he was fine with creating a new word.
This seems to be the running consensus so far. I am expecting bare 'except' clauses to not be allowed.
Fine by me. I only tossed that in since originally people were suggesting BaseException or SafeException and the ilk. -Brett

Brett Cannon wrote:
I'm not sure what you meant with "the middle", but if this refers to the renaming and reordering of the exception inheritance tree, you can have my -1 on this. I don't have any problem with making all exception inherit from Exception and disallowing bare except clauses. So that's a bit inverse of what you've obviously expected :-) The reason for my -1 on the renaming and reordering is that it would completely break compatibility of Python 2.x applications with Python 3. Furthermore, there would be next to no chance of writing new applications that run in Python 3 and 2, so you have breakage in both directions. Whether this is desired or not is a different discussion, I just want to point out some important things to consider: When moving from Python 2.x to 3.0, renaming could be solved with the help of some scripts, grep et al. However, there would have to be a good reason for each of these renamings, otherwise this only introduces additional porting overhead. Aliases might be a way to provide soft introduction. Something that scripts will not be able to help with are changes in the inheritance tree, e.g. if an application catches a ValueError, the programmer might also expect UnicodeErrors to get caught, if the application catches a TypeError, this may not be aware that the TypeError could actually be an AttributeError. Many applications define their own exception classes which then normally inherit from StandardError. In the application itself, you'd then usually use StandardError for the generic "catch-all except SystemExit" try-excepts. With the new hierarchy, catching Exception (renamed from StandardError) would let e.g. AssertionErrors, SyntaxErrors, etc. that may well have been produced by debugging code or calls to compile() pass through. Since exceptions are also used in the interpreter itself for masking certain situations and, of course, in the gazillion 3rd party extensions out there, your proposed change would also have to be applied to C code - where finding such subtleties is even harder than in plain Python. This is all fine, if we really intend to go for complete breakage and basically require that Python 3 applications need to be written from scratch. I, for one, don't find the existing exception structure all too bad. It has worked for me for many many years without ever having a true need to work around some quirks in the hierarchy. I've had these issues with some 3rd party extensions using the existing Exception class as base-class for their own error classes in cases where a StandardError inheritance would have been much more appropriate, but even there, listing the exceptions in a tuple has so far always helped. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 30 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

M.-A. Lemburg wrote:
I think the problems with this can be minimised by avoiding making changes we don't need to. I think only a few changes are needed to get a significantly cleaner structure. Here's a fairly minimal proposal, which is closer to the existing 2.4 structure: New Hierarchy ============= Raisable (formerly Exception) +-- CriticalException (new) +-- KeyboardInterrupt +-- MemoryError +-- SystemError +-- ControlFlowException (new) +-- GeneratorExit +-- StopIteration +-- SystemExit +-- Exception (formerly StandardError) +-- AssertionError +-- AttributeError +-- ImportError +-- TypeError +-- WeakReferenceError (formerly ReferenceError) +-- ArithmeticError +-- FloatingPointError +-- DivideByZeroError +-- OverflowError +-- EnvironmentError +-- OSError +-- WindowsError (possibly remove this from builtins) +-- IOError +-- EOFError +-- LookupError +-- IndexError +-- KeyError +-- NameError +-- UnboundLocalError +-- RuntimeError +-- NotImplementedError +-- SyntaxError +-- IndentationError +-- TabError +-- ValueError +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- FutureWarning +-- PendingDeprecationWarning +-- SemanticsWarning (formerly RuntimeWarning) +-- SyntaxWarning +-- UserWarning Changes from Py 2.4: ==================== The big changes: Exception renamed to Raisable StandardError renamed to Exception Rationale for this renaming is that too many people do "except Exception:" when they really mean "except StandardError:". Most applications should cope with this semantics change without much hassle, as the only exceptions that "slip through the net" are KeyboardInterrupt, MemoryError and SystemError. New exception ControlFlowException Make SystemExit a subclass of this Make StopIteration a subclass of this Make GeneratorExit a subclass of this In Python 2.4, the only two exceptions not under StandardError are SystemExit and StopIteration. These are both control flow exceptions - one indicates termination of the application, the other indicates completion of an iterator. Python 2.5 will most likely add GeneratorExit for termination of a generator. Grouping these under a common superclass has a few benefits. - documentation benefit (its obvious why these exceptions are special) - OOWTDI for breaking out of nested loops: raise and catch CFE - module developers can use it to clearly mark their own CFE classes New exception CriticalException Make KeyboardInterrupt a subclass of this Make MemoryError a subclass of this Make SystemError a subclass of this All of these exceptions inherit from StandardError in Python 2.4, but each of them indicates something is seriously wrong. KI indicates Ctrl-C has been pressed, ME indicates the VM has run out of memory, and SE indicates the VM's internals are out of whack. There isn't much a program can or should be doing about these. Certainly, none of them should be getting caught by "except Exception:". The following changes are comparatively minor cleanups: Make EOFError a subclass of IOError Trying to read past the end of a file _is_ an IOError! ReferenceError renamed to WeakReferenceError This recaptures the context that was lost when making this a builtin. RuntimeWarning renamed to SemanticsWarning This better captures the meaning of the warning, and avoids confusion with RuntimeError, which has a very different purpose. A few key differences from Brett's original proposal: I severely culled the Critical Exceptions list. The CE list was dropped back to those where either the user has indicated they want the program to stop, or the VM has reported that there is a severe problem. Everything else went back under Exception. ControlFlowException was made a peer category to Exception and CriticalException. This preserves the location of StopIteration. I also added SystemExit to this group, because it relates to control flow of the entire application (i.e., it is the graceful way to say "get out now"). AttributeError is left as a standalone exception. NameError is not renamed as it actually indicates: - this name is not in locals - this name is not in any containing lexical scope - this name is not in the module globals - this name is not in builtins UnboundLocalError indicates "this name is in locals, but was referenced before it was set". So, I'm not sure what UnboundGlobalError would actually _mean_. Various other exceptions (RuntimeError, EnvironmentError, etc) either weren't deleted or weren't moved. The changes I kept had decent justifications for how they improved the clarity of the exception structure - those I dropped were those I haven't found convincing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On Sat, Jul 30, 2005, Nick Coghlan wrote:
Based on skimming (not close examination): +1 We can probably quibble about bits, but I agree that a Grand Restructure should be avoided. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

At 11:43 PM 7/30/2005 +1000, Nick Coghlan wrote:
I like this a lot, and a good bit of it could actually be done in 2.5, apart from the Exception/StandardError move, assuming also that the renamed errors were also available under their old names. We could probably go so far as to add Raisable to the hierarchy, but I don't think we could actually get quite to your proposed structure without breaking any programs. On the other hand, if we introduce CriticalException and ControlFlowException in 2.5 (inheriting from Exception), and create a new Error base for StandardError, then promote subclassing and catching it instead of Exception, then there will be less to do in 3.0. So, my thoughts for the 2.x series are: Exception CriticalException ControlFlowException Error StandardError with the children of these being as you've proposed. Now, "except Error:" would be the replacement for a bare except in most cases, and we would recommend subclassing one of the three Exception subclasses instead of Exception itself, maybe even introduce a PendingDeprecationWarning for direct Exception subclasses outside the exceptions module, with a DeprecationWarning in 2.6 and beyond. Note that subclassing Error instead of Exception won't hurt any programs that already catch Exception, and if they have to deal with libraries that haven't made the move, they can always use this pattern: except (CriticalException,ControlFlowException): raise except: # whatever

Phillip J. Eby wrote:
If we leave Exception at the top of the hierarchy for Py3k, and use Error to replace StandardError, the hierarchy could look like this: Exception +-- ControlFlowException (new) +-- GeneratorExit +-- KeyboardInterrupt +-- StopIteration +-- SystemExit +-- CriticalError (new) +-- MemoryError +-- SystemError +-- Error (formerly StandardError) +-- AssertionError +-- AttributeError +-- ImportError +-- TypeError +-- WeakReferenceError (formerly ReferenceError) I wouldn't mind using Exception/Error instead of Raisable/Exception - and it seriously reduces the pain of making this transition. Indeed, most of it becomes doable within the 2.x series - the only tricky parts are semantic changes involved with moving the KeyboardInterrupt, MemoryError and SystemError out from under StandardError, and moving EOFError under IOError. So the question is whether or not the Raisable/Exception combination is liked enough that we want to dance through the requisite hoops to get there. Notice that I've classified KeyboardInterrupt as user-initiated control flow and put it under ControlFlowException above. This means that everything under CriticalError and Error actually ends with the word 'Error'. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
I can live with this, but that will require Guido's stamp of approval.
I don't know if I like this change in inheritance. While we do tend to use KeyboardInterrupt as a way to kill a program, is that really control flow, or a critical exception that the program needs to stop because an serious event occurred? I prefer the latter explanation. -Brett

Brett Cannon wrote:
You're probably right. How does the following reasoning sound: SystemExit, GeneratorExit and StopIteration are all deliberately triggered by certain well-defined elements of normal application code. That is, only certain operations will ever result in a ControlFlowException being raised. KeyboardInterrupt is a better fit with MemoryError and SystemError - something that occurs unexpectedly, at an arbitary point during program execution. That is, a CriticalError may be raised when attempting to execute almost any operation. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

At 05:05 PM 7/31/2005 +1000, Nick Coghlan wrote:
Ugh. A KeyboardInterrupt isn't an error, let alone a critical one. The fact that it occurs unexpectedly has nothing to do with it. A CriticalError needs different treatment than a KeyboardInterrupt for things like logging, notifications, etc. Heck, it needs different treatment just because a KeyboardInterrupt is something you can sanely recover from and gracefully shut down with. The odds of you recovering from an actual CriticalError are negligible. It's not the same thing, no matter what explanation you prefer. ;)

On Sunday 31 July 2005 12:17, Brett Cannon wrote:
Yeah, those explanations work for me. I think I am going to have to write an explanation for every exception so its usage is clear.
That said, I agree with Phillip; a KeyboardInterrupt isn't an error, it's asynchronous user input. That makes it a child of Raisable, not Error. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/31/05, Brett Cannon <bcannon@gmail.com> wrote:
I does not seem right to me to think of KeyboardInterrupt as a means to cause program halting. An interpreter could in principle recover from it and resume execution of the program. The behaviour of the current Python interpreters is that upon encountering an uncaught KeyboardInterrupt (as with any uncaught exception), computation is aborted and a backtrace printed. But that is not how it /must be/ as there might be alternative interpreters that upon encountering an uncaught KeyboardInterrupt will pause execution of the program, but then allow the user to either continue or abort execution, effectively not stopping but pausing the program. Because of this possible recovery, thinking of KeyboardInterrupt as "in order to abort the program, the user has caused a keyboard interrupt" is wrong; it should be more like just "the user has interrupted the computation" and whether or not the program is consequently aborted is not predefined. - Willem

Willem Broekema wrote:
Actually, even in some such theoretical "Did you really mean that?" interpreter, the KeyboardInterrupt only gets raised if the interpreter determines that yes, the user really did want the program terminated. Otherwise, no exception is raised at all, and execution continues as usual. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On Sunday 31 July 2005 06:36, Willem Broekema wrote:
Somewhat. An interrupt may well not mean that the program should terminate, but may simply mean that the current operation should be stopped (as in many sorts of command interpreters). It's still *reasonable* to exit the Python interpreter if the application doesn't handle it; I can't think of any other reasonable default interpretation. (The interactive interpreter is simply an application that does handle the interrupt in an application-specific way.) So I think its reasonable to consider it a critical exception, because it's not predictable the way the control flow exceptions are; it's user-generated instead of application-generated. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/31/05, Willem Broekema <metawilm@gmail.com> wrote:
Same goes for MemoryError as well, but you probably don't want to catch that exception either.
But interpreter termination is not guaranteed for any exception, even SystemError since it can be caught and not re-raised. Plus the common use of hittting ctrl-c while an app is running is to kill it, not to pause the execution. But it doesn't sound like you are arguing against putting KeyboardInterrupt under CriticalException, but just the explanation I gave, right? -Brett

On 7/31/05, Brett Cannon <bcannon@gmail.com> wrote:
Well, an possible scenario is that if allocation of memory fails, then the interpreter (not the Python program in it) can detect that it is not caught explicitly and print possible ways of execution, like "try the allocation again" or "abort the program", letting the user determine how to proceed. Although in this case immediately retrying the allocation will fail again, so the user has to have a way to free some objects in the meantime. I realize it's major work to add recovery features to the CPython interpreter, so I don't think CPython will have anything like it soon and therefore also Python-the-language will not. Instead, my reason for mentioning this is to get the _concept_ of recoveries across. I think including (hypothetical, for now) recovery features in a discussion about exceptions is valuable, because that influences whether one thinks a label like "critical" for an exception is appropriate. I'm working on an implementation of Python in Common Lisp. The CL condition system offers recovery features, so this implementation could, too. Instead of the interpreter handling the interrupt in an application-specific way, as Fred said, the interpreter could handle the interrupt by leaving the choice to the user. Concretely, this is how KeyboardInterrupt is handled by a CL interpreter, and thus also how a Python interpreter could handle it: (defun foo () (loop for i from 0 do (format t "~A " i))) (foo) => 0 1 2 3 <CTRL-C> Error: Received signal number 2 (Keyboard interrupt) [condition type: INTERRUPT-SIGNAL] Restart actions (select using :continue): 0: continue computation 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process. :continue 0 => 4 5 6 ...
I hope the above makes the way I'm thinking more clear. Like Phillip J. Eby, I think that labeling KeyboardInterrupt a CriticalException seems wrong; it is not an error and not critical. - Willem

Willem Broekema <metawilm@gmail.com> writes:
Heh, I talked about this at EuroPython... http://starship.python.net/crew/mwh/recexc.pdf The technical barriers are insignificant, really. Cheers, mwh -- Our Constitution never promised us a good or efficient government, just a representative one. And that's what we got. -- http://www.advogato.org/person/mrorganic/diary.html?start=109

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> I hope the above makes the way I'm thinking more clear. Willem> Like Phillip J. Eby, I think that labeling Willem> KeyboardInterrupt a CriticalException seems wrong; it is Willem> not an error and not critical. Uh, according to your example in Common LISP it is indeed an error, and if an unhandled signal whose intended interpretation is "drop the gun and put your hands on your head!" isn't critical, what is?<wink> I didn't miss your point, but I don't see a good reason to oppose that label based on the usual definitions of the words or Common LISP usage, either. It seems to me the relevant question is "is it likely that catching KeyboardInterrupt with 'except Exception:' will get sane behavior from a generic user-defined handler?" I think not; usually you'd like generic error recovery to _not_ bother the user, but KeyboardInterrupt sort of demands interaction with the user, no? So you're going to need a separate routine for KeyboardInterrupt, anyway. I expect that's going to be the normal case. So I would say KeyboardInterrupt should derive from CriticalException, not from Exception. I definitely agree that implementing recovery features is a good idea, and in interactive operation (or with an option to the interpreter), to allow for such recovery in the interpreter itself. For example, the interpreter could keep a small nest egg of memory for the purpose of interacting with the user; this would be harder for a program to do. And in many quickie scripts it would be convenient if the interpreter would drop into interactive mode, not die, if the program encounters a critical exception. But it's still a critical exception to the program written in Python, even if it's easy for the user to handle and the interpreter provides the capability to pass the buck to the user. The program has completely lost control! -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN

On 8/1/05, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Uh, according to your example in Common LISP it is indeed an error,
I think you are referring to the first word of this line: Error: Received signal number 2 (Keyboard interrupt) [condition type: INTERRUPT-SIGNAL] Well, that refers to the fact that it was raised with (error ...). It says nothing about the type of a Keyboad interrupt condition. (The function 'error' vs 'signal' mark the distinction between raising conditions that must be handled otherwise you'll end up in the debugger, and conditions that when not handled are silently ignored.) The CL ANSI standard does not define what kind of condition a Keyboard interrupt is, so the implementations have to make that decision. Although this implementation (Allegro CL) has currently defined it as a subclass of 'error', I'm told it should have been a 'serious-condition' instead ('error' is a subclass of 'serious-condition', which is a subclass of 'condition'), precisely because forms like ignore-errors, like a bare except in Python, will catch it right now when they shouldn't. I assume most of the other Lisp implementations have already defined it as serious-condition. So, in short, Keyboard interrupts in Lisp are a serious-condition, not an error. (And what is labeled CriticalException in this discussion, has in serious-condition Lisp's counterpart.)
and if an unhandled signal whose intended interpretation is "drop the gun and put your hands on your head!" isn't critical, what is?<wink>
Eh, are you serious? <wink>
Well, I'm not opposed to KeyboardInterrupt being in a class that's not a subclass of 'Exception', when the latter is the class used in a bare 'except'. But when CriticalException, despite its name, is not a subclass of Exception, that is a bit strange. I'd prefer the 'condition' and 'error' terminology, and to label a keyboard interrupt a condition, not any kind of exception or error.
I agree with you that it should not be caught in a bare 'except' (or an 'except Exception', when that is equivalent). - Willem

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> So, in short, Keyboard interrupts in Lisp are a Willem> serious-condition, not an error. Willem> (And what is labeled CriticalException in this discussion, Willem> has in serious-condition Lisp's counterpart.) I don't see it that way. Rather, "Raisable" is the closest equivalent to "serious-condition", and "CriticalException" is an intermediate class that has no counterpart in Lisp usage. >> and if an unhandled signal whose intended interpretation is >> "drop the gun and put your hands on your head!" isn't critical, >> what is?<wink> Willem> Eh, are you serious? <wink> Yes. Unhandled, KeyboardInterrupt means that the user has forcibly taken control away from the program without giving it a chance to preserve state, finish responding to (realtime) external conditions, or even activate vacation(1), and the program is entirely at the mercy of the user. Usually, the program then proceeds to die without dignity. If it's a realtime application, killing it is probably the only merciful thing to do. If you were the program, wouldn't you consider that critical? Willem> But when CriticalException, despite its name, is not a Willem> subclass of Exception, that is a bit strange. Granted. It doesn't bother me, but since it bothers both you and Philip Eby, I concede the point; we should find a better name (or not bother with such a class, see below). Willem> I'd prefer the 'condition' and 'error' terminology, and to Willem> label a keyboard interrupt a condition, not any kind of Willem> exception or error. Now, that does bother me.<wink> Anything we will not permit a program to ignore with a bare "except: pass" if it so chooses had better be more serious than merely a "condition". Also, to me a "condition" is something that I poll for, it does not interrupt me. To me, a condition (even a serious one) is precisely the kind of thing that I should be able to ignore with a bare except! Your description of the CL hierarchy makes me wonder if there's any benefit to having a class between Raisable and KeyboardInterrupt. Unlike SystemShutdown or PowerFailure, KeyboardInterrupt does imply presence of a user demanding attention; I suppose that warrants special treatment. On the other hand, I don't see a need for a class whose members share only the property that they are not catchable with a bare except, leading to Raisable -+- Exception +- KeyboardInterrupt +- SystemShutdown +- PowerFailure +- (etc) or even Exception -+- CatchableException +- KeyboardInterrupt +- SystemShutdown +- PowerFailure +- (etc) The latter is my mental model, and would work well with bare excepts. It also would encourage the programmer to think about whether an Exception should be catchable or is a special case, but I don't think that's really helpful except for Python developers, who presumably would be aware of the issues. The former would be a compromise to allow "except Exception" to be a natural idiom, which I prefer to bare excepts on stylistic grounds. On balance, that's what I advocate. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

At 12:25 PM 8/2/2005 +0900, Stephen J. Turnbull wrote:
I don't think that Lisp's idea of an exception hierarchy has much bearing here.
You just said, "Unhandled, KeyboardInterrupt means..." If the program doesn't *want* to handle KeyboardInterrupt, then it obviously *isn't* critical, because it doesn't care. Conversely, if it *does* handle KeyboardInterrupt, then once again, it's not critical by your definition. So, clearly, KeyboardInterrupt is thus *not* critical, and doesn't belong in the CriticalException hierarchy. Note, by the way, that Python programs can disable a KeyboardInterrupt from ever occurring in the first place, whereas none of the other CriticalException classes can be "disabled" because they're actually *error* conditions, while KeyboardInterrupt is just an asynchronous notification - for control flow purposes. Ergo, it's a control flow exception. (Similarly, a Python program can avoid raising any of the other control flow errors; they are by and large optional features.)
On the contrary, it is control-flow exceptions that bare except clauses are most harmful to: StopIteration, SystemExit, and... you guessed it... KeyboardInterrupt. An exception that's being used for control flow is precisely the kind of thing you don't want anything but an explicit except clause to catch. Whether critical errors should also pass bare except clauses is a distinct issue, one which KeyboardInterrupt really doesn't enter into. If you think that a KeyboardInterrupt is an error, then it's an indication that Python's documentation and the current exception class hierarchy has failed to educate you sufficiently, and that we *really* need to add a class like ControlFlowException into the hierarchy to help make sure that other people don't end up sharing your misunderstanding. ;-)

"Phillip" == Phillip J Eby <pje@telecommunity.com> writes:
Phillip> You just said, "Unhandled, KeyboardInterrupt means..." Phillip> If the program doesn't *want* to handle Phillip> KeyboardInterrupt, then it obviously *isn't* critical, Phillip> because it doesn't care. Conversely, if it *does* handle Phillip> KeyboardInterrupt, then once again, it's not critical by Phillip> your definition. That's not my definition. By that argument, no condition that can be handled can be critical. By my definition, the condition only needs to prevent the program from continuing normally when it arises. KeyboardInterrupt is a convention that is used to tell a program that continuing normally is not acceptable behavior, and therefore "critical" by my definition. Under either definition, we'll still need to do something special with MemoryError, KeyboardInterrupt, et amicae, and they still shouldn't be caught by a generic "except Exception". We agree on that, don't we? Phillip> Note, by the way, that Python programs can disable a Phillip> KeyboardInterrupt [...]. Ergo, it's a control flow Phillip> exception. Sure, in some sense---but not in the Python language AFAIK. Which control constructs in the Python language define semantics for continuation after KeyboardInterrupt occurs? Anything that can stop a program but the language doesn't define semantics for continuation is critical and exceptional by my definition. Willem> I'd prefer the 'condition' and 'error' terminology, and to Willem> label a keyboard interrupt a condition, not any kind of Willem> exception or error. >> Now, that does bother me.<wink> [...] Phillip> On the contrary, it is control-flow exceptions that bare Phillip> except clauses are most harmful to: StopIteration, Phillip> SystemExit, and... you guessed it... KeyboardInterrupt. That is a Python semantics issue, but as far as I can see there's unanimity on it. I and (AFAICS) Willem were discussing the connotations of the _names_ at this point, and whether they were suggestive of the semantics we (all!) seem to agree on. I do not find the word "condition" suggestive of the "things 'bare except' should not catch" semantics. I believe enough others will agree with me that the word "condition", even "serious condition", should be avoided. Phillip> An exception that's being used for control flow is Phillip> precisely the kind of thing you don't want anything but Phillip> an explicit except clause to catch. Which is exactly the conclusion I reached: [It] makes me wonder if there's any benefit to having a class [ie, CriticalException] between Raisable and KeyboardInterrupt. ...I don't see a need for a class whose members share only the property that they are not catchable with a bare except.... Now, somebody proposed: Raisable -+- Exception +- ... +- ControlFlowException -+- StopIteration +- KeyboardInterrupt As I wrote above, I see no use for that; I think that's what you're saying too, right? AIUI, you want Raisable -+- Exception +- ... +- StopIteration +- KeyboardInterrupt so that only the appropriate control construct or an explicit except can catch a control flow exception. At least, you've convinced me that "critical exception" is not a concept that should be implemented in the Python language specification. Rather, (for those who think as I do, if there are others<wink>) "critical exception" would be an intuitive guide to a subclass of exceptions that shouldn't be caught by a bare except (or a handler for any superclass except Raisable, for that matter). By the same token, "control flow exception" is a pedagogical concept, not something that should be reified in a ControlFlowException class, right? Phillip> If you think that a KeyboardInterrupt is an error, I have used the word "error" only in quoting Willem, and that's quite deliberate. I don't think that a condition need be an error to be "critical". -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Stephen J. Turnbull wrote:
The use for it is : try: # do stuff except ControlFlowException: raise except Raisable: # handle anything else Sure, you could write it as: try: # do stuff except (CriticalException, Exception, Warning): # handle anything else But the former structure better reflects the programmers intent (handle everything except control flow exceptions). It's a fact that Python uses exceptions for control flow - KeyboardInterrupt [1], StopIteration, SystemExit (and soon to be GeneratorExit as well). Grouping them under a common parent allows them to be dealt with as a group, rather than their names being spelt out explicitly. Actually having this in the exception hierarchy is beneficial from a pedagogical point of view as well - the hierarchy is practically the first thing you encounter when you run "help ('exceptions')" at the interactive prompt. I have a Python 2.5 candidate hierarchy below, which uses dual inheritance to avoid breaking backward compatibility - any existing except clauses will catch all of the exceptions they used to catch. The only new inheritance introduced is to new exceptions, also avoiding backward compatibility problems, as any existing except clauses will let by all of the exceptions they used to let by. There are no removals, but the deprecation process is started in order to change the names of ReferenceError and RuntimeWarning to WeakReferenceError and SemanticsWarning. With this hierarchy, the recommended parent class for application errors becomes Error, and "except Error:" is preferred to any of "except:", "except Exception:" and "except StandardError:" (although these three continue to catch everything they used to catch). The recommended workaround for libraries raising errors which still inherit directly from Exception is: try: # Use library except (ControlFlowException, CriticalError): raise except Exception: # Do stuff (Remove the 'Exception' part if the library is so outdated that it still raises string exceptions) Applications which use exceptions to control the flow of execution rather than to indicate an error (e.g. breaking out of multiple nested loops) are free to use ControlFlowException directly, or else define their own subclasses of ControlFlowException. This hierarchy achieves my main goal for the exception reorganisation, which is to make it easy for scripts and applications to avoid inadvertently swallowing the control flow exceptions and critical errors, while still being able to provide generic error handlers for application faults. (Hmm, the pre-PEP doesn't include that as a goal in the 'Philosophy' section. . .) Python 2.4 Compatible Improved Exception Hierarchy v 0.1 ======================================================== Exception +-- ControlFlowException (new) +-- GeneratorExit (new) +-- StopIteration +-- SystemExit +-- KeyboardInterrupt (dual-inheritance new) +-- StandardError +-- KeyboardInterrupt (dual-inheritance new) +-- CriticalError (new) +-- MemoryError +-- SystemError +-- Error (new) +-- AssertionError +-- AttributeError +-- EOFError +-- ImportError +-- TypeError +-- ReferenceError (deprecated), WeakReferenceError (new alias) +-- ArithmeticError +-- FloatingPointError +-- DivideByZeroError +-- OverflowError +-- EnvironmentError +-- OSError +-- WindowsError +-- IOError +-- LookupError +-- IndexError +-- KeyError +-- NameError +-- UnboundLocalError +-- RuntimeError +-- NotImplementedError +-- SyntaxError +-- IndentationError +-- TabError +-- ValueError +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- FutureWarning +-- PendingDeprecationWarning +-- RuntimeWarning (deprecated), SemanticsWarning (new alias) +-- SyntaxWarning +-- UserWarning Cheers, Nick. [1] PJE has convinced me that I was right in thinking that KeyboardInterrupt was a better fit under ControlFlowExceptions than it was under CriticalError. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Nick Coghlan wrote:
+1. I like this approach of using multiple inheritence to solve the b/w compatibility problem. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Tue, 2005-08-02 at 11:00, Nick Coghlan wrote:
With this hierarchy, the recommended parent class for application errors becomes Error, ...
And presumably Error could also be the recommended exception for quick'n'dirty scripts. Mark Russell

At 08:00 PM 8/2/2005 +1000, Nick Coghlan wrote:
Couldn't we make Error a parent of StandardError, here, and then make the CriticalError subclasses dual-inherit StandardError, i.e.: Error CriticalError MemoryError (also subclass StandardError) SystemError (also subclass StandardError) StandardError ... In this way, we can encourage people to inherit from Error. Or maybe we should just make the primary hierarchy the way we want it to be, and only cross-link exceptions to StandardError that were previously under StandardError, i.e.: Raisable ControlFlowException ... (cross-inherit to StandardError as needed) CriticalError ... (cross-inherit to StandardError as needed) Exception ... This wouldn't avoid "except Exception" and bare except being problems, but at least you can catch the uncatchables and reraise them. Hm. Maybe we should include a Reraisable base for ControlFlowException and CriticalError? Then you could do "except Reraisable: raise" as a nice way to do the right thing until Python 3.0. It seems to me that multiple inheritance is definitely the right idea, though. That way, we can get the hierarchy we really want with only a minimum of boilerplate in pre-3.0 to make it actually work.

On 8/2/05, Phillip J. Eby <pje@telecommunity.com> wrote:
I think that is acceptable. Using multiple inheritance to make sure that the exceptions that have been moved out of the main exception branch seems like it will be the best solution for giving some form of backwards-compatibility for now while allowing things to still move forward and not cripple the changes we want to make.
As in exceptions that don't inherit from Error/StandError/whatever_the_main_exception_is can easily be caught separately?
Yeah. I think name aliasing and multiple inheritance will take us a long way. Warnings should be able to take us the rest of the way. -Brett (who is still waiting for a number; Barry, David, you out there?)

Brett Cannon wrote:
And it will let us get rid of some of the ugliness in my v 0.1 proposal, too (like Error being a child of StandardError, instead of the other way around). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

At 04:13 PM 8/2/2005 +0900, Stephen J. Turnbull wrote:
No, I want ControlFlowException to exist as a parent so that code today can work around the fact that bare "except:" and "except Exception:" catch everything. In Python 3.0, we should have "except Error:" and be able to have it catch everything but control flow exceptions and possibly critical errors.

On Aug 2, 2005, at 12:31 AM, Phillip J. Eby wrote:
No... KeyboardInterrupt (just like other asynchronous exceptions) really should be treated as a critical error. Doing anything other than killing your process off after receiving it is just inviting disaster. Because the exception can have occurred absolutely anywhere, it is unsuitable for normal use. Aborting a function between two arbitrary bytecodes and trying to continue operation is simply a recipe for disaster. For example, in threadable.py between line 200 "saved_state = self._release_save()" and 201 "try: # restore state no matter what (e.g., KeyboardInterrupt)" would be a bad place to hit control-c if you ever need to use that Condition again. This kind of problem is pervasive and unavoidable. If you want to do a clean shutdown on control-c, the only sane way is to install a custom signal handler that doesn't throw an asynchronous exception at you. There's a reason asynchronously killing off threads was deprecated in java. James

At 10:53 AM 8/2/2005 -0400, James Y Knight wrote:
In my personal experience with using KeyboardInterrupt I've only ever needed to do some minor cleanup of external state, such as removing lockfiles, abandoning connections, etc., so I haven't encountered this issue before. I can see, however, why it would be a problem if you were trying to keep the program *running* - but I've been assuming that KeyboardInterrupt is something that always means "attempt to shutdown gracefully". I suppose considering it a critical error might put it more clearly in that category. I'm not 100% convinced, but you've definitely given me something to think about. On the other hand, any exception can happen "between two arbitrary bytecodes", so there are always circumstances that need special attention, or require a "with block_signals" statement or something. I suppose this issue may have to come down to BDFL pronouncement.

On 8/2/05, Stephen J. Turnbull <stephen@xemacs.org> wrote:
That would imply that all raisables are 'serious' in the Lisp sense, which is defined as "all conditions serious enough to require interactive intervention if not handled". Yet Python warnings are raisable (as raisable is the root), but are certainly not serious in the Lisp sense. (This is complicated by that warnings are raised using 'signal'. More below.) Willem:
To clarify myself: a 'serious-condition' in CL stands for "all conditions serious enough to require interactive intervention if not handled"; I meant to label KI a 'serious-condition'. Stephen:
If I understand your position correctly, it is probably not changed yet by the above clarification. <wink> Maybe it will surprise you, that in Lisp a bare except (ignore-errors) does not catch non-serious things like warnings. And if left uncatched, a warning leaks out to the top level, gets printed and subsequently ignored. That's because non-serious conditions are (usually) raised using 'signal', not 'error'. The default top-level warnings handler just prints it, but does not influence the program control flow, so the execution resumes just after the (warn ..) form. This probably marks a very important difference between Python and CL. I think one could say that where in Python one would use a bare except to catch both non-serious and serious exceptions, in CL one normally doesn't bother with catching the non-serious ones because they will not create havoc at an outer level anyway. So in Python a warning must be catched by a bare except, while in Lisp it would not. And from this follow different contraints on the hierarchy. By the way, this is the condition hierarchy in Allegro CL (most of which is prescribed by the ANSI standard): <http://www.franz.com/support/documentation/7.0/doc/errors.htm> - Willem

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> On 8/2/05, Stephen J. Turnbull <stephen@xemacs.org> wrote: >> I don't see it that way. Rather, "Raisable" is the closest >> equivalent to "serious-condition", and "CriticalException" is >> an intermediate class that has no counterpart in Lisp usage. Willem> That would imply that all raisables are 'serious' in the Willem> Lisp sense, No, it implies that Phillip was right when he wrote that the Lisp hierarchy of signals is not relevant (as a whole) to the discussion of Python Raisables. Of course partial analogies are useful. In any case, Nick's idiom of "except ControlFlowException: raise" clarified everything for me. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Jul 30, 2005, at 8:57 PM, Nick Coghlan wrote:
It seems to me that it *increases* the pain of transition. I do not see any reason why adding a new class above the current hierarchy causes any problems. However, changing the recommended base class for user-defined exceptions from "Exception" to "Error" will cause problems for *every* library. Moving KeyboardInterrupt, MemoryError, and SystemError out from under Exception will be a backwards compatibility issue, but that's the case in all proposals. Additionally, I predict the pain from doing that will be minor. In cases where apps want to catch *all* exceptions, they must already use "except:" instead of "except Exception:", as not all exceptions derive from Exception. And catching the above three in any circumstance other than when explicitly mentioned and when wanting to catch every exception is probably wrong. So moving them will probably not cause many programs to malfunction. I think the following could be done in Py2.5: a) add Raisable above Exception b) move KeyboardInterrupt, MemoryError, and SystemError somewhere under Raisable but not under Exception c) pending deprecation warning for "except:" and raising objects that aren't subclasses of "Raisable". d) (not really related, but while I'm at it...) Allow exceptions derived from object (with Raisable too, of course) For Py2.6: a) change pending deprecation warning above to deprecation warning. James

At 12:12 PM 7/31/2005 -0400, James Y Knight wrote:
I think you're ignoring the part where most exception handlers are already broken. At least adding CriticalException and ControlFlowException makes it possible to add this: try: ... except (CriticalException,ControlFlowException): raise except: ... This isn't great, I admit, but at least it would actually *work*. I also don't see how changing the recommended base class from Exception to Error causes *problems* for every library. Sure, it forces them to move (eventually!), but it's a trivial change, and makes it *possible* to do the right thing with exceptions (e.g. except Error:) as soon as all the libraries you depend on have moved to using Error.
Actually, in my tweak of Nick's proposal they're still Exception subclasses, so nothing breaks.
In which case, maybe we should just implement Nick's full proposal, since the only thing that it would break is code that uses "except Exception:" to catch SystemExit or StopIteration.

On Jul 31, 2005, at 12:49 PM, Phillip J. Eby wrote:
Exactly. That is the problem. Adding a new class above Exception in the hierarchy allows everything to work nicely *now*. Recommended practice has been to have exceptions derive from Exception for a looong time. Changing everybody now will take approximately forever, which means the Error class is pretty much useless. By keeping the definition of Exception as "the standard thing you should derive from and catch", and adding a superclass with things you shouldn't catch, you make conversion a lot simpler. If you're not worried about compatibility with ye olde string exceptions, you can start using "except Exception" immediately. If you are, you can do as your example above. And when Python v.Future comes around, "except Exception" will be the only reasonable thing to do. If, on the other hand, we use Exception as the base class and Error as the thing you should use, I predict that even by the time Python v.Future comes out, many libraries/prgrams will still have exceptions deriving from Exception, thus making the Exception/Error distinction somewhat broken. James

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Nick, are you going go start subbing in for Tim when he is busy and take my work that I spent hours on, come up with an alternative that took 10 minutes, and have everyone end up loving your newfangled idea 10x more than my original? =)
As I have said I am fine with moving ControlFlowException out to here, but it depends if others agree.
Good point with SystemExit! I am definitely +1 on changing that inheritance.
I am still up for moving AttributeError, but with the amount of arguing going on between where it should go I am not going to be shocked if we go with the status quo.
If we are going to lack exceptions for other OSs, I say remove it. No reason Windows should get special treatment with a built-in exception.
What about UnboundGlobalError?
+-- RuntimeError
I still don't like the name.
+-- NotImplementedError
Interesting idea, but I don't view them as the same. Everyone uses RuntimeError as a Poor Man's Exception, not as an actual runtime error.
Don't like the idea of having DeprecationWarning inherit from PendingDeprecationWarning? -Brett

Brett Cannon wrote:
It's like editing creative writing - I find it far, far easier to take something good and tweak it to make it better, than to come up with something good in the first place ;)
Exactly my thought. I had it under "NameError" for a while, but had difficulty coming up with a case where lumping it in with the lexical scoping errors was actually beneficial. Eventually, the fact that it is easy to add another exception to an except clause, but hard to remove a child you don't want that inherits from a parent you do want persauded me to leave this one alone.
True. And the interface looks the same as the normal OSError interface, so it should be possible to replace all uses with a basic OSError.
I realised this was a misnomer. A failed name lookup actually means: - the name was not in locals - the name was not in any lexically containing scope - the name was not in the module globals - the name was not in builtins The program doesn't know which of those namespaces was *meant* to contain the name - it only knows that none of them actually contained it. This criticism also applies to the current wording of the NameError text used in this situation (which implies the name should have been in the module globals). Now, a case could possibly be made for separate errors for cases like the following: def f(): global x print x # UnboundGlobalError (currently NameError, with usual text) def f(): def g(): print x g() # UnboundFreeVariableError (currently NameError, with specific text) x = 1 Like UnboundLocalError, in both of these cases, the name is potentially known to the compiler - it simply hasn't been bound to anything yet.
+-- RuntimeError
I still don't like the name.
I'm not that fond of it either - but as the builtin exception most likely to be used (abused?) by user code, I expect changing its name would be more pain than it's worth.
This particular inheritance is copied from Python 2.4 :) I find the main trick with making RuntimeError more palatable is to avoid thinking of 'runtime' in the sense of 'Python runtime', as in 'Python VM'. That's not what this error is about - a Python VM error is a SystemError. Instead, a RuntimeError is a generic application error - something which happened during happened at program runtime. Renaming RuntimeWarning to SemanticsWarning should help with that distinction.
Not really. Mainly because I'm not sure which way the inheritance should go - I could understand someone wanting to suppress PendingDeprecationWarnings without suppressing DeprecationWarnings. On the other hand, there's your argument that a 'DeprecationWarning is just a PendingDeprecationWarning with a shorter timeframe'. Again, I fell back on the concept that Python's except clause makes it easy to ask for both if you want both, but makes it relatively difficult to undo exception inheritance if it isn't what you want. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
You stand on the shoulder of a giant (and I am only partially kidding; people who have met me will get the joke)!
I think this one will require BDFL pronouncement to be moved since this already looks like a contested idea. And I don't think Guido is going to care enough to want to see it moved.
Glad you agree. =)
That was what I was thinking as the use case for UnboundGlobalError. Also, if you read the docs on NameError, it explicitly states that it is for global names that are not found. I am willing to toss in an exception for the failure to find a free variable (although I would call it UnboundFreeError) to flesh this namespace hierarchy out. Then again you could argue you should inherit from each other since a missing local is kind of lack missing the global namespace as well, but that kind of purity just doesn't work for me in this case. [rest of my response to this email is forthcoming] -Brett

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Maybe, but I am still going to nudge for a change. It might get shot down in the end, but at least I can say I tried.
Ah, oops. =) Well, I still don't think they should inherit like that and instead be decoupled.
Right, you shouldn't think that, but isn't that what you thought initially? That is why I want to rename RuntimeError.
Renaming RuntimeWarning to SemanticsWarning should help with that distinction.
Well, not once the name is changed since people won't know about the former name connection. =)
True, but the hierarchy should still properly reflect increasing severity in my opinion. I am going to push for this; we will see if I get pushed back enough to not do it. -Brett

On Saturday 30 July 2005 22:20, Brett Cannon wrote:
I have no idea what you mean by "properly reflect increasing severity". Should the more severe or less severe case be derived from the other? I doubt there is a single answer that we can readily agree on. If DeprecationWarning and PendingDeprecationWarning need a class-hierarchy relationship (and I think they do; it *is* reasonable for someone to deal with them generically if they can reasonably want to deal with either), then it seems they need a common base: +--AnyDeprecationWarning +--DeprecationWarning +--PendingDeprecationWarning -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

Nick Coghlan wrote:
If CriticalException and ControlFlowException are to be siblings of Exception rather than subclasses of it, they should be renamed so that they don't end with "Exception". Otherwise there will be a confusing mismatch between the actual inheritance hierarchy and the one suggested by the naming. Also, I'm not entirely happy about Exception no longer being at the top, because so far the word "exception" in relation to Python has invariably meant "anything that can be raised". This terminology is even embedded in the syntax with the try-except statement. Changing this could to lead to some awkward circumlocutions in the documentation and confusion in discussions. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

On 7/30/05, M.-A. Lemburg <mal@egenix.com> wrote:
Actually, it was referencing the bare 'except' clauses and changing their semantics.
Do you mean all inherit from Exception in order for it be allowed to be passed to 'raise'?
So that's a bit inverse of what you've obviously expected :-)
If this becomes a theme, yes. But specifically coming from you, MAL, no, not really. =)
My view of Python 3.0 was that backwards-compatibility would not be a gimme in anyway. I personally am willing to break stuff in the name of clarity, which is the point of this whole endeavour. While I am willing to back off on some the proposed changes, I do think the basic spirit of it is correct.
Right, so the renaming is not a huge problem.
Right, but if we introduce aliases in Python 2.9 or sooner the transition will be much easier and they will know to rename things. And obviously warnings can be used for people to know that the hierarchy will change. I bet we will put in a PendingDeprecationWarning or SemanticsWarning saying that the inheritance will be different in Python 3.0 . Yes, this will incur more work than had we left it alone, but the whole point of Python 3.0 is to remove quirks.
If we fiddle with the exception code raising a Warning will not be too bad for this. Plus you can grep for PyExc_TypeError easily enough to see where you are using classes that have a new inheritance tree.
I for one don't expect Python 2.x code to run automatically without some fiddling. I think saying from scratch is a little strong.
Lot's of things "just work", but it doesn't mean they can't be improved upon. But it looks like some of my suggestions were overreaching so they will most likely get scaled back. I will also add a section to the PEP discussing how a transition from Python 2.x to Python 3.0 in this regard can be handled. -Brett

On Sat, Jul 30, 2005, Brett Cannon wrote:
My take is that for Python 3.0, backwards compatibility is no longer a critical priority -- but any breakage still needs to be argued for and balanced. We want to avoid unnecessary breakage. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

On 7/29/05, Brett Cannon <bcannon@gmail.com> wrote:
Thanks for getting this ball rolling! (I wonder what happened to Ping's PEP 344 -- he just dropped out, it seems.) Below is some feedback.
I appreciate the attempt to make bare except: less dangerous by not catching certain exceptions, but I worry that these changes might actually make bare except: *more* likely to be used, which is contrary to the intention. Maybe we should just get rid of it, and encourage people to write except Exception: or except Raisable: depending on what they want.
MacError UnixError
Do we really need these? Let's not add things that won't be used.
OTOH there's something to say to unify NameError and AttributeError, isn't there?
-1000. Depending on whether you use open() or os.open() you'll get one or the other for pretty much the same thing. Also, I think that socket.error and select.error should inherit from EnvironmentError (or maybe even from OSError).
RuntimeError
-0.5. I use it all the time as the "application" exception in apps (scripts) that are too small to define their own exception hierarchy.
Yes, but how useful it this? I don't expect to ever come across a situation where I'd want to catch both, so this is more for organization of the docs than for anything else. IMO a good principle for determining the ideal exception hierarchy is whether there's a use case for catching the common base class.
IOError
No longer subclasses EnvironmentError.
-1000, see above.
Then we'll make it a word. Is Java's equivalent, Throwable, any more a word? In this case I like the parallel with Java.
Should Bare ``except`` Clauses be Removed?
Yes, see above. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 7/29/05, Guido van Rossum <gvanrossum@gmail.com> wrote:
No problem. Had this bouncing around in my head for a while. I just needed time to finally type it out.
(I wonder what happened to Ping's PEP 344 -- he just dropped out, it seems.)
Don't know. If he doesn't come back I will pick up the attribute part and roll it into this PEP or a separate PEP.
Fine by me. I would just make sure that it is suggested people typically catch Exception most of the time and discourage direct catching of Raisable unless they know what they are doing.
Is WindowsError used enough to warrant its existence?
Somewhat. You could say that an object is just its own namespace. But I don't see a strong enough correlation to warrant the merging. Or you just saying we should rename AttributeError to NameError?
But it doesn't have to stay that way. This is Python 3.0 we are talking about, so we can change how both work. But if you want to keep EnvironmentError that's fine.
Also, I think that socket.error and select.error should inherit from EnvironmentError (or maybe even from OSError).
Shouldn't the former inherit from IOError?
How about GenericException? I just don't think RuntimeError is quite the right name for this.
Well, Robert says there is a use case in CherryPy.
Fine by me.
Should Bare ``except`` Clauses be Removed?
Yes, see above.
OK. -Brett

Brett Cannon wrote:
+-- ControlFlowException (new)
While I like the idea of ControlFlowException as the "one obvious way" to break out of multiple nested loops, I'm not convinced StopIteration and GeneratorExit should be inheriting from it. However, I'm even less sure StopIteration and GeneratorExit should be in the Exception portion of the hierarchy at all. This is particularly true of GeneratorExit - I would be very concerned if "except Exception:" were to suppress GeneratorExit. If ControlFlowException was moved up to be a peer of Exception, I'd be much happier with the idea - we would then have four major categories of raisable classes: CriticalException - something is seriously wrong ControlFlowException - an exception is being used to affect control flow in a particular way, and should be intercepted only if you know what that control flow is doing Exception - your every day, run of the mill, exceptions Warning - not really an exception.
+-- GeneratorExit (introduced by PEP 342 [PEP342]_; subclass StopIteration?)
Definitely not. GeneratorExit was added because the following idiom prevents the use of StopIteration or a subclass to reliably terminate a generator: try: yield itr.next() catch StopIteration: pass
I don't think there is a particularly strong argument to change the name of LookupError. While Py3K *allows* backwards incompatibility, we shouldn't be gratutitous about it.
I'd actually be inclined to make AttributeError a subclass of NameError: +-- NameError +-- AttributeError (subclassing new) +-- UnboundGlobalError (new) +-- UnboundLocalError Current uses of NameError are replaced with UnboundGlobalError. The re-use of NameError reduces the backwards incompatibility, and also concisely summarises the common feature of these three exceptions.
Like Guido, I believe RuntimeError is very useful for quick-and-dirty scripts. Also, gratuitous incompatibility should be avoided, even for Py3k.
I like Raisable, because it states exactly what the base class indicates: something which can be raised (i.e., used with the "raise" statement). I'm also unclear on why anyone would say it isn't an actual word: http://dictionary.reference.com/search?q=raisable
So long as the tutorial says that anything broader then "except Exception" is going to be wrong 99.9% of the time, I don't think there will be a problem.
Change the name of Exception? -----------------------------
Again, there is a strong backwards compatibility argument for keeping the name Exception for the "this is what you should normally catch" category. Mainly, there's a lot of code out there that already uses it this way. Without a compelling argument in favour of a different name, stick with Exception. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
I can live with this. I was waivering on putting ControlFlowException under CriticalException, but obviously keeping that dichotomy was more important back when bare 'except' clauses were going to be allowed, but now that it looks like they are being removed I am much more open to putting stuff to inherit directly from Raisable.
Good enough for me.
It looks like the backwards-compatilibity gods are starting to stomp on me; I can sacrifice a renaming to appease them.
Hmm. Well, I think it was either Tim or Barry (or both) who once suggested AttributeError inherit from TypeError, but they have not spoken up nor has anyone else so far supporting the idea. But then again people have argued for LookupError instead. I am going to have to think about this one before I attempt to reply on this topic.
OK, but is the name so great? Yes, it is a runtime error, but it just doesn't hit the point of it being a dirty little exception to use when you don't want to create a new one. How about GenericException or SimpleException? Or are the backwards-compatibility gods going to smack me again for this suggestion and spout something off containing the words "not" and "Perl 6"? =)
I think the point was it was not in someone's spell checker. But I consider this point settled since Guido said he was fine with creating a new word.
This seems to be the running consensus so far. I am expecting bare 'except' clauses to not be allowed.
Fine by me. I only tossed that in since originally people were suggesting BaseException or SafeException and the ilk. -Brett

Brett Cannon wrote:
I'm not sure what you meant with "the middle", but if this refers to the renaming and reordering of the exception inheritance tree, you can have my -1 on this. I don't have any problem with making all exception inherit from Exception and disallowing bare except clauses. So that's a bit inverse of what you've obviously expected :-) The reason for my -1 on the renaming and reordering is that it would completely break compatibility of Python 2.x applications with Python 3. Furthermore, there would be next to no chance of writing new applications that run in Python 3 and 2, so you have breakage in both directions. Whether this is desired or not is a different discussion, I just want to point out some important things to consider: When moving from Python 2.x to 3.0, renaming could be solved with the help of some scripts, grep et al. However, there would have to be a good reason for each of these renamings, otherwise this only introduces additional porting overhead. Aliases might be a way to provide soft introduction. Something that scripts will not be able to help with are changes in the inheritance tree, e.g. if an application catches a ValueError, the programmer might also expect UnicodeErrors to get caught, if the application catches a TypeError, this may not be aware that the TypeError could actually be an AttributeError. Many applications define their own exception classes which then normally inherit from StandardError. In the application itself, you'd then usually use StandardError for the generic "catch-all except SystemExit" try-excepts. With the new hierarchy, catching Exception (renamed from StandardError) would let e.g. AssertionErrors, SyntaxErrors, etc. that may well have been produced by debugging code or calls to compile() pass through. Since exceptions are also used in the interpreter itself for masking certain situations and, of course, in the gazillion 3rd party extensions out there, your proposed change would also have to be applied to C code - where finding such subtleties is even harder than in plain Python. This is all fine, if we really intend to go for complete breakage and basically require that Python 3 applications need to be written from scratch. I, for one, don't find the existing exception structure all too bad. It has worked for me for many many years without ever having a true need to work around some quirks in the hierarchy. I've had these issues with some 3rd party extensions using the existing Exception class as base-class for their own error classes in cases where a StandardError inheritance would have been much more appropriate, but even there, listing the exceptions in a tuple has so far always helped. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 30 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

M.-A. Lemburg wrote:
I think the problems with this can be minimised by avoiding making changes we don't need to. I think only a few changes are needed to get a significantly cleaner structure. Here's a fairly minimal proposal, which is closer to the existing 2.4 structure: New Hierarchy ============= Raisable (formerly Exception) +-- CriticalException (new) +-- KeyboardInterrupt +-- MemoryError +-- SystemError +-- ControlFlowException (new) +-- GeneratorExit +-- StopIteration +-- SystemExit +-- Exception (formerly StandardError) +-- AssertionError +-- AttributeError +-- ImportError +-- TypeError +-- WeakReferenceError (formerly ReferenceError) +-- ArithmeticError +-- FloatingPointError +-- DivideByZeroError +-- OverflowError +-- EnvironmentError +-- OSError +-- WindowsError (possibly remove this from builtins) +-- IOError +-- EOFError +-- LookupError +-- IndexError +-- KeyError +-- NameError +-- UnboundLocalError +-- RuntimeError +-- NotImplementedError +-- SyntaxError +-- IndentationError +-- TabError +-- ValueError +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- FutureWarning +-- PendingDeprecationWarning +-- SemanticsWarning (formerly RuntimeWarning) +-- SyntaxWarning +-- UserWarning Changes from Py 2.4: ==================== The big changes: Exception renamed to Raisable StandardError renamed to Exception Rationale for this renaming is that too many people do "except Exception:" when they really mean "except StandardError:". Most applications should cope with this semantics change without much hassle, as the only exceptions that "slip through the net" are KeyboardInterrupt, MemoryError and SystemError. New exception ControlFlowException Make SystemExit a subclass of this Make StopIteration a subclass of this Make GeneratorExit a subclass of this In Python 2.4, the only two exceptions not under StandardError are SystemExit and StopIteration. These are both control flow exceptions - one indicates termination of the application, the other indicates completion of an iterator. Python 2.5 will most likely add GeneratorExit for termination of a generator. Grouping these under a common superclass has a few benefits. - documentation benefit (its obvious why these exceptions are special) - OOWTDI for breaking out of nested loops: raise and catch CFE - module developers can use it to clearly mark their own CFE classes New exception CriticalException Make KeyboardInterrupt a subclass of this Make MemoryError a subclass of this Make SystemError a subclass of this All of these exceptions inherit from StandardError in Python 2.4, but each of them indicates something is seriously wrong. KI indicates Ctrl-C has been pressed, ME indicates the VM has run out of memory, and SE indicates the VM's internals are out of whack. There isn't much a program can or should be doing about these. Certainly, none of them should be getting caught by "except Exception:". The following changes are comparatively minor cleanups: Make EOFError a subclass of IOError Trying to read past the end of a file _is_ an IOError! ReferenceError renamed to WeakReferenceError This recaptures the context that was lost when making this a builtin. RuntimeWarning renamed to SemanticsWarning This better captures the meaning of the warning, and avoids confusion with RuntimeError, which has a very different purpose. A few key differences from Brett's original proposal: I severely culled the Critical Exceptions list. The CE list was dropped back to those where either the user has indicated they want the program to stop, or the VM has reported that there is a severe problem. Everything else went back under Exception. ControlFlowException was made a peer category to Exception and CriticalException. This preserves the location of StopIteration. I also added SystemExit to this group, because it relates to control flow of the entire application (i.e., it is the graceful way to say "get out now"). AttributeError is left as a standalone exception. NameError is not renamed as it actually indicates: - this name is not in locals - this name is not in any containing lexical scope - this name is not in the module globals - this name is not in builtins UnboundLocalError indicates "this name is in locals, but was referenced before it was set". So, I'm not sure what UnboundGlobalError would actually _mean_. Various other exceptions (RuntimeError, EnvironmentError, etc) either weren't deleted or weren't moved. The changes I kept had decent justifications for how they improved the clarity of the exception structure - those I dropped were those I haven't found convincing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On Sat, Jul 30, 2005, Nick Coghlan wrote:
Based on skimming (not close examination): +1 We can probably quibble about bits, but I agree that a Grand Restructure should be avoided. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

At 11:43 PM 7/30/2005 +1000, Nick Coghlan wrote:
I like this a lot, and a good bit of it could actually be done in 2.5, apart from the Exception/StandardError move, assuming also that the renamed errors were also available under their old names. We could probably go so far as to add Raisable to the hierarchy, but I don't think we could actually get quite to your proposed structure without breaking any programs. On the other hand, if we introduce CriticalException and ControlFlowException in 2.5 (inheriting from Exception), and create a new Error base for StandardError, then promote subclassing and catching it instead of Exception, then there will be less to do in 3.0. So, my thoughts for the 2.x series are: Exception CriticalException ControlFlowException Error StandardError with the children of these being as you've proposed. Now, "except Error:" would be the replacement for a bare except in most cases, and we would recommend subclassing one of the three Exception subclasses instead of Exception itself, maybe even introduce a PendingDeprecationWarning for direct Exception subclasses outside the exceptions module, with a DeprecationWarning in 2.6 and beyond. Note that subclassing Error instead of Exception won't hurt any programs that already catch Exception, and if they have to deal with libraries that haven't made the move, they can always use this pattern: except (CriticalException,ControlFlowException): raise except: # whatever

Phillip J. Eby wrote:
If we leave Exception at the top of the hierarchy for Py3k, and use Error to replace StandardError, the hierarchy could look like this: Exception +-- ControlFlowException (new) +-- GeneratorExit +-- KeyboardInterrupt +-- StopIteration +-- SystemExit +-- CriticalError (new) +-- MemoryError +-- SystemError +-- Error (formerly StandardError) +-- AssertionError +-- AttributeError +-- ImportError +-- TypeError +-- WeakReferenceError (formerly ReferenceError) I wouldn't mind using Exception/Error instead of Raisable/Exception - and it seriously reduces the pain of making this transition. Indeed, most of it becomes doable within the 2.x series - the only tricky parts are semantic changes involved with moving the KeyboardInterrupt, MemoryError and SystemError out from under StandardError, and moving EOFError under IOError. So the question is whether or not the Raisable/Exception combination is liked enough that we want to dance through the requisite hoops to get there. Notice that I've classified KeyboardInterrupt as user-initiated control flow and put it under ControlFlowException above. This means that everything under CriticalError and Error actually ends with the word 'Error'. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
I can live with this, but that will require Guido's stamp of approval.
I don't know if I like this change in inheritance. While we do tend to use KeyboardInterrupt as a way to kill a program, is that really control flow, or a critical exception that the program needs to stop because an serious event occurred? I prefer the latter explanation. -Brett

Brett Cannon wrote:
You're probably right. How does the following reasoning sound: SystemExit, GeneratorExit and StopIteration are all deliberately triggered by certain well-defined elements of normal application code. That is, only certain operations will ever result in a ControlFlowException being raised. KeyboardInterrupt is a better fit with MemoryError and SystemError - something that occurs unexpectedly, at an arbitary point during program execution. That is, a CriticalError may be raised when attempting to execute almost any operation. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

At 05:05 PM 7/31/2005 +1000, Nick Coghlan wrote:
Ugh. A KeyboardInterrupt isn't an error, let alone a critical one. The fact that it occurs unexpectedly has nothing to do with it. A CriticalError needs different treatment than a KeyboardInterrupt for things like logging, notifications, etc. Heck, it needs different treatment just because a KeyboardInterrupt is something you can sanely recover from and gracefully shut down with. The odds of you recovering from an actual CriticalError are negligible. It's not the same thing, no matter what explanation you prefer. ;)

On Sunday 31 July 2005 12:17, Brett Cannon wrote:
Yeah, those explanations work for me. I think I am going to have to write an explanation for every exception so its usage is clear.
That said, I agree with Phillip; a KeyboardInterrupt isn't an error, it's asynchronous user input. That makes it a child of Raisable, not Error. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/31/05, Brett Cannon <bcannon@gmail.com> wrote:
I does not seem right to me to think of KeyboardInterrupt as a means to cause program halting. An interpreter could in principle recover from it and resume execution of the program. The behaviour of the current Python interpreters is that upon encountering an uncaught KeyboardInterrupt (as with any uncaught exception), computation is aborted and a backtrace printed. But that is not how it /must be/ as there might be alternative interpreters that upon encountering an uncaught KeyboardInterrupt will pause execution of the program, but then allow the user to either continue or abort execution, effectively not stopping but pausing the program. Because of this possible recovery, thinking of KeyboardInterrupt as "in order to abort the program, the user has caused a keyboard interrupt" is wrong; it should be more like just "the user has interrupted the computation" and whether or not the program is consequently aborted is not predefined. - Willem

Willem Broekema wrote:
Actually, even in some such theoretical "Did you really mean that?" interpreter, the KeyboardInterrupt only gets raised if the interpreter determines that yes, the user really did want the program terminated. Otherwise, no exception is raised at all, and execution continues as usual. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On Sunday 31 July 2005 06:36, Willem Broekema wrote:
Somewhat. An interrupt may well not mean that the program should terminate, but may simply mean that the current operation should be stopped (as in many sorts of command interpreters). It's still *reasonable* to exit the Python interpreter if the application doesn't handle it; I can't think of any other reasonable default interpretation. (The interactive interpreter is simply an application that does handle the interrupt in an application-specific way.) So I think its reasonable to consider it a critical exception, because it's not predictable the way the control flow exceptions are; it's user-generated instead of application-generated. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/31/05, Willem Broekema <metawilm@gmail.com> wrote:
Same goes for MemoryError as well, but you probably don't want to catch that exception either.
But interpreter termination is not guaranteed for any exception, even SystemError since it can be caught and not re-raised. Plus the common use of hittting ctrl-c while an app is running is to kill it, not to pause the execution. But it doesn't sound like you are arguing against putting KeyboardInterrupt under CriticalException, but just the explanation I gave, right? -Brett

On 7/31/05, Brett Cannon <bcannon@gmail.com> wrote:
Well, an possible scenario is that if allocation of memory fails, then the interpreter (not the Python program in it) can detect that it is not caught explicitly and print possible ways of execution, like "try the allocation again" or "abort the program", letting the user determine how to proceed. Although in this case immediately retrying the allocation will fail again, so the user has to have a way to free some objects in the meantime. I realize it's major work to add recovery features to the CPython interpreter, so I don't think CPython will have anything like it soon and therefore also Python-the-language will not. Instead, my reason for mentioning this is to get the _concept_ of recoveries across. I think including (hypothetical, for now) recovery features in a discussion about exceptions is valuable, because that influences whether one thinks a label like "critical" for an exception is appropriate. I'm working on an implementation of Python in Common Lisp. The CL condition system offers recovery features, so this implementation could, too. Instead of the interpreter handling the interrupt in an application-specific way, as Fred said, the interpreter could handle the interrupt by leaving the choice to the user. Concretely, this is how KeyboardInterrupt is handled by a CL interpreter, and thus also how a Python interpreter could handle it: (defun foo () (loop for i from 0 do (format t "~A " i))) (foo) => 0 1 2 3 <CTRL-C> Error: Received signal number 2 (Keyboard interrupt) [condition type: INTERRUPT-SIGNAL] Restart actions (select using :continue): 0: continue computation 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process. :continue 0 => 4 5 6 ...
I hope the above makes the way I'm thinking more clear. Like Phillip J. Eby, I think that labeling KeyboardInterrupt a CriticalException seems wrong; it is not an error and not critical. - Willem

Willem Broekema <metawilm@gmail.com> writes:
Heh, I talked about this at EuroPython... http://starship.python.net/crew/mwh/recexc.pdf The technical barriers are insignificant, really. Cheers, mwh -- Our Constitution never promised us a good or efficient government, just a representative one. And that's what we got. -- http://www.advogato.org/person/mrorganic/diary.html?start=109

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> I hope the above makes the way I'm thinking more clear. Willem> Like Phillip J. Eby, I think that labeling Willem> KeyboardInterrupt a CriticalException seems wrong; it is Willem> not an error and not critical. Uh, according to your example in Common LISP it is indeed an error, and if an unhandled signal whose intended interpretation is "drop the gun and put your hands on your head!" isn't critical, what is?<wink> I didn't miss your point, but I don't see a good reason to oppose that label based on the usual definitions of the words or Common LISP usage, either. It seems to me the relevant question is "is it likely that catching KeyboardInterrupt with 'except Exception:' will get sane behavior from a generic user-defined handler?" I think not; usually you'd like generic error recovery to _not_ bother the user, but KeyboardInterrupt sort of demands interaction with the user, no? So you're going to need a separate routine for KeyboardInterrupt, anyway. I expect that's going to be the normal case. So I would say KeyboardInterrupt should derive from CriticalException, not from Exception. I definitely agree that implementing recovery features is a good idea, and in interactive operation (or with an option to the interpreter), to allow for such recovery in the interpreter itself. For example, the interpreter could keep a small nest egg of memory for the purpose of interacting with the user; this would be harder for a program to do. And in many quickie scripts it would be convenient if the interpreter would drop into interactive mode, not die, if the program encounters a critical exception. But it's still a critical exception to the program written in Python, even if it's easy for the user to handle and the interpreter provides the capability to pass the buck to the user. The program has completely lost control! -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN

On 8/1/05, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Uh, according to your example in Common LISP it is indeed an error,
I think you are referring to the first word of this line: Error: Received signal number 2 (Keyboard interrupt) [condition type: INTERRUPT-SIGNAL] Well, that refers to the fact that it was raised with (error ...). It says nothing about the type of a Keyboad interrupt condition. (The function 'error' vs 'signal' mark the distinction between raising conditions that must be handled otherwise you'll end up in the debugger, and conditions that when not handled are silently ignored.) The CL ANSI standard does not define what kind of condition a Keyboard interrupt is, so the implementations have to make that decision. Although this implementation (Allegro CL) has currently defined it as a subclass of 'error', I'm told it should have been a 'serious-condition' instead ('error' is a subclass of 'serious-condition', which is a subclass of 'condition'), precisely because forms like ignore-errors, like a bare except in Python, will catch it right now when they shouldn't. I assume most of the other Lisp implementations have already defined it as serious-condition. So, in short, Keyboard interrupts in Lisp are a serious-condition, not an error. (And what is labeled CriticalException in this discussion, has in serious-condition Lisp's counterpart.)
and if an unhandled signal whose intended interpretation is "drop the gun and put your hands on your head!" isn't critical, what is?<wink>
Eh, are you serious? <wink>
Well, I'm not opposed to KeyboardInterrupt being in a class that's not a subclass of 'Exception', when the latter is the class used in a bare 'except'. But when CriticalException, despite its name, is not a subclass of Exception, that is a bit strange. I'd prefer the 'condition' and 'error' terminology, and to label a keyboard interrupt a condition, not any kind of exception or error.
I agree with you that it should not be caught in a bare 'except' (or an 'except Exception', when that is equivalent). - Willem

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> So, in short, Keyboard interrupts in Lisp are a Willem> serious-condition, not an error. Willem> (And what is labeled CriticalException in this discussion, Willem> has in serious-condition Lisp's counterpart.) I don't see it that way. Rather, "Raisable" is the closest equivalent to "serious-condition", and "CriticalException" is an intermediate class that has no counterpart in Lisp usage. >> and if an unhandled signal whose intended interpretation is >> "drop the gun and put your hands on your head!" isn't critical, >> what is?<wink> Willem> Eh, are you serious? <wink> Yes. Unhandled, KeyboardInterrupt means that the user has forcibly taken control away from the program without giving it a chance to preserve state, finish responding to (realtime) external conditions, or even activate vacation(1), and the program is entirely at the mercy of the user. Usually, the program then proceeds to die without dignity. If it's a realtime application, killing it is probably the only merciful thing to do. If you were the program, wouldn't you consider that critical? Willem> But when CriticalException, despite its name, is not a Willem> subclass of Exception, that is a bit strange. Granted. It doesn't bother me, but since it bothers both you and Philip Eby, I concede the point; we should find a better name (or not bother with such a class, see below). Willem> I'd prefer the 'condition' and 'error' terminology, and to Willem> label a keyboard interrupt a condition, not any kind of Willem> exception or error. Now, that does bother me.<wink> Anything we will not permit a program to ignore with a bare "except: pass" if it so chooses had better be more serious than merely a "condition". Also, to me a "condition" is something that I poll for, it does not interrupt me. To me, a condition (even a serious one) is precisely the kind of thing that I should be able to ignore with a bare except! Your description of the CL hierarchy makes me wonder if there's any benefit to having a class between Raisable and KeyboardInterrupt. Unlike SystemShutdown or PowerFailure, KeyboardInterrupt does imply presence of a user demanding attention; I suppose that warrants special treatment. On the other hand, I don't see a need for a class whose members share only the property that they are not catchable with a bare except, leading to Raisable -+- Exception +- KeyboardInterrupt +- SystemShutdown +- PowerFailure +- (etc) or even Exception -+- CatchableException +- KeyboardInterrupt +- SystemShutdown +- PowerFailure +- (etc) The latter is my mental model, and would work well with bare excepts. It also would encourage the programmer to think about whether an Exception should be catchable or is a special case, but I don't think that's really helpful except for Python developers, who presumably would be aware of the issues. The former would be a compromise to allow "except Exception" to be a natural idiom, which I prefer to bare excepts on stylistic grounds. On balance, that's what I advocate. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

At 12:25 PM 8/2/2005 +0900, Stephen J. Turnbull wrote:
I don't think that Lisp's idea of an exception hierarchy has much bearing here.
You just said, "Unhandled, KeyboardInterrupt means..." If the program doesn't *want* to handle KeyboardInterrupt, then it obviously *isn't* critical, because it doesn't care. Conversely, if it *does* handle KeyboardInterrupt, then once again, it's not critical by your definition. So, clearly, KeyboardInterrupt is thus *not* critical, and doesn't belong in the CriticalException hierarchy. Note, by the way, that Python programs can disable a KeyboardInterrupt from ever occurring in the first place, whereas none of the other CriticalException classes can be "disabled" because they're actually *error* conditions, while KeyboardInterrupt is just an asynchronous notification - for control flow purposes. Ergo, it's a control flow exception. (Similarly, a Python program can avoid raising any of the other control flow errors; they are by and large optional features.)
On the contrary, it is control-flow exceptions that bare except clauses are most harmful to: StopIteration, SystemExit, and... you guessed it... KeyboardInterrupt. An exception that's being used for control flow is precisely the kind of thing you don't want anything but an explicit except clause to catch. Whether critical errors should also pass bare except clauses is a distinct issue, one which KeyboardInterrupt really doesn't enter into. If you think that a KeyboardInterrupt is an error, then it's an indication that Python's documentation and the current exception class hierarchy has failed to educate you sufficiently, and that we *really* need to add a class like ControlFlowException into the hierarchy to help make sure that other people don't end up sharing your misunderstanding. ;-)

"Phillip" == Phillip J Eby <pje@telecommunity.com> writes:
Phillip> You just said, "Unhandled, KeyboardInterrupt means..." Phillip> If the program doesn't *want* to handle Phillip> KeyboardInterrupt, then it obviously *isn't* critical, Phillip> because it doesn't care. Conversely, if it *does* handle Phillip> KeyboardInterrupt, then once again, it's not critical by Phillip> your definition. That's not my definition. By that argument, no condition that can be handled can be critical. By my definition, the condition only needs to prevent the program from continuing normally when it arises. KeyboardInterrupt is a convention that is used to tell a program that continuing normally is not acceptable behavior, and therefore "critical" by my definition. Under either definition, we'll still need to do something special with MemoryError, KeyboardInterrupt, et amicae, and they still shouldn't be caught by a generic "except Exception". We agree on that, don't we? Phillip> Note, by the way, that Python programs can disable a Phillip> KeyboardInterrupt [...]. Ergo, it's a control flow Phillip> exception. Sure, in some sense---but not in the Python language AFAIK. Which control constructs in the Python language define semantics for continuation after KeyboardInterrupt occurs? Anything that can stop a program but the language doesn't define semantics for continuation is critical and exceptional by my definition. Willem> I'd prefer the 'condition' and 'error' terminology, and to Willem> label a keyboard interrupt a condition, not any kind of Willem> exception or error. >> Now, that does bother me.<wink> [...] Phillip> On the contrary, it is control-flow exceptions that bare Phillip> except clauses are most harmful to: StopIteration, Phillip> SystemExit, and... you guessed it... KeyboardInterrupt. That is a Python semantics issue, but as far as I can see there's unanimity on it. I and (AFAICS) Willem were discussing the connotations of the _names_ at this point, and whether they were suggestive of the semantics we (all!) seem to agree on. I do not find the word "condition" suggestive of the "things 'bare except' should not catch" semantics. I believe enough others will agree with me that the word "condition", even "serious condition", should be avoided. Phillip> An exception that's being used for control flow is Phillip> precisely the kind of thing you don't want anything but Phillip> an explicit except clause to catch. Which is exactly the conclusion I reached: [It] makes me wonder if there's any benefit to having a class [ie, CriticalException] between Raisable and KeyboardInterrupt. ...I don't see a need for a class whose members share only the property that they are not catchable with a bare except.... Now, somebody proposed: Raisable -+- Exception +- ... +- ControlFlowException -+- StopIteration +- KeyboardInterrupt As I wrote above, I see no use for that; I think that's what you're saying too, right? AIUI, you want Raisable -+- Exception +- ... +- StopIteration +- KeyboardInterrupt so that only the appropriate control construct or an explicit except can catch a control flow exception. At least, you've convinced me that "critical exception" is not a concept that should be implemented in the Python language specification. Rather, (for those who think as I do, if there are others<wink>) "critical exception" would be an intuitive guide to a subclass of exceptions that shouldn't be caught by a bare except (or a handler for any superclass except Raisable, for that matter). By the same token, "control flow exception" is a pedagogical concept, not something that should be reified in a ControlFlowException class, right? Phillip> If you think that a KeyboardInterrupt is an error, I have used the word "error" only in quoting Willem, and that's quite deliberate. I don't think that a condition need be an error to be "critical". -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Stephen J. Turnbull wrote:
The use for it is : try: # do stuff except ControlFlowException: raise except Raisable: # handle anything else Sure, you could write it as: try: # do stuff except (CriticalException, Exception, Warning): # handle anything else But the former structure better reflects the programmers intent (handle everything except control flow exceptions). It's a fact that Python uses exceptions for control flow - KeyboardInterrupt [1], StopIteration, SystemExit (and soon to be GeneratorExit as well). Grouping them under a common parent allows them to be dealt with as a group, rather than their names being spelt out explicitly. Actually having this in the exception hierarchy is beneficial from a pedagogical point of view as well - the hierarchy is practically the first thing you encounter when you run "help ('exceptions')" at the interactive prompt. I have a Python 2.5 candidate hierarchy below, which uses dual inheritance to avoid breaking backward compatibility - any existing except clauses will catch all of the exceptions they used to catch. The only new inheritance introduced is to new exceptions, also avoiding backward compatibility problems, as any existing except clauses will let by all of the exceptions they used to let by. There are no removals, but the deprecation process is started in order to change the names of ReferenceError and RuntimeWarning to WeakReferenceError and SemanticsWarning. With this hierarchy, the recommended parent class for application errors becomes Error, and "except Error:" is preferred to any of "except:", "except Exception:" and "except StandardError:" (although these three continue to catch everything they used to catch). The recommended workaround for libraries raising errors which still inherit directly from Exception is: try: # Use library except (ControlFlowException, CriticalError): raise except Exception: # Do stuff (Remove the 'Exception' part if the library is so outdated that it still raises string exceptions) Applications which use exceptions to control the flow of execution rather than to indicate an error (e.g. breaking out of multiple nested loops) are free to use ControlFlowException directly, or else define their own subclasses of ControlFlowException. This hierarchy achieves my main goal for the exception reorganisation, which is to make it easy for scripts and applications to avoid inadvertently swallowing the control flow exceptions and critical errors, while still being able to provide generic error handlers for application faults. (Hmm, the pre-PEP doesn't include that as a goal in the 'Philosophy' section. . .) Python 2.4 Compatible Improved Exception Hierarchy v 0.1 ======================================================== Exception +-- ControlFlowException (new) +-- GeneratorExit (new) +-- StopIteration +-- SystemExit +-- KeyboardInterrupt (dual-inheritance new) +-- StandardError +-- KeyboardInterrupt (dual-inheritance new) +-- CriticalError (new) +-- MemoryError +-- SystemError +-- Error (new) +-- AssertionError +-- AttributeError +-- EOFError +-- ImportError +-- TypeError +-- ReferenceError (deprecated), WeakReferenceError (new alias) +-- ArithmeticError +-- FloatingPointError +-- DivideByZeroError +-- OverflowError +-- EnvironmentError +-- OSError +-- WindowsError +-- IOError +-- LookupError +-- IndexError +-- KeyError +-- NameError +-- UnboundLocalError +-- RuntimeError +-- NotImplementedError +-- SyntaxError +-- IndentationError +-- TabError +-- ValueError +-- UnicodeError +-- UnicodeDecodeError +-- UnicodeEncodeError +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- FutureWarning +-- PendingDeprecationWarning +-- RuntimeWarning (deprecated), SemanticsWarning (new alias) +-- SyntaxWarning +-- UserWarning Cheers, Nick. [1] PJE has convinced me that I was right in thinking that KeyboardInterrupt was a better fit under ControlFlowExceptions than it was under CriticalError. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

Nick Coghlan wrote:
+1. I like this approach of using multiple inheritence to solve the b/w compatibility problem. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Tue, 2005-08-02 at 11:00, Nick Coghlan wrote:
With this hierarchy, the recommended parent class for application errors becomes Error, ...
And presumably Error could also be the recommended exception for quick'n'dirty scripts. Mark Russell

At 08:00 PM 8/2/2005 +1000, Nick Coghlan wrote:
Couldn't we make Error a parent of StandardError, here, and then make the CriticalError subclasses dual-inherit StandardError, i.e.: Error CriticalError MemoryError (also subclass StandardError) SystemError (also subclass StandardError) StandardError ... In this way, we can encourage people to inherit from Error. Or maybe we should just make the primary hierarchy the way we want it to be, and only cross-link exceptions to StandardError that were previously under StandardError, i.e.: Raisable ControlFlowException ... (cross-inherit to StandardError as needed) CriticalError ... (cross-inherit to StandardError as needed) Exception ... This wouldn't avoid "except Exception" and bare except being problems, but at least you can catch the uncatchables and reraise them. Hm. Maybe we should include a Reraisable base for ControlFlowException and CriticalError? Then you could do "except Reraisable: raise" as a nice way to do the right thing until Python 3.0. It seems to me that multiple inheritance is definitely the right idea, though. That way, we can get the hierarchy we really want with only a minimum of boilerplate in pre-3.0 to make it actually work.

On 8/2/05, Phillip J. Eby <pje@telecommunity.com> wrote:
I think that is acceptable. Using multiple inheritance to make sure that the exceptions that have been moved out of the main exception branch seems like it will be the best solution for giving some form of backwards-compatibility for now while allowing things to still move forward and not cripple the changes we want to make.
As in exceptions that don't inherit from Error/StandError/whatever_the_main_exception_is can easily be caught separately?
Yeah. I think name aliasing and multiple inheritance will take us a long way. Warnings should be able to take us the rest of the way. -Brett (who is still waiting for a number; Barry, David, you out there?)

Brett Cannon wrote:
And it will let us get rid of some of the ugliness in my v 0.1 proposal, too (like Error being a child of StandardError, instead of the other way around). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

At 04:13 PM 8/2/2005 +0900, Stephen J. Turnbull wrote:
No, I want ControlFlowException to exist as a parent so that code today can work around the fact that bare "except:" and "except Exception:" catch everything. In Python 3.0, we should have "except Error:" and be able to have it catch everything but control flow exceptions and possibly critical errors.

On Aug 2, 2005, at 12:31 AM, Phillip J. Eby wrote:
No... KeyboardInterrupt (just like other asynchronous exceptions) really should be treated as a critical error. Doing anything other than killing your process off after receiving it is just inviting disaster. Because the exception can have occurred absolutely anywhere, it is unsuitable for normal use. Aborting a function between two arbitrary bytecodes and trying to continue operation is simply a recipe for disaster. For example, in threadable.py between line 200 "saved_state = self._release_save()" and 201 "try: # restore state no matter what (e.g., KeyboardInterrupt)" would be a bad place to hit control-c if you ever need to use that Condition again. This kind of problem is pervasive and unavoidable. If you want to do a clean shutdown on control-c, the only sane way is to install a custom signal handler that doesn't throw an asynchronous exception at you. There's a reason asynchronously killing off threads was deprecated in java. James

At 10:53 AM 8/2/2005 -0400, James Y Knight wrote:
In my personal experience with using KeyboardInterrupt I've only ever needed to do some minor cleanup of external state, such as removing lockfiles, abandoning connections, etc., so I haven't encountered this issue before. I can see, however, why it would be a problem if you were trying to keep the program *running* - but I've been assuming that KeyboardInterrupt is something that always means "attempt to shutdown gracefully". I suppose considering it a critical error might put it more clearly in that category. I'm not 100% convinced, but you've definitely given me something to think about. On the other hand, any exception can happen "between two arbitrary bytecodes", so there are always circumstances that need special attention, or require a "with block_signals" statement or something. I suppose this issue may have to come down to BDFL pronouncement.

On 8/2/05, Stephen J. Turnbull <stephen@xemacs.org> wrote:
That would imply that all raisables are 'serious' in the Lisp sense, which is defined as "all conditions serious enough to require interactive intervention if not handled". Yet Python warnings are raisable (as raisable is the root), but are certainly not serious in the Lisp sense. (This is complicated by that warnings are raised using 'signal'. More below.) Willem:
To clarify myself: a 'serious-condition' in CL stands for "all conditions serious enough to require interactive intervention if not handled"; I meant to label KI a 'serious-condition'. Stephen:
If I understand your position correctly, it is probably not changed yet by the above clarification. <wink> Maybe it will surprise you, that in Lisp a bare except (ignore-errors) does not catch non-serious things like warnings. And if left uncatched, a warning leaks out to the top level, gets printed and subsequently ignored. That's because non-serious conditions are (usually) raised using 'signal', not 'error'. The default top-level warnings handler just prints it, but does not influence the program control flow, so the execution resumes just after the (warn ..) form. This probably marks a very important difference between Python and CL. I think one could say that where in Python one would use a bare except to catch both non-serious and serious exceptions, in CL one normally doesn't bother with catching the non-serious ones because they will not create havoc at an outer level anyway. So in Python a warning must be catched by a bare except, while in Lisp it would not. And from this follow different contraints on the hierarchy. By the way, this is the condition hierarchy in Allegro CL (most of which is prescribed by the ANSI standard): <http://www.franz.com/support/documentation/7.0/doc/errors.htm> - Willem

"Willem" == Willem Broekema <metawilm@gmail.com> writes:
Willem> On 8/2/05, Stephen J. Turnbull <stephen@xemacs.org> wrote: >> I don't see it that way. Rather, "Raisable" is the closest >> equivalent to "serious-condition", and "CriticalException" is >> an intermediate class that has no counterpart in Lisp usage. Willem> That would imply that all raisables are 'serious' in the Willem> Lisp sense, No, it implies that Phillip was right when he wrote that the Lisp hierarchy of signals is not relevant (as a whole) to the discussion of Python Raisables. Of course partial analogies are useful. In any case, Nick's idiom of "except ControlFlowException: raise" clarified everything for me. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Jul 30, 2005, at 8:57 PM, Nick Coghlan wrote:
It seems to me that it *increases* the pain of transition. I do not see any reason why adding a new class above the current hierarchy causes any problems. However, changing the recommended base class for user-defined exceptions from "Exception" to "Error" will cause problems for *every* library. Moving KeyboardInterrupt, MemoryError, and SystemError out from under Exception will be a backwards compatibility issue, but that's the case in all proposals. Additionally, I predict the pain from doing that will be minor. In cases where apps want to catch *all* exceptions, they must already use "except:" instead of "except Exception:", as not all exceptions derive from Exception. And catching the above three in any circumstance other than when explicitly mentioned and when wanting to catch every exception is probably wrong. So moving them will probably not cause many programs to malfunction. I think the following could be done in Py2.5: a) add Raisable above Exception b) move KeyboardInterrupt, MemoryError, and SystemError somewhere under Raisable but not under Exception c) pending deprecation warning for "except:" and raising objects that aren't subclasses of "Raisable". d) (not really related, but while I'm at it...) Allow exceptions derived from object (with Raisable too, of course) For Py2.6: a) change pending deprecation warning above to deprecation warning. James

At 12:12 PM 7/31/2005 -0400, James Y Knight wrote:
I think you're ignoring the part where most exception handlers are already broken. At least adding CriticalException and ControlFlowException makes it possible to add this: try: ... except (CriticalException,ControlFlowException): raise except: ... This isn't great, I admit, but at least it would actually *work*. I also don't see how changing the recommended base class from Exception to Error causes *problems* for every library. Sure, it forces them to move (eventually!), but it's a trivial change, and makes it *possible* to do the right thing with exceptions (e.g. except Error:) as soon as all the libraries you depend on have moved to using Error.
Actually, in my tweak of Nick's proposal they're still Exception subclasses, so nothing breaks.
In which case, maybe we should just implement Nick's full proposal, since the only thing that it would break is code that uses "except Exception:" to catch SystemExit or StopIteration.

On Jul 31, 2005, at 12:49 PM, Phillip J. Eby wrote:
Exactly. That is the problem. Adding a new class above Exception in the hierarchy allows everything to work nicely *now*. Recommended practice has been to have exceptions derive from Exception for a looong time. Changing everybody now will take approximately forever, which means the Error class is pretty much useless. By keeping the definition of Exception as "the standard thing you should derive from and catch", and adding a superclass with things you shouldn't catch, you make conversion a lot simpler. If you're not worried about compatibility with ye olde string exceptions, you can start using "except Exception" immediately. If you are, you can do as your example above. And when Python v.Future comes around, "except Exception" will be the only reasonable thing to do. If, on the other hand, we use Exception as the base class and Error as the thing you should use, I predict that even by the time Python v.Future comes out, many libraries/prgrams will still have exceptions deriving from Exception, thus making the Exception/Error distinction somewhat broken. James

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Nick, are you going go start subbing in for Tim when he is busy and take my work that I spent hours on, come up with an alternative that took 10 minutes, and have everyone end up loving your newfangled idea 10x more than my original? =)
As I have said I am fine with moving ControlFlowException out to here, but it depends if others agree.
Good point with SystemExit! I am definitely +1 on changing that inheritance.
I am still up for moving AttributeError, but with the amount of arguing going on between where it should go I am not going to be shocked if we go with the status quo.
If we are going to lack exceptions for other OSs, I say remove it. No reason Windows should get special treatment with a built-in exception.
What about UnboundGlobalError?
+-- RuntimeError
I still don't like the name.
+-- NotImplementedError
Interesting idea, but I don't view them as the same. Everyone uses RuntimeError as a Poor Man's Exception, not as an actual runtime error.
Don't like the idea of having DeprecationWarning inherit from PendingDeprecationWarning? -Brett

Brett Cannon wrote:
It's like editing creative writing - I find it far, far easier to take something good and tweak it to make it better, than to come up with something good in the first place ;)
Exactly my thought. I had it under "NameError" for a while, but had difficulty coming up with a case where lumping it in with the lexical scoping errors was actually beneficial. Eventually, the fact that it is easy to add another exception to an except clause, but hard to remove a child you don't want that inherits from a parent you do want persauded me to leave this one alone.
True. And the interface looks the same as the normal OSError interface, so it should be possible to replace all uses with a basic OSError.
I realised this was a misnomer. A failed name lookup actually means: - the name was not in locals - the name was not in any lexically containing scope - the name was not in the module globals - the name was not in builtins The program doesn't know which of those namespaces was *meant* to contain the name - it only knows that none of them actually contained it. This criticism also applies to the current wording of the NameError text used in this situation (which implies the name should have been in the module globals). Now, a case could possibly be made for separate errors for cases like the following: def f(): global x print x # UnboundGlobalError (currently NameError, with usual text) def f(): def g(): print x g() # UnboundFreeVariableError (currently NameError, with specific text) x = 1 Like UnboundLocalError, in both of these cases, the name is potentially known to the compiler - it simply hasn't been bound to anything yet.
+-- RuntimeError
I still don't like the name.
I'm not that fond of it either - but as the builtin exception most likely to be used (abused?) by user code, I expect changing its name would be more pain than it's worth.
This particular inheritance is copied from Python 2.4 :) I find the main trick with making RuntimeError more palatable is to avoid thinking of 'runtime' in the sense of 'Python runtime', as in 'Python VM'. That's not what this error is about - a Python VM error is a SystemError. Instead, a RuntimeError is a generic application error - something which happened during happened at program runtime. Renaming RuntimeWarning to SemanticsWarning should help with that distinction.
Not really. Mainly because I'm not sure which way the inheritance should go - I could understand someone wanting to suppress PendingDeprecationWarnings without suppressing DeprecationWarnings. On the other hand, there's your argument that a 'DeprecationWarning is just a PendingDeprecationWarning with a shorter timeframe'. Again, I fell back on the concept that Python's except clause makes it easy to ask for both if you want both, but makes it relatively difficult to undo exception inheritance if it isn't what you want. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
You stand on the shoulder of a giant (and I am only partially kidding; people who have met me will get the joke)!
I think this one will require BDFL pronouncement to be moved since this already looks like a contested idea. And I don't think Guido is going to care enough to want to see it moved.
Glad you agree. =)
That was what I was thinking as the use case for UnboundGlobalError. Also, if you read the docs on NameError, it explicitly states that it is for global names that are not found. I am willing to toss in an exception for the failure to find a free variable (although I would call it UnboundFreeError) to flesh this namespace hierarchy out. Then again you could argue you should inherit from each other since a missing local is kind of lack missing the global namespace as well, but that kind of purity just doesn't work for me in this case. [rest of my response to this email is forthcoming] -Brett

On 7/30/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Maybe, but I am still going to nudge for a change. It might get shot down in the end, but at least I can say I tried.
Ah, oops. =) Well, I still don't think they should inherit like that and instead be decoupled.
Right, you shouldn't think that, but isn't that what you thought initially? That is why I want to rename RuntimeError.
Renaming RuntimeWarning to SemanticsWarning should help with that distinction.
Well, not once the name is changed since people won't know about the former name connection. =)
True, but the hierarchy should still properly reflect increasing severity in my opinion. I am going to push for this; we will see if I get pushed back enough to not do it. -Brett

On Saturday 30 July 2005 22:20, Brett Cannon wrote:
I have no idea what you mean by "properly reflect increasing severity". Should the more severe or less severe case be derived from the other? I doubt there is a single answer that we can readily agree on. If DeprecationWarning and PendingDeprecationWarning need a class-hierarchy relationship (and I think they do; it *is* reasonable for someone to deal with them generically if they can reasonably want to deal with either), then it seems they need a common base: +--AnyDeprecationWarning +--DeprecationWarning +--PendingDeprecationWarning -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

Nick Coghlan wrote:
If CriticalException and ControlFlowException are to be siblings of Exception rather than subclasses of it, they should be renamed so that they don't end with "Exception". Otherwise there will be a confusing mismatch between the actual inheritance hierarchy and the one suggested by the naming. Also, I'm not entirely happy about Exception no longer being at the top, because so far the word "exception" in relation to Python has invariably meant "anything that can be raised". This terminology is even embedded in the syntax with the try-except statement. Changing this could to lead to some awkward circumlocutions in the documentation and confusion in discussions. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

On 7/30/05, M.-A. Lemburg <mal@egenix.com> wrote:
Actually, it was referencing the bare 'except' clauses and changing their semantics.
Do you mean all inherit from Exception in order for it be allowed to be passed to 'raise'?
So that's a bit inverse of what you've obviously expected :-)
If this becomes a theme, yes. But specifically coming from you, MAL, no, not really. =)
My view of Python 3.0 was that backwards-compatibility would not be a gimme in anyway. I personally am willing to break stuff in the name of clarity, which is the point of this whole endeavour. While I am willing to back off on some the proposed changes, I do think the basic spirit of it is correct.
Right, so the renaming is not a huge problem.
Right, but if we introduce aliases in Python 2.9 or sooner the transition will be much easier and they will know to rename things. And obviously warnings can be used for people to know that the hierarchy will change. I bet we will put in a PendingDeprecationWarning or SemanticsWarning saying that the inheritance will be different in Python 3.0 . Yes, this will incur more work than had we left it alone, but the whole point of Python 3.0 is to remove quirks.
If we fiddle with the exception code raising a Warning will not be too bad for this. Plus you can grep for PyExc_TypeError easily enough to see where you are using classes that have a new inheritance tree.
I for one don't expect Python 2.x code to run automatically without some fiddling. I think saying from scratch is a little strong.
Lot's of things "just work", but it doesn't mean they can't be improved upon. But it looks like some of my suggestions were overreaching so they will most likely get scaled back. I will also add a section to the PEP discussing how a transition from Python 2.x to Python 3.0 in this regard can be handled. -Brett

On Sat, Jul 30, 2005, Brett Cannon wrote:
My take is that for Python 3.0, backwards compatibility is no longer a critical priority -- but any breakage still needs to be argued for and balanced. We want to avoid unnecessary breakage. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.
participants (14)
-
Aahz
-
Brett Cannon
-
Brian Beck
-
Fred L. Drake, Jr.
-
Greg Ewing
-
Guido van Rossum
-
James Y Knight
-
M.-A. Lemburg
-
Mark Russell
-
Michael Hudson
-
Nick Coghlan
-
Phillip J. Eby
-
Stephen J. Turnbull
-
Willem Broekema