[Python-Dev] PEP, take 2: Exception Reorganization for Python 3.0

[Brett Cannon]

OK, having taken in all of the suggestions, here is another revision
round.  I think I still have a place or two I partially ignored people
just because there was not a severe uproar and I still think the
original idea is good (renaming RuntimeError, for instance).  I also
added notes on handling the transition and rejected idea.

There is now only one open issue, which is whether
ControlFlowException should be removed.

And I am still waiting on a PEP number to be able to check this into
CVS and push me to flesh out the references.  =)


--------------------------------------------------------------

PEP: XXX
Title: Exception Reorganization for Python 3.0
Version: $Revision: 1.5 $
Last-Modified: $Date: 2005/06/07 13:17:37 $
Author: Brett Cannon <brett at python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Jul-2005
Post-History: XX-XXX-XXX

.. contents::


Abstract
========

Python, as of version 2.4, has 38 exceptions (including warnings) in
the built-in namespace in a rather shallow hierarchy.
This list of classes has grown over the years without a chance to
learn from mistakes and cleaning up the hierarchy.
This PEP proposes doing a reorganization for Python 3.0 when
backwards-compatibility is not an issue.
Along with this reorganization, adding a requirement that all objects passed to
a ``raise`` statement must inherit from a specific superclass is proposed.
Lastly, the removal of bare ``except`` class is suggested.


Rationale
=========

Exceptions are a critical part of Python.
While exceptions are traditionally used to signal errors in a program,
they have also grown to be used for flow control for things such as
iterators.
There importance is great.

But the organization of the exception hierarchy is suboptimal.
Mostly for backwards-compatibility reasons, the hierarchy has stayed
very flat and old exceptions who usefulness have not been proven have
been left in.
Making exceptions more hierarchical would help facilitate exception
handling by making catching exceptions using inheritance much more
logical.
This should also help lead to less errors from being too broad in what
exceptions are caught in an ``except`` clause.

A required superclass for all exceptions is also being proposed
[Summary2004-08-01]_.
By requiring any object that is used in a ``raise`` statement to
inherit from a specific superclass, certain attributes (such as those
laid out in PEP 344 [PEP344]_) can be guaranteed to exist.
This also will lead to the planned removal of string exceptions.

Lastly, bare ``except`` clauses are to be removed [XXX Guido's reply to my
initial draft]_.
Often people use a bare ``except`` when what they really wanted were
non-critical exceptions to be caught while more system-specific ones,
such as MemoryError, to pass through and to halt the interpreter.
This leads to errors that can be hard to debug thanks to exceptions' sometimes
unpredictable execution flow.
It also causes ``except`` statements to follow the "explicit is better than
implicit" tenant of Python [XXX]_.


Philosophy of Reorganization
============================

There are several goals in this reorganization that defined the
philosophy used to guide the work.
One goal was to prune out unneeded exceptions.
Extraneous exceptions should not be left in since it just serves to
clutter the built-in namespace.
Unneeded exceptions also dilute the importance of other exceptions by
splitting uses between several exceptions when all uses should have
been under a single exception.

Another goal was to introduce any exceptions that were deemed needed
to fill any holes in the hierarchy.
Most new exceptions were done to flesh out the inheritance hierarchy
to make it easier to catch a category of exceptions with a simpler
``except`` clause.

Changing inheritance to make it more reasonable was a goal.
As stated above, having proper inheritance allows for more accurate
``except`` statements when catching exceptions based on the
inheritance tree.

Lastly, any renaming to make an exception's use more obvious from its
name was done.
Having to look up what an exception is meant to be used for because
the name does not proper reflect its usage is annoying and slows down
debugging.
Having a proper name also makes debugging easier on new programmers.
But for simplicity of existing user's and for transitioning to Python
3.0, only exceptions whose names were fairly out of alignment with
their stated purpose have been renamed.

New Hierarchy
=============

Exception
+-- CriticalException (new)
    +-- KeyboardInterrupt
    +-- MemoryError
    +-- SystemError
+-- ControlFlowException (new)
    +-- StopIteration
    +-- GeneratorExit
    +-- SystemExit
+-- StandardError
    +-- AssertionError
    +-- SyntaxError
        +-- IndentationError
            +-- TabError
    +-- UserException (rename of RuntimeError)
    +-- ArithmeticError
        +-- FloatingPointError
        +-- DivideByZeroError
        +-- OverflowError
    +-- UnicodeError
        +-- UnicodeDecodeError
        +-- UnicodeEncodeError
        +-- UnicodeTranslateError
    +-- LookupError
        +-- IndexError
        +-- KeyError
    +-- TypeError
    +-- AttributeError
    +-- EnvironmentError
	+-- OSError
	+-- IOError
	    +-- EOFError (new inheritance)
    +-- ImportError
    +-- NotImplementedError (new inheritance)
    +-- NamespaceError (rename of NameError)
        +-- UnboundGlobalError (new)
        +-- UnboundLocalError
	+-- UnboundFreeError (new)
    +-- WeakReferenceError (rename of ReferenceError)
    +-- ValueError
+-- Warning
    +-- UserWarning
    +-- AnyDeprecationWarning (new)
	+-- PendingDeprecationWarning 
        +-- DeprecationWarning
    +-- SyntaxWarning
    +-- SemanticsWarning (rename of RuntimeWarning)
    +-- FutureWarning


Differences Compared to Python 2.4
==================================

Changes to exceptions from Python 2.4 can take shape in three forms:
removal, renaming, or change in their superclass.
There are also new exceptions introduced in the proposed hierarchy.


New Exceptions
--------------

CriticalException
'''''''''''''''''

The superclass for exceptions for which a severe error has occurred that one
would not want to recover from.
The name is meant to reflect the point that these exceptions are
usually raised only when the interpreter should most likely be
terminated.
All classes that inherit from this class are raised when the virtual machine
has a asynchronous exception to raise about its state.


ControlFlowException
''''''''''''''''''''

This exception exists as a superclass for all exceptions that directly deal
with control flow.
Inheriting from Exception instead of StandardError 
prevents them from being caught accidently when one wants to catch errors.
The name, by not mentioning "Error", does not lead to one to confuse
the subclasses as errors.


UnboundGlobalError
''''''''''''''''''

Raised when a global variable was not found.


UnboundFreeError
''''''''''''''''

Raised when a free variable is not found.


AnyDeprecationWarning
'''''''''''''''''''''

A common superclass for all deprecation-related exceptions.
While having DeprecationWarning inherit from PendingDeprecationWarning was
suggested because a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is happening now, the logic was not agreed upon
by a majority.
But since the exceptions are related, creating a common superclass is
warranted.


Removed Exceptions
------------------

WindowsError
''''''''''''

Too OS-specific to be kept in the built-in exception hierarchy.


Renamed Exceptions
------------------

RuntimeError
''''''''''''

Renamed UserException.

Meant for use as a generic exception to be used when one does not want to
create a new exception class but do not want to raise an exception that might
be caught based on inheritance, RuntimeError is poorly named.
It's name in Python 2.4 seems to suggest an error that occurred at runtime,
possibly an error in the VM.
Renaming the exception to UserException more clearly states the purpose for
the exception as quick-and-dirty exception for the user to use.
The name also keeps it in line with UserWarning.


ReferenceError
''''''''''''''

Renamed WeakReferenceError.

ReferenceError was added to the built-in exception hierarchy in Python
2.2 [exceptionsmodule]_.
Taken directly from the ``weakref`` module, its name comes directly
from its original name when it resided in the module.
Unfortunately its name does not suggest its connection to weak
references and thus deserves a renaming.


NameError
'''''''''

Renamed NamespaceError.

While NameError suggests its common use, it is not entirely apparent.
Making it more of a superclass for namespace-related exceptions warrants a
renaming to make it abundantly clear its use.
Plus the documentation of the exception module[XXX]_ states that it is
actually meant for global names and not for just any exception.


RuntimeWarning
''''''''''''''

Renamed SemanticsWarning.

RuntimeWarning is to represent semantic changes coming in the future.
But while saying that affects "runtime" is true, flat-out stating it
is a semantic change is much clearer, eliminating any possible
association of "runtime" with the virtual machine specifically.


Changed Inheritance
-------------------

AttributeError
''''''''''''''

Inherits from StandardError.

Originally inheriting from NotImplementedError, AttributeError is typically
raised because of the lack of an attribute which does not necessarily mean it
was not implemented but just not set yet.
Thus it has been decoupled from NotImplementedError.


EOFError
''''''''

Subclasses IOError.

Since an EOF comes from I/O it only makes sense that it be considered
an I/O error.


Required Superclass for ``raise``
=================================

By requiring all objects passed to a ``raise`` statement inherit from
a specific superclass, one is guaranteed that all exceptions will have
certain attributes.
If PEP 342 [PEP344]_ is accepted, the attributes outlined there will
be guaranteed to be on all exceptions raised.
This should help facilitate debugging by making the querying of
information from exceptions much easier.

The proposed hierarchy has Exception as the required class that one
must inherit from.


Implementation
--------------

Enforcement is straight-forward.
Modifying ``RAISE_VARARGS`` to do an inheritance check first before raising
an exception should be enough.  For the C API, all functions that set an
exception will have the same inheritance check.


Removal of Bare ``except`` Clauses
==================================

One of Python's basic tenants is "explicit is better than implicit".
Unfortunately a bare ``except`` clause implicitly states it should
catch all exceptions.
While useful as a way to catch all exceptions when any object can be
raised, requiring a specific superclass be inherited in order to raise
an object gives a single class to catch to cover all exceptions.
With this in mind, the removal of bare ``except`` statements is justified.


Implementation
--------------

A simple change to the grammar is all that is needed for implementation.


Transition Plan
===============

Exception Hierarchy Changes
---------------------------

New Exceptions
''''''''''''''

New exceptions can simply be added to the built-in namespace.
Any pre-existing objects with the same name will mask the new exceptions,
preserving backwards-compatibility.


Renamed Exceptions
''''''''''''''''''

Renamed exceptions will directly subclass the new names.
When the old exceptions are instantiated (which occurs when an exception is
caught, either by a ``try`` statement or by propagating to the top of the
execution stack), a PendingDeprecationWarning will be raised.

This should properly preserve backwards-compatibility as old usage won't change
and the new names can be used to also catch exceptions using the old name.
The warning of the deprecation is also kept simple.


New Inheritance for Old Exceptions
''''''''''''''''''''''''''''''''''

Using multiple inheritance to our advantage, exceptions whose inheritance has
changed in such a way as for them to not necessarily be caught by pre-existing
``except`` clauses can be made backwards-compatible.
By inheriting from both the new superclasses as well as the original
superclasses existing ``except`` clauses will continue to work as before while
allowing the new inheritance to be used for new clauses.

A PendingDeprecationWarning will be raised based on whether the bytecode
``COMPARE_OP(10)`` results in an exception being caught that would not have
under the new hierarchy.  This will require hard-coding in the implementation
of the bytecode.


Removed Exceptions
''''''''''''''''''

Exceptions scheduled for removal will be transitioned much like the old names
of renamed exceptions.
Upon instantiation a PendingDeprecationWarning will be raised stating the the
exception is due to be removed by Python 3.0 .


Required Superclass for ``raise``
---------------------------------

A SemanticsWarning will be raised when an object is passed to ``raise`` that
does not have the proper inheritance.


Removal of Bare ``except`` Clauses
----------------------------------

A PendingDeprecationWarning will be raised when a bare ``except`` clause is
used.


Rejected Ideas
==============

Threads on python-dev discussing this PEP can be found at [XXX]_.


KeyboardInterrupt inheriting from ControlFlowException
------------------------------------------------------

KeyboardInterrupt has been a contentious point within this hierarchy.
Some view the exception as more control flow being caused by the user.
But with its asynchronous cause thanks to the user being able to trigger the
exception at any point in code it has a more proper place inheriting from
CriticalException.  It also keeps the name of the exception from being
"CriticalError".


Renaming Exception to Raisable, StandardError to Exception
----------------------------------------------------------

While the naming makes sense and emphasizes the required superclass as what
must be inherited from for raising an object, the naming is not required.
Keeping the existing names minimizes code change to use the new names.


DeprecationWarning Inheriting From PendingDeprecationWarning
------------------------------------------------------------

Originally proposed because a DeprecationWarning can be viewed as a
PendingDeprecationWarning that is being removed in the next version.
But enough people thought the inheritance could logically work the other way
the idea was dropped.


AttributeError Inheriting From TypeError or NameError
-----------------------------------------------------

Viewing attributes as part of the interface of a type caused the idea of
inheriting from TypeError.
But that partially defeats the thinking of duck typing and thus was dropped.

Inheriting from NameError was suggested because objects can be viewed as having
their own namespace that the attributes lived in and when they are not found it
is a namespace failure.  This was also dropped as a possibility since not
everyone shared this view.


Removal of EnvironmentError
---------------------------

Originally proposed based on the idea that EnvironmentError was an unneeded
distinction, the BDFL overruled this idea [XXX]_.


Introduction of MacError and UnixError
--------------------------------------

Proposed to add symmetry to WindowsError, the BDFL said they won't be used
enough [XXX]_.  The idea of then removing WindowsError was proposed and
accepted as reasonable, thus completely negating the idea of adding these
exceptions.


SystemError Subclassing SystemExit
----------------------------------

Proposed because a SystemError is meant to lead to a system exit, the idea was
removed since CriticalException signifies this better.


ControlFlowException Under StandardError
----------------------------------------

It has been suggested that ControlFlowException inherit from StandardError.
This idea has been rejected based on the thinking that control flow exceptions
are typically not desired to be caught in a generic fashion as StandardError
will usually be used.


Open Issues
===========

Remove ControlFlowException?
----------------------------

It has been suggested that ControlFlowException is not needed.
Since the desire to catch any control flow exception will be atypical, the
suggestion is to just remove the exception and let the exceptions that
inherited from it inherit directly from Exception.  This still preserves the
seperation from StandardError which is one of the driving factors behind the
introduction of the exception.

Acknowledgements
================

Thanks to Robert Brewer, Josiah Carlson, Nick Coghlan, Timothy
Delaney, Jack Diedrich, Fred L. Drake, Jr., Philip J. Eby, Greg Ewing,
James Y. Knight, MA Lemburg, Guido van Rossum, Stephen J. Turnbull and
everyone else I missed for participating in the discussion.


References
==========

.. [PEP342] PEP 342 (Coroutines via Enhanced Generators)
            (http://www.python.org/peps/pep-0342.html)

.. [PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks)
            (http://www.python.org/peps/pep-0344.html)

.. [exceptionsmodules] 'exceptions' module
            (http://docs.python.org/lib/module-exceptions.html)

.. [Summary2004-08-01] python-dev Summary (An exception is an
exception, unless it doesn't inherit from Exception)
            (http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-is-an-exception-unless-it-doesn-t-inherit-from-exception)

.. [Summary2004-09-01] python-dev Summary (Cleaning the Exception House)
            (http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-exception-house)

.. [python-dev1] python-dev email (Exception hierarchy)
            (http://mail.python.org/pipermail/python-dev/2004-August/047908.html)

.. [python-dev2] python-dev email (Dangerous exceptions)
            (http://mail.python.org/pipermail/python-dev/2004-September/048681.html)


Copyright
=========

This document has been placed in the public domain.