[IronPython] New proposed exception model for IronPython

Dino Viehland dinov at exchange.microsoft.com
Thu Dec 15 23:54:11 CET 2005

One of the changes we're interesting in making for the next release of IronPython is a new exception model that will provide us with a better story for Python, CLI, and developers working in both worlds.  The document below describes the changes (and we've also posted this to our Wiki: http://channel9.msdn.com/wiki/default.aspx/IronPython.ExceptionModel)

The New IronPython Exception Model

With our next release we're planning on offering a new exception model that attempts to fix the problems with our current model.  This document discusses the problems with the current model and the current thinking for the new model.
One World or where we are. 

In IronPython's current exception implementation we have a unified exception type hierarchy.  The Exception class that is visible Python programmers is an instance of PythonException.  PythonException in turn derives from System.Exception and below PythonException there is the standard Python exception hierarchy.  This results in a couple of perceptible differences between IronPython and CPython.  

There's a small difference in that our current exception hierarchy does not start at "Exception".  That's not likely to break anyone anytime soon.  There's a more significant difference in that our exceptions currently are built-in types that can't have arbitrary attributes added.  And there's also a difference in that we are currently limited in what Python programmers can throw; in CPython you can throw any object but we currently only allow you to throw Exceptions.

All of those aren't very big problems and seem like they could mostly be tolerated or at the very least we could add some small workarounds to accommodate adding arbitrary attributes, hiding Exceptions base, or simply throwing arbitrary objects (as the CLI does allow this).  But there are larger issues of what our interop story looks like with the rest of the CLI world.  Before we look at where we're going let's look at what the interop story looks like today.

First let's look at what it means to raise an exception from IronPython.  When we encounter the raise statement we'll end up calling one of two IronPython's Ops.Raise method.  Ultimately this method will either throw the exception.  If the user passes in a pre-created instance (raise Exception()) we'll just do throw of that object.  If the user passed in a type we'll go ahead and allocate the instance of that type and throw that.  And as a special case today we allow throwing strings which we wrap up specially.

This exception is thrown using the CLI's normal exception handling process.  If user code has a try/except statement that handles the exception it will be caught and handled within the Python world.  If running at the interpreter it will be caught by the interpreter and displayed to the user.  And if called from some arbitrary CLI code we'll leak the exception from Python into the CLI code as a "PythonException", "PythonValueError", "PythonTypeError", etc.  

Now let's look at the interop story from the Python side.  If you called some CLI code directly or if we failed to catch an error condition coming from some CLI code we call you'd see an unwrapped CLI exception.  For example you might see an ArgumentNullException.  But you'd only see it if you had imported System and explicitly done "except System.Exception:"!  Otherwise there was no way for you to catch these exceptions.  

Obviously this isn't a very good story.  If you're a CLI developer you're seeing Python exceptions that you don't know how to handle leak into your code.  If you're a Python developer you're seeing CLI exceptions leak into your code.  Only if you're living in a pure Python world does the story even begin to work - and only with the caveats previously mentioned.

Two Worlds or where we're going. 

What we want to achieve is a model where Python and CLI see a unified exception model.  At the same time we want each environment to see its own exception model to its full fidelity.  

The proposed exception model is one where we break apart the existing type hierarchy into two separate hierarchies.  Under this model the Python Exception hierarchy will start at a base class "Exception" which will be an old-style class.   There will be additional old-style classes that inherit from this (e.g. StandardError and further down ValueError) that fill out the rest of the exception type hierarchy.  Because the Python exception hierarchy is represented by old-style classes these will never actually be thrown by IronPython.  Because we keep the current Python exception hierarchy unchanged we'll have all the standard old-style Python classes.  If you write pure Python code with no reference to CLS libraries you'll see exactly the current Python exception system.

On the other side of the world we have the CLI exception hierarchy.  These are the exceptions that are understood by all languages that target the CLS.  In order to maximize interoperability with other languages, IronPython will only throw these standard CLS exception objects.  This means that other CLS languages will only see the exceptions they understand.  Where the CLS exception hierarchy is not rich enough to capture the Python exception hierarchy we will derive new classes from the appropriate location in the CLS hierarchy.

When Worlds Collide.

So now if you're a Python developer you see the Python exception hierarchy and that's all you see and life is wonderful.  Likewise if you're a CLI developer calling Python code see exceptions that are just like you'd expect from any other CLI code.  But what happens when the two worlds collide?  This starts to happen when you have Python code that is directly calling other CLI code.

The core of this is: we respect except [expression, .].  If expression is a Python exception that is the exception we will catch and give you.  If expression is a CLI exception that is also the exception we will catch and give you.  That's great if you're working with some framework and want to catch a specific exception it throws.

A good example of this would be calling a file I/O API.  Previously we've had bug reports where users would get a FileIOException instead of the more Python friendly IOError.  Under the new model even if a CLI library throws a FileIOException or one of its subclasses the Python programmer will automatically see the IOError.  Another similar example was when we failed to trap a bad argument to the intern function.  Here we would throw ArgumentNullException but now this automatically gets translated into a TypeError.

Likewise we have a similar problem with Python code raises an exception.  For example if Python code wanted to raise an EOFError CLS code would previously had no idea how to process this.  Under this proposal CLS code will instead receive the EndOfStreamException is is expecting.

In general, Python exceptions and CLI exceptions should remain nicely separate because there are no overlapping names so it is always clear which exception you want to work with.  The one potential troublemaker is Exception.  If you do "from System import *" then exception suddenly has a new value.  Therefore all of your exception Exception . statements will now behave differently - you'll end up catching CLS exceptions instead of Python exceptions.

The one other tricky case is what should IronPython print when dumping a stack trace for an unhandled exception.  In this case we have no clues to what world the user would like to see.  Our current thinking is that we should display both sets of information and allow this to be user configurable.  Another thought has been allowing a user-defined function that can be overridden to customize this functionality at runtime.

We'd love to hear what you think about the proposed change.

Implementation Details

The start of this is changing the exceptions that IronPython throws.  Currently we throw our PythonException subclasses and now we'll throw CLI classes.  For example instead of throwing a ValueError exception we'll throw an ArgumentException now.  If a user calls raise we will likewise translate this into the closest CLI type and store the Python exception in the Data field of the exception.

The next change is in how IronPython catches exceptions.  When an exception is caught the translation must be done.  If the exception was raised from Python code the translation is easy - we merely need to extract the Python exception from the Data property.  If the exception originated in CLI code (including the IronPython runtime) the exception will go through translation table.  Ultimately we'll create a new instance of the Python type that will be visible to the Python code.  

If this exception is re-thrown then the original exception is pulled from the Python object.  The original exception is also available for Python code to access if it wants to get additional information about the exception.  This original exception value is also used in the event that the exception gets re-raised; it will be the exception value that gets thrown via the CLI.

Here's the full proposed exception hierarchy mapping with a couple of notes on ones we're still thinking about.  Some of these exceptions map nicely to their CLS equivalents and in those cases we'll use the CLS exceptions.  In other cases CLS has no equivalent exceptions and in those cases we'll define custom exceptions inside of IronpPython.

Exception					System.Exception
    SystemExit				IP.O.SystemExit
    StopIteration				System.InvalidOperationException subtype
    StandardError				System.SystemException
        KeyboardInterrupt		IP.O.KeyboardInterruptException
        ImportError			IP.O.PythonImportError
        EnvironmentError		IP.O.PythonEnvironmentError
            IOError			System.IO.IOException
            OSError			S.R.InteropServices.ExternalException (investigate where OSError comes from)
                WindowsError		System.ComponentModel.Win32Exception
        EOFError				System.IO.EndOfStreamException
        RuntimeError			IP.O.RuntimeException
            NotImplementedError	System.NotImplementedException
        NameError				IP.O.NameException
            UnboundLocalError		IP.O.UnboundLocalException
        AttributeError			System.MissingMemberException
        SyntaxError			IP.O.SyntaxErrorException (System.Data has something close)
            IndentationError		IP.O.IndentationErrorException
            TabError			IP.O.TabErrorException
        TypeError				ArgumentTypeException
        AssertionError			IP.O.AssertionException
        LookupError			IP.O.LookupException
            IndexError			System.IndexOutOfRangeException
            KeyError			S.C.G.KeyNotFoundException
        ArithmeticError			System.ArithmeticException
            OverflowError		System.OverflowException
            ZeroDivisionError		System.DivideByZeroException
            FloatingPointError	IP.O.PythonFloatingPointError
        ValueError			ArgumentException
            UnicodeError		IP.O.UnicodeException
                UnicodeEncodeError	System.Text.EncoderFallbackException
                UnicodeDecodeError	System.Text.DecoderFallbackException
                UnicodeTranslateError IP.O.UnicodeTranslateException
        ReferenceError			IP.O.ReferenceException
        SystemError			IP.O.PythonSystemError
        MemoryError			System.OutOfMemoryException
    Warning					System.ComponentModel.WarningException
        UserWarning			IP.O.PythonUserWarning
        DeprecationWarning		IP.O.PythonDeprecationWarning
        PendingDeprecationWarning	IP.O.PythonPendingDeprecationWarning
        SyntaxWarning			IP.O.PythonSyntaxWarning
        OverflowWarning			IP.O.PythonOverflowWarning
        RuntimeWarning			IP.O.PythonRuntimeWarning
        FutureWarning			IP.O.PythonFutureWarning

More information about the Ironpython-users mailing list