Exceptions *must*? be old-style classes?

Phillip J. Eby wrote (in http://mail.python.org/pipermail/python-dev/2005-January/050854.html)
The base of the Exception hierarchy happens to be a classic class. But why are they "required" to be classic? More to the point, is this a bug, a missing feature, or just a bug in the documentation for not mentioning the restriction? You can inherit from both Exception and object. (Though it turns out you can't raise the result.) My first try with google failed to produce an explanation -- and I'm still not sure I understand, beyond "it doesn't happen to work at the moment." Neither the documentation nor the tutorial mention this restriction. http://docs.python.org/lib/module-exceptions.html http://docs.python.org/tut/node10.html#SECTION0010500000000000000000 I didn't find any references to this restriction in exception.c. I did find some code implying this in errors.c and ceval.c, but that wouldn't have caught my eye if I weren't specifically looking for it *after* having just read the discussion about (rejected) PEP 317. -jJ

It's an unfortunate feature; it should be mentioned in the docs; it should also be fixed, but fixing it isn't easy (believe me, or it would have been fixed in Python 2.2). To be honest, I don't recall the exact reasons why this wasn't fixed in 2.2; I believe it has something to do with the problem of distinguishing between string and class exception, and between the various forms of raise statements. I think the main ambiguity is raise "abc", which could be considered short for raise str, "abc", but that would be incompatible with except "abc". I also think that the right way out of there is to simply hardcode a check that says that raise "abc" raises a string exception and raising any other instance raises a class exception. But there's a lot of code that has to be changed. It's been suggested that all exceptions should inherit from Exception, but this would break tons of existing code, so we shouldn't enforce that until 3.0. (Is there a PEP for this? I think there should be.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 2005-01-16, at 02.57, Guido van Rossum wrote:
What would happen if Exception were made a new-style class, enforce inheritance from Exception for all new-style exceptions, and allow all old-style exceptions as before. Am I wrong in assuming that only the most esoteric exceptions inheriting from Exception would break by Exception becoming new-style? //Simon

On 2005 Jan 16, at 10:27, Martin v. Löwis wrote:
Couldn't we just specialcase strings specifically, to keep grandfathering them in?
In addition, code may break which assumes that exceptions are classic instances, e.g. that they are picklable, have an __dict__, and so on.
There would be no problem giving the new class Extension(object): ... a __dict__ and the ability to get pickled, particularly since both come by default. The "and so on" would presumably refer to whether special methods should be looked up on the instance or the type. But as I understand the question (raised in the threads about copy.py) the planned solution is to make special methods their own kind of descriptors, so even that exoteric issue could well be finessed.
It seems to me that if the new-style Exception is made very normally and strings are grandfathered in, we ARE down to exoteric breakage cases (potentially fixable by those new magic descriptors as above for specialmethods). Alex

Alex Martelli wrote:
Sure. That just wouldn't be the change that Simon described, anymore. You don't specify in which way you would like to specialcase strings. Two alternatives are possible: 1. Throwing strings is still allowed, and to catch them, you need the identical string (i.e. the current behaviour) 2. Throwing strings is allowed, and they can be caught by either the identical string, or by catching str In the context of Simon's proposal, the first alternative would be more meaningful, I guess.
The "and so on" would presumably refer to whether special methods should be looked up on the instance or the type.
Perhaps. That type(exc) changes might also cause problems.
This would be worth a try. Does anybody have a patch to implement it? Regards, Martin

At 05:57 PM 1/15/05 -0800, Guido van Rossum wrote:
Couldn't we require new-style exceptions to inherit from Exception? Since there are no new-style exceptions that work now, this can't break existing code. Then, the code path is just something like: if isinstance(ob,Exception): # it's an exception, use its type else: # all the other tests done now This way, the other tests that would be ambiguous wrt new-style classes can be skipped, but non-Exception classic classes would still be handled by the existing checks. Or am I missing something?

On 2005 Jan 16, at 10:28, Martin v. Löwis wrote:
Not necessarily, since Python supports multiple inheritance: class MyException(Exception, object): ..... there -- a newstyle exception class inheriting from oldstyle Exception. (ClassType goes to quite some trouble to allow this, getting the metaclass from _following_ bases if any). Without inheritance you might similarly say: class AnotherOne(Exception): __metaclass__ = type ...
This, in itself, could break existing code.
Not necessarily, see my previous post. But anyway, PJE's proposal is less invasive than making Exception itself newstyle. Alex

Guido van Rossum wrote:
There's actually a bug open on the fact that exceptions can't be new-style classes: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=518846&group_id=5470 I added some comments to try to stir it up but there ended up being a lot of confusion and I don't think I helped much. The problem is that people want to solve the larger issues (raising strings, wanting to force all exceptions to be new-style, etc) but those all have long-term solutions, while the current bug just languishes. robey

Guido van Rossum <gvanrossum@gmail.com> writes:
A few months back I hacked an attempt to make all exceptions new-style. It's not especially hard, but it's tedious. There's lots of code (more than I expected, anyway) to change and my attempt ended up being pretty messy. I suspect allowing both old- and new-style classes would be no harder, but even more tedious and messy. It would still be worth doing, IMHO. Cheers, mwh -- <cube> If you are anal, and you love to be right all the time, C++ gives you a multitude of mostly untimportant details to fret about so you can feel good about yourself for getting them "right", while missing the big picture entirely -- from Twisted.Quotes

Hi, On Fri, Jan 14, 2005 at 07:20:31PM -0500, Jim Jewett wrote:
The base of the Exception hierarchy happens to be a classic class. But why are they "required" to be classic?
For reference, PyPy doesn't have old-style classes at all so far, so we had to come up with something about exceptions. After some feedback from python-dev it appears that the following scheme works reasonably well. Actually it's surprizing how little problems we actually encountered by removing the old-/new-style distinction (particularly when compared with the extremely obscure workarounds we had to go through in PyPy itself, e.g. precisely because we wanted exceptions that are member of some (new-style) class hierarchy). Because a bit of Python code tells more than long and verbose explanations, here it is: def app_normalize_exception(etype, value, tb): """Normalize an (exc_type, exc_value) pair: exc_value will be an exception instance and exc_type its class. """ # mistakes here usually show up as infinite recursion, which is fun. while isinstance(etype, tuple): etype = etype[0] if isinstance(etype, type): if not isinstance(value, etype): if value is None: # raise Type: we assume we have to instantiate Type value = etype() elif isinstance(value, tuple): # raise Type, Tuple: assume Tuple contains the constructor # args value = etype(*value) else: # raise Type, X: assume X is the constructor argument value = etype(value) # raise Type, Instance: let etype be the exact type of value etype = value.__class__ elif type(etype) is str: # XXX warn -- deprecated if value is not None and type(value) is not str: raise TypeError("string exceptions can only have a string value") else: # raise X: we assume that X is an already-built instance if value is not None: raise TypeError("instance exception may not have a separate" " value") value = etype etype = value.__class__ # for the sake of language consistency we should not allow # things like 'raise 1', but it's probably fine (i.e. # not ambiguous) to allow them in the explicit form 'raise int, 1' if not hasattr(value, '__dict__') and not hasattr(value, '__slots__'): raise TypeError("raising built-in objects can be ambiguous, " "use 'raise type, value' instead") return etype, value, tb Armin

[Armin]
That is stricter than classic Python though -- it allows the value to be anything (and you get the value back unadorned in the except 's', x: clause). [Michael]
It would still be worth doing, IMHO.
Then let's do it. Care to resurrect your patch? (And yes, classic classes should also be allowed for b/w compatibility.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <gvanrossum@gmail.com> writes:
I found it and uploaded it here: http://starship.python.net/crew/mwh/new-style-exception-hacking.diff The change to type_str was the sort of unexpected change I was talking about. TBH, I'm not sure it's really worth working from my patch, a more sensible course would be to just do the work again, but paying a bit more attention to getting a maintainable result. Questions: a) Is Exception to be new-style? b) Somewhat but not entirely independently, would demanding that all new-style exceptions inherit from Exception be reasonable? Cheers, mwh -- ZAPHOD: You know what I'm thinking? FORD: No. ZAPHOD: Neither do I. Frightening isn't it? -- The Hitch-Hikers Guide to the Galaxy, Episode 11

At 04:06 PM 1/17/05 +0000, Michael Hudson wrote:
a) Is Exception to be new-style?
Probably not in 2.5; Martin and others have suggested that this could introduce instability for users' existing exception classes.
b) Somewhat but not entirely independently, would demanding that all new-style exceptions inherit from Exception be reasonable?
Yes. Right now you can't have a new-style exception at all, so it would be quite reasonable to require new ones to inherit from Exception.

On Mon, 17 Jan 2005 11:35:53 -0500, Phillip J. Eby <pje@telecommunity.com> wrote:
Really? I thought that was eventually decided to be a very small amount of code.
That would be much more reasonable if Exception itself was a new-style class. As long as it isn't, you'd have to declare new-style classes like this: class MyError(Exception, object): ... which is ugly. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

At 10:16 AM 1/17/05 -0800, Guido van Rossum wrote:
Guess I missed that part of the thread in the ongoing flood of PEP 246 stuff. :)
I was thinking the use case was that you were having to add 'Exception', not that you were adding 'object'. The two times in the past that I wanted to make a new-style class an exception, I *first* made it a new-style class, and *then* tried to make it an exception. I believe the OP on this thread described the same thing. But whatever; as long as it's *possible*, I don't care much how it's done, and I can't think of anything in my code that would break from making Exception new-style.

Well, right now you would only want to make an exception a new style class if you had a very specific use case for wanting the new style class. But once we allow new-style exceptions *and* require them to inherit from Exception, we pretty much send the message "if you're not using new-style exceptions derived from Exception your code is out of date" and that means it should be as simple as possible to make code conform. And that means IMO making Exception a new style class. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I still think that only an experiment could decide: somebody should come up with a patch that does that, and we will see what breaks. I still have the *feeling* that this has significant impact, but I could not pin-point this to any specific problem I anticipate. Regards, Martin

This sounds like a good approach. We should do this now in 2.5, and as alpha and beta testing progresses we can decide whether to roll it back of what kind of backwards compatibility to provide. (Most exceptions are very short classes with very limited behavior, so I expect that in the large majority of cases it won't matter. The question is of course how small the remaining minority is.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

"Martin v. Löwis" <martin@v.loewis.de> writes:
Well, some code is certainly going to break such as this from warnings.py: assert isinstance(category, types.ClassType), "category must be a class" or this from traceback.py: if type(etype) == types.ClassType: stype = etype.__name__ else: stype = etype I hope to have a new patch (which makes PyExc_Exception new-style, but allows arbitrary old-style classes as exceptions) "soon". It may even pass bits of "make test" :) Cheers, mwh -- SPIDER: 'Scuse me. [scuttles off] ZAPHOD: One huge spider. FORD: Polite though. -- The Hitch-Hikers Guide to the Galaxy, Episode 11

Michael Hudson <mwh@python.net> writes:
Done: http://www.python.org/sf/1104669 It passed 'make test' apart from failures I really don't think are my fault. I'll run "regrtest -uall" overnight... Cheers, mwh -- [1] If you're lost in the woods, just bury some fibre in the ground carrying data. Fairly soon a JCB will be along to cut it for you - follow the JCB back to civilsation/hitch a lift. -- Simon Burr, cam.misc

Michael Hudson <mwh@python.net> writes:
Now I think it's really done, apart from documentation. My design decision was to make Exception new-style. Things can be raised if they are instances of old-style classes or instances of Exception. If this meets with general agreement, I'd like to check the above patch in. It will break some highly introspective code, so it's IMO best to get it in early in the 2.5 cycle. The other option is to keep Exception old-style but allow new-style subclasses, but I think all this will do is break the above mentioned introspective code in a quieter way... The patch also updates the PendingDeprecationWarning on raising a string exception to a full DeprecationWarning (something that should be done anyway). Cheers, mwh -- python py.py ~/Source/python/dist/src/Lib/test/pystone.py Pystone(1.1) time for 5000 passes = 19129.1 This machine benchmarks at 0.261381 pystones/second

I like it, but didn't you forget to mention that strings can still be raised? I think we can't break that (but we can insert a deprecation warning for this in 2.5 so we can hopefully deprecate it in 2.6, or 2.7 at the latest).
What I said. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <gvanrossum@gmail.com> writes:
I try to forget that as much as possible :)
:) I'll try to bash the documentation into shape next. Cheers, mwh -- please realize that the Common Lisp community is more than 40 years old. collectively, the community has already been where every clueless newbie will be going for the next three years. so relax, please. -- Erik Naggum, comp.lang.lisp
participants (9)
-
"Martin v. Löwis"
-
Alex Martelli
-
Armin Rigo
-
Guido van Rossum
-
Jim Jewett
-
Michael Hudson
-
Phillip J. Eby
-
Robey Pointer
-
Simon Percivall