Exception handling in objects
While using any library there is a need to handle exceptions it raises. In ideal conditions author of library defines base exception and builds hierarchy of exceptions based on it. Moreover, all exceptions a library raises (including those that do not inherit its base exception) are documented and this information is kept up to date. Unfortunately, this is not always the case. Library authors are free to use the language exception mechanism in any way. So instead of using documented features of library as a black box, the user often has to plunge into the source code of a library itself and look for exceptions it could raise in order to be prepared for them. The essence of my idea is simple and does not affects compatibility with existing code: to automatically define a magic attribute of module under the working name "__exception__" while importing any module. It can be used in usual way, for example:
Here any exception an author of library raises can be handled. This way you can handle any exception that occurs in the library without even knowing what exceptions it can raise. The important point is that only exceptions raised in library itself are handled. Those exceptions that can be raised in underlying libraries and so on through the dependency chain should be handled separately. Developing this idea, it could be more universal mechanism that works not only with modules, but with any objects of the language. For instance:
It is important to understand the key difference between this mechanism and the capture of broad exception: only those exceptions are handled that are RAISED EXPLICITLY inside the object, while broad exception captures ANY exceptions that may occur in the object. It seems to me this idea will help library users more correctly handle their exceptions if this is problematic. This will save time and make code more reliable.
Sorry, the code was lost. I bring it again: try: any_module.anything except any_module.__ exception__: ... try: any_function() except any_function.__ exception__: ... try: AnyClass.anything except AnyClass.__ exception__: ...
Marat Sharafutdinov wrote:
I'm not convinced that there are realistic use cases for this. Personally I've never been interested in catching exceptions based on which module they originated from. If there's a specific condition I want to catch and it has a distinctive exception class, I find out its class either from the documentation or by experiment. Otherwise it's just a matter of "something unexpected went wrong" and I treat it the same regardless of which module or library raised it. If you have an actual use case from real live code, please tell us more. -- Greg
On Aug 14, 2019, at 06:19, Marat Sharafutdinov <decaz89@gmail.com> wrote:
I think you need to come up with a complete implementation mechanism (not necessarily trying to actually code it in C, just being able to write it in pseudocode or reference-style description). Without that, there are just way too many questions for the idea to really be judged. And trying to answer those questions one by one will probably just lead to something contradictory and unimplementable, but describing an implementation will immediately answer most of the questions. (And if some of them are answered in a way you don’t like, it’ll force you to think about how you could change the design to get the right answer.) For example: How do you define “library” here? Does it include submodules? What if it’s a namespace package, with submodules provides by different libraries? What about things that are defined in a private sibling module but publicly re-exported by the module (as with, e.g., most C accelerators in the stdlib)? Or monkeypatched into the module? What about things defined by exec within a module? Or things like namedtuple, where the semantics are clearly defined by the module that uses namedtuple but the code is compiled by exec in the collections module? (Does it matter whose globals are passed to exec?) Or exceptions raised by decorators like dataclass or total_ordering or partial that are semantically meant to be treated as exceptions raised by the decorated class or function but that are defined in the decorator code? If you reraise an exception with bare raise, can it now be caught with your module’s __exceptions__? What if you reraise it explicitly with `except Exception as e: raise e`? What if the exception is constructed in one module but raised in another? What about generator.throw, where the actual raiser is the interpreter itself, but clearly the reader sees it as being raised by the caller of throw? Or user-level code that acts like generator.throw (e.g., to throw an exception into a thread or run loop); is the module the thrower or the implementor of throw? Does it only handle Exception and its subclasses raised by the module, or everything? Whatever the rule, how is it applied? Does the importer build an __exceptions__ tuple while or after compiling the code? Then what happens with C modules, dynamically-constructed modules, etc.? (If it’s a tuple but not obviously a tuple, what happens when someone quite naturally writes except (IOError, yourmod.__exceptions__):?) If it’s not a tuple but something semi-magical that tests something dynamic like the __ module__ of the exception being raised, what is the magic? How does this it fit into the existing mechanism that just stacks up types for try blocks? And what’s the value of __exceptions__ if you inspect it? Can a module override the behavior? For example, if a stdlib or third party module already went to a lot of trouble to define an error base class that serves the intended function here (possibly making different judgment calls on some of those questions, but they’re the ones the user wants, or at least the ones the author thinks the user wants), can that module assign __exceptions__ = error?
Developing this idea, it could be more universal mechanism that works not only with modules, but with any objects of the language.
Does a class’s __exceptions__ only include exceptions from the class body? From any methods defined in the class body? From any methods found in the class’s __dict__ at runtime? (Maybe the __module__ gets added at descriptor __get__ time?) Does it include nested classes? Do any of base classes, metaclasses, or class decorators count as part of the class? Can you use it on any object, to catch exceptions raised explicitly by that object’s class but only when self matches the object? (What happens if you call a classmethod on the object, possibly without even realizing it’s a classmethod?) Or some other rule? Whatever that rule is, is it ignored by the special rules for module, type, and function, or do they just add their special rules to the generic object rule? On a function, does it include local functions? Classes defined within the function? Exec from within the function (does it depend whose locals are passed)? What about a callable that’s not a function or class? Does it just catch exceptions raised inside the __call__ method itself? And so on.
On Wed, Aug 14, 2019 at 11:34 PM Marat Sharafutdinov <decaz89@gmail.com> wrote:
Can you elaborate on either (a) the problem that this is solving, or (b) the intended way this feature would be used? I'm a bit confused here. What kind of exception handling are you looking at where you need to handle all exceptions from one module in the same way, but everything else in a different way? ChrisA
On 14/08/2019 14:19:05, Marat Sharafutdinov wrote: the above sentence) whether you mean exceptions *raised* in a module, or exceptions *defined* (and raised) in a module. What if a module says e.g. raise ValueError would that trapped by except any_module.__ exception__: ? Please clarify. Best wishes Rob Cliffe
On Wed, Aug 14, 2019 at 01:19:05PM -0000, Marat Sharafutdinov wrote:
While using any library there is a need to handle exceptions it raises.
Is that true? For most exceptions, an exceptions means there is a bug in your code. The exception is a sign that you are doing something wrong, and have to fix your code, not something for you to catch and handle.
Is __exception__ generated automatically? How is the interpreter supposed to know what exceptions can be raised by the library, and which ones should be caught by calling code? Why do you think that a *single* except block for *everything* in the module is sufficient?
How can you handle an exception without knowing what sort of exception it is and what it means? It is easy to write toy pseudo-code where the exception handling is "..." but it is another story about writing real code that catches real exceptions. Let's take pathlib as an example. It can raise (amoung other exceptions) KeyError, NotImplementedError, TypeError, FileExistsError, and FileNotFoundError. What sort of meaningful error handler can you write that will treat all of those exceptions, and all the others I haven't listed, from the same except block, without knowing which exception actually occurred? -- Steven
Sorry, the code was lost. I bring it again: try: any_module.anything except any_module.__ exception__: ... try: any_function() except any_function.__ exception__: ... try: AnyClass.anything except AnyClass.__ exception__: ...
Marat Sharafutdinov wrote:
I'm not convinced that there are realistic use cases for this. Personally I've never been interested in catching exceptions based on which module they originated from. If there's a specific condition I want to catch and it has a distinctive exception class, I find out its class either from the documentation or by experiment. Otherwise it's just a matter of "something unexpected went wrong" and I treat it the same regardless of which module or library raised it. If you have an actual use case from real live code, please tell us more. -- Greg
On Aug 14, 2019, at 06:19, Marat Sharafutdinov <decaz89@gmail.com> wrote:
I think you need to come up with a complete implementation mechanism (not necessarily trying to actually code it in C, just being able to write it in pseudocode or reference-style description). Without that, there are just way too many questions for the idea to really be judged. And trying to answer those questions one by one will probably just lead to something contradictory and unimplementable, but describing an implementation will immediately answer most of the questions. (And if some of them are answered in a way you don’t like, it’ll force you to think about how you could change the design to get the right answer.) For example: How do you define “library” here? Does it include submodules? What if it’s a namespace package, with submodules provides by different libraries? What about things that are defined in a private sibling module but publicly re-exported by the module (as with, e.g., most C accelerators in the stdlib)? Or monkeypatched into the module? What about things defined by exec within a module? Or things like namedtuple, where the semantics are clearly defined by the module that uses namedtuple but the code is compiled by exec in the collections module? (Does it matter whose globals are passed to exec?) Or exceptions raised by decorators like dataclass or total_ordering or partial that are semantically meant to be treated as exceptions raised by the decorated class or function but that are defined in the decorator code? If you reraise an exception with bare raise, can it now be caught with your module’s __exceptions__? What if you reraise it explicitly with `except Exception as e: raise e`? What if the exception is constructed in one module but raised in another? What about generator.throw, where the actual raiser is the interpreter itself, but clearly the reader sees it as being raised by the caller of throw? Or user-level code that acts like generator.throw (e.g., to throw an exception into a thread or run loop); is the module the thrower or the implementor of throw? Does it only handle Exception and its subclasses raised by the module, or everything? Whatever the rule, how is it applied? Does the importer build an __exceptions__ tuple while or after compiling the code? Then what happens with C modules, dynamically-constructed modules, etc.? (If it’s a tuple but not obviously a tuple, what happens when someone quite naturally writes except (IOError, yourmod.__exceptions__):?) If it’s not a tuple but something semi-magical that tests something dynamic like the __ module__ of the exception being raised, what is the magic? How does this it fit into the existing mechanism that just stacks up types for try blocks? And what’s the value of __exceptions__ if you inspect it? Can a module override the behavior? For example, if a stdlib or third party module already went to a lot of trouble to define an error base class that serves the intended function here (possibly making different judgment calls on some of those questions, but they’re the ones the user wants, or at least the ones the author thinks the user wants), can that module assign __exceptions__ = error?
Developing this idea, it could be more universal mechanism that works not only with modules, but with any objects of the language.
Does a class’s __exceptions__ only include exceptions from the class body? From any methods defined in the class body? From any methods found in the class’s __dict__ at runtime? (Maybe the __module__ gets added at descriptor __get__ time?) Does it include nested classes? Do any of base classes, metaclasses, or class decorators count as part of the class? Can you use it on any object, to catch exceptions raised explicitly by that object’s class but only when self matches the object? (What happens if you call a classmethod on the object, possibly without even realizing it’s a classmethod?) Or some other rule? Whatever that rule is, is it ignored by the special rules for module, type, and function, or do they just add their special rules to the generic object rule? On a function, does it include local functions? Classes defined within the function? Exec from within the function (does it depend whose locals are passed)? What about a callable that’s not a function or class? Does it just catch exceptions raised inside the __call__ method itself? And so on.
On Wed, Aug 14, 2019 at 11:34 PM Marat Sharafutdinov <decaz89@gmail.com> wrote:
Can you elaborate on either (a) the problem that this is solving, or (b) the intended way this feature would be used? I'm a bit confused here. What kind of exception handling are you looking at where you need to handle all exceptions from one module in the same way, but everything else in a different way? ChrisA
On 14/08/2019 14:19:05, Marat Sharafutdinov wrote: the above sentence) whether you mean exceptions *raised* in a module, or exceptions *defined* (and raised) in a module. What if a module says e.g. raise ValueError would that trapped by except any_module.__ exception__: ? Please clarify. Best wishes Rob Cliffe
On Wed, Aug 14, 2019 at 01:19:05PM -0000, Marat Sharafutdinov wrote:
While using any library there is a need to handle exceptions it raises.
Is that true? For most exceptions, an exceptions means there is a bug in your code. The exception is a sign that you are doing something wrong, and have to fix your code, not something for you to catch and handle.
Is __exception__ generated automatically? How is the interpreter supposed to know what exceptions can be raised by the library, and which ones should be caught by calling code? Why do you think that a *single* except block for *everything* in the module is sufficient?
How can you handle an exception without knowing what sort of exception it is and what it means? It is easy to write toy pseudo-code where the exception handling is "..." but it is another story about writing real code that catches real exceptions. Let's take pathlib as an example. It can raise (amoung other exceptions) KeyError, NotImplementedError, TypeError, FileExistsError, and FileNotFoundError. What sort of meaningful error handler can you write that will treat all of those exceptions, and all the others I haven't listed, from the same except block, without knowing which exception actually occurred? -- Steven
participants (6)
-
Andrew Barnert -
Chris Angelico -
Greg Ewing -
Marat Sharafutdinov -
Rob Cliffe -
Steven D'Aprano