Enhance exceptions by attaching some more information to them
More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem. For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range In this case there's no reference to neither the length of the array nor to the offending index. I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging. For example, one such use case would be to have a modified test runner, that in case of exceptions automatically prints some debug information. Another would be a logger system that in case of an exception also logs some debug info that could be relevant to understand and solve the issue. I propose extending (at least) the following exceptions with the following attributes: KeyError: key, object IndexError: index, object AttributeError: attribute, object NameError: name Of course that populating these attributes could be controlled by a flag. I know that some of this information is already present in some exceptions, depending on the context. However, I propose adding these attributes, as in this way a tool could easily and reliably extract the information and work with it, as opposed to have to parse the huge variety of different messages there are. For the first use case mentioned above I have a working prototype, although it does not use this proposal, but a series of hacks (I'm modifying the bytecode to save a reference to the key and object :() and parsing of the exception messages. But I want to share what the output of such a tool could be. ====================================================================== ERROR: test_attribute (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 18, in test_attribute AttributeError: 'str' object has no attribute 'Lower'. Did you mean 'islower', 'lower'? Debug info: Object: '' Type: <type 'str'> ====================================================================== ERROR: test_index (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 6, in test_index IndexError: list index out of range Debug info: Object: [1, 2] Object len: 2 Index: 2 ====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 10, in test_key KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo' ====================================================================== ERROR: test_name (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 14, in test_name NameError: global name 'fooo' is not defined. Did you mean 'foo'? ---------------------------------------------------------------------- Ran 4 tests in 0.005s -- Sebastian Kreft
On 16 February 2014 01:40, Sebastian Kreft <skreft@gmail.com> wrote:
More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem.
For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range
In this case there's no reference to neither the length of the array nor to the offending index.
I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging.
Yes, having particular exceptions expose more structured data would be a good thing, and some improvements have already been made in that direction (ImportError now has name and path attributes, for example) The barriers are largely practical ones rather than philosophical: 1. Each updated exception needs to have the new optional attributes and keyword-only arguments defined (in C for the building exceptions, although Argument Clinic makes the argument parsing aspects easier now), along with appropriate docstring and documentation updates, as well as changes to the default representation and/or implicit calculation of an args[0]. (They typically have to be optional keyword only parameters for backwards compatibility reasons) 2. To actually benefit end users, the new exception capabilities need to be used appropriately in the core interpreter and standard library. It's easy to look at the scope of work involved in step 2, and back away from working on step 1. However, it's important to realise that step 2 can be done incrementally - step 1 is about making it *possible* to provide more structured information in the exceptions, and then step 2 can happen more opportunistically by tackling a particular part of the interpreter or standard library at a time. (for example, that is how the improved introspection for builtins and extension modules is being rolled out). Step 1 could likely benefit from being elaborated in a PEP, as that then provides a useful starting point for other implementations when they go to update their own builtin exception implementations accordingly. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 02/15/2014 04:40 PM, Sebastian Kreft wrote:
More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem.
For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range
In this case there's no reference to neither the length of the array nor to the offending index.
I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging.
For example, one such use case would be to have a modified test runner, that in case of exceptions automatically prints some debug information. Another would be a logger system that in case of an exception also logs some debug info that could be relevant to understand and solve the issue.
I propose extending (at least) the following exceptions with the following attributes: KeyError: key, object IndexError: index, object AttributeError: attribute, object NameError: name
Of course that populating these attributes could be controlled by a flag.
I know that some of this information is already present in some exceptions, depending on the context. However, I propose adding these attributes, as in this way a tool could easily and reliably extract the information and work with it, as opposed to have to parse the huge variety of different messages there are.
For the first use case mentioned above I have a working prototype, although it does not use this proposal, but a series of hacks (I'm modifying the bytecode to save a reference to the key and object :() and parsing of the exception messages. But I want to share what the output of such a tool could be.
====================================================================== ERROR: test_attribute (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 18, in test_attribute AttributeError: 'str' object has no attribute 'Lower'. Did you mean 'islower', 'lower'? Debug info: Object: '' Type: <type 'str'>
====================================================================== ERROR: test_index (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 6, in test_index IndexError: list index out of range Debug info: Object: [1, 2] Object len: 2 Index: 2
====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 10, in test_key KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo'
====================================================================== ERROR: test_name (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 14, in test_name NameError: global name 'fooo' is not defined. Did you mean 'foo'?
---------------------------------------------------------------------- Ran 4 tests in 0.005s
I very much approve this proposal (also the comments by Nick). Actually, I had a similar idea of systematically adding _relevant data_ to error reports; and also proposing it as standard practice in guidelines, not only when using standard error types, but for custom ones too. That these data are set as attributes on error objects, making them reusable in custom error report formats, is also an excellent point. I would however take the opporunity to improve a bit the traceback part of error reports by (1) renaming it (2) setting it apart (with a simple line). (As they are now, tracebacks are real enigmas to novice (python) programmers, and they mess up the rest of error messages.) For instance: ====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo' ---------------------------------------------------------------------- Function call chain chronology: ---------------------------------------------------------------------- File "_.py", line 9, in <module> main() File "_.py", line 7, in main test() File "_.py", line 5, in test test_key() File "_.py", line 3, in test_key print(x['fooo']) ====================================================================== Maybe "Function call chain" in not the best term --propose your own-- but it still helps and understand what it's all about. "Chronology" seems self-explaining to me and avoids "(most recent call last)". Placing the tracback after the message just reflects the fact that users (especially novice ones) read top down. I do agree that experienced programmers often read python error messages backwards --starting at the bottom-- but it's because what's usually the most relevant info is placed there, at the end of the message, in standard python error format: Traceback (most recent call last): File "_.py", line 10, in <module> main() File "_.py", line 8, in main test() File "_.py", line 6, in test test_key() File "_.py", line 4, in test_key print(x['fooo']) KeyError: 'fooo' But I would not fight for this. d
+1 on the idea (I can't comment on the implementation). One of Python's great strengths is its excellent error messages. But if they can be made still better, that would be fantastic. Rob Cliffe On 15/02/2014 15:40, Sebastian Kreft wrote:
More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem.
For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range
In this case there's no reference to neither the length of the array nor to the offending index.
I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging.
For example, one such use case would be to have a modified test runner, that in case of exceptions automatically prints some debug information. Another would be a logger system that in case of an exception also logs some debug info that could be relevant to understand and solve the issue.
I propose extending (at least) the following exceptions with the following attributes: KeyError: key, object IndexError: index, object AttributeError: attribute, object NameError: name
Of course that populating these attributes could be controlled by a flag.
I know that some of this information is already present in some exceptions, depending on the context. However, I propose adding these attributes, as in this way a tool could easily and reliably extract the information and work with it, as opposed to have to parse the huge variety of different messages there are.
For the first use case mentioned above I have a working prototype, although it does not use this proposal, but a series of hacks (I'm modifying the bytecode to save a reference to the key and object :() and parsing of the exception messages. But I want to share what the output of such a tool could be.
====================================================================== ERROR: test_attribute (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py <http://example.py.py>", line 18, in test_attribute AttributeError: 'str' object has no attribute 'Lower'. Did you mean 'islower', 'lower'? Debug info: Object: '' Type: <type 'str'>
====================================================================== ERROR: test_index (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py <http://example.py.py>", line 6, in test_index IndexError: list index out of range Debug info: Object: [1, 2] Object len: 2 Index: 2
====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py <http://example.py.py>", line 10, in test_key KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo'
====================================================================== ERROR: test_name (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py <http://example.py.py>", line 14, in test_name NameError: global name 'fooo' is not defined. Did you mean 'foo'?
---------------------------------------------------------------------- Ran 4 tests in 0.005s
-- Sebastian Kreft
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14
Thanks for the comments, I will start working on a PEP for Step 1. On Mon, Feb 17, 2014 at 2:38 PM, Rob Cliffe <rob.cliffe@btinternet.com>wrote:
+1 on the idea (I can't comment on the implementation). One of Python's great strengths is its excellent error messages. But if they can be made still better, that would be fantastic. Rob Cliffe
On 15/02/2014 15:40, Sebastian Kreft wrote:
More than once I've been in a situation where I wish that some of the stdlib exceptions had a better message or some more information to help me diagnose the problem.
For example: a = [1, 2, 3, 4, 5] a[5] IndexError: list index out of range
In this case there's no reference to neither the length of the array nor to the offending index.
I'm of the idea that we could extend the exceptions by adding some more information to them, so 3rd party libraries could use them for debugging/logging.
For example, one such use case would be to have a modified test runner, that in case of exceptions automatically prints some debug information. Another would be a logger system that in case of an exception also logs some debug info that could be relevant to understand and solve the issue.
I propose extending (at least) the following exceptions with the following attributes: KeyError: key, object IndexError: index, object AttributeError: attribute, object NameError: name
Of course that populating these attributes could be controlled by a flag.
I know that some of this information is already present in some exceptions, depending on the context. However, I propose adding these attributes, as in this way a tool could easily and reliably extract the information and work with it, as opposed to have to parse the huge variety of different messages there are.
For the first use case mentioned above I have a working prototype, although it does not use this proposal, but a series of hacks (I'm modifying the bytecode to save a reference to the key and object :() and parsing of the exception messages. But I want to share what the output of such a tool could be.
====================================================================== ERROR: test_attribute (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 18, in test_attribute AttributeError: 'str' object has no attribute 'Lower'. Did you mean 'islower', 'lower'? Debug info: Object: '' Type: <type 'str'>
====================================================================== ERROR: test_index (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 6, in test_index IndexError: list index out of range Debug info: Object: [1, 2] Object len: 2 Index: 2
====================================================================== ERROR: test_key (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 10, in test_key KeyError_: 'fooo', did you mean 'foo'? Debug info: Object: {'foo': 1} Key: 'fooo'
====================================================================== ERROR: test_name (example.ExampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/skreft/test/debug_exception/example.py.py", line 14, in test_name NameError: global name 'fooo' is not defined. Did you mean 'foo'?
---------------------------------------------------------------------- Ran 4 tests in 0.005s
-- Sebastian Kreft
_______________________________________________ Python-ideas mailing listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Sebastian Kreft
participants (4)
-
Nick Coghlan -
Rob Cliffe -
Sebastian Kreft -
spir