Re: [Python-ideas] Arguments to exceptions
Paul, I think you are fixating too much on Ken's example. I think I understand what he is saying and I agree with him. It is a problem I struggle with routinely. It occurs in the following situations: 1. You are handling an exception that you are not raising. This could be because Python itself is raising the exception, as in Ken's example, or it could be raised by some package you did not write. 2. You need to process or transform the message in some way. Consider this example: import json >>> s = '{"abc": 0, "cdf: 1}' >>> try: ... d = json.loads(s) ... except Exception as e: ... print(e) ... print(e.args) Unterminated string starting at: line 1 column 12 (char 11) ('Unterminated string starting at: line 1 column 12 (char 11)',) Okay, I have caught an exception for which I have no control over how the exception was raised. Now, imagine that I am writing an application that highlights json errors in place. To do so, I would need the line and column numbers to highlight the location of the error, and ideally I'd like to strip them from the base message and just show that. You can see from my second print statement that the line and column numbers were not passed as separate arguments. Thus I need to parse the error message to extract them. Not a difficult job, but fragile. Any change to the error message could break my code. I don't know what this code smell is that people keep referring to, but to me, that code would smell. Jeff
On 3 July 2017 at 20:46, Jeff Walker <jeff.walker00@yandex.com> wrote:
Possibly. I hadn't reread the original email. Having done so, I'm confused as to how the proposal and the example are related. The proposal makes no difference unless the places where (for example) NameError are raised are changed. But the proposal doesn't suggest changing how the interpreter raises NameError. So how will the proposal make a difference? I'd understood from the example that Ken's need was to be able to find the name that triggered the NameError. His proposal doesn't do that (unless we are talking about user-raised NameError exceptions, as opposed to ones the interpreter raises - in which case why not just use a user-defined exception? So I'm -1 on his proposal, as I don't see anything in it that couldn't be done in user code for user-defined exceptions, and there's nothing in the proposal suggesting a change in how interpreter-raised exceptions are created.
Then yes, you need to know the API presented by the exception. Projects (and the core interpreter) are not particularly good at documenting (or designing) the API for their exceptions, but that doesn't alter the fact that exceptions are user-defined classes and as such do have an API. I'd be OK with arguments that the API of built in exceptions as raised by the interpreter could be improved. Indeed, I thought that was Ken's proposal. But his proposal seems to be that if we add a ___str__ method to BaseException, that will somehow automatically improve the API of all other exceptions. To quote Ken:
Alternatively, I could argue that code which uses print(str(e)) as its exception handling isn't very well written, and the fact that people do this is what leads to people passing only one argument to their exceptions when creating them. Look, I see that there might be something that could be improved here. But I don't see an explanation of how, if we implement just the proposed change to BaseException, the user code that Ken's quoting as having a problem could be improved. There seems to be an assumption of "and because of that change, people raising exceptions would change what they do". Frankly, no they wouldn't. There's no demonstrated benefit for them, and they'd have to maintain a mess of backward compatibility code. So why would they bother? Anyway, you were right that I'd replied to just the example, not the original proposal. I apologise for that, I should have read the thread more carefully. But if I had done so, it wouldn't have made much difference - I still don't see a justification for the proposed change. Paul
Paul, Indeed, nothing gets better until people change the way they do their exceptions. Ken's suggested enhancement to BaseException does not directly solve the problem, but it removes the roadblocks that discourage people from passing the components to the message. Seems to me that to address this problem, four things are needed: 1. Change BaseException. This allows people to pass the components to the message without ruining str(e). 2. A PEP that educates the Python community on this better way of writing exceptions. 3. Changes to the base language and standard library to employ the new approach. These would changes would be quite small and could be done opportunistically. 4. Time. Over time things will just get better as more people see the benefits of this new approach to exceptions and adopt it. And if they don't, we are no worse off than we are now. And frankly, I don't see any downside. The changes he is proposing are tiny and simple and backward compatible. -Jeff 03.07.2017, 14:23, "Paul Moore" <p.f.moore@gmail.com>:
On 3 July 2017 at 21:56, Jeff Walker <jeff.walker00@yandex.com> wrote:
As noted, I disagree that people are not passing components because str(e) displays them the way it does. But we're both just guessing at people's motivations, so there's little point in speculating.
I dispute this is the essential place to start. If nothing else, the proposed approach encourages people to use a position-based "args" attribute for exceptions, rather than properly named attributes.
2. A PEP that educates the Python community on this better way of writing exceptions.
Educating the community can be done right now, and doesn't need a PEP. Someone could write a blog post, or an article, that explains how to code exception classes, how to create the exceptions, and how client code can/should use the API. This can be done now, all you need to do is to start with "at the moment, BaseException doesn't implement these features, so you should create an application-specific base exception class to minimise duplication of code". If project authors take up the proposed approach, then that makes a good argument for moving the supporting code into the built in BaseException class.
And I've never said that there's a problem with these. Although I do dispute that using an args list is the best approach here - I'd much rather see NameError instances have a "name" attribute that had the name that couldn't be found as its value. Opportunistic changes to built in exceptions can implement the most appropriate API for the given exception - why constrain such changes to a "lowest common denominator" API that is ideal for no-one? class NameError(BaseException): def __init__(self, name): self.name = name def __str__(self): return f"name '{self.name}' is not defined" Of course, that's not backward compatible as it stands, but it could probably be made so, just as easily as implementing the proposed solution.
The same could be said of any improved practice. And I agree, let's encourage people to learn to write better code, and promote good practices. There's definitely no reason not to do this.
And frankly, I don't see any downside. The changes he is proposing are tiny and simple and backward compatible.
Well, the main downside I see is that I don't agree that the proposed changes are the best possible approach. Implementing them in the built in exceptions therefore makes it harder for people to choose better approaches (or at least encourages them not to look for better approaches). There's no way I'd consider that e.args[0] as a better way to get the name that triggered a NameError than e.name. This seems to me to be something that should be experimented with and proven outside of the stdlib, before we rush to change the language. I don't see anything that makes that impossible. Paul
On 2017-07-03 22:44, Paul Moore wrote:
Maybe exceptions could put any keyword arguments into the instance's __dict__: class BaseException: def __init__(self, *args, **kwargs): self.args = args self.__dict__.update(kwargs) You could then raise: raise NameError('name {!a} is not defined', name='foo') and catch: try: ... except NameError as e: print('{}: nicht gefunden.'.format(e.name))
All, My primary concern is gaining access to the components that make up the messages. I am not hung up on the implementation. I just proposed the minimum that I thought would resolve the issue and introduce the least amount of risk. Concerning MRAB's idea of making the named arguments attributes, I am good with it. I considered it, though I was thinking of using __getattr__(), but thought that perhaps it was a step to far for the BaseException. -Ken On Tue, Jul 04, 2017 at 12:04:16AM +0100, MRAB wrote:
Paul Moore wrote:
I've no doubt that the current situation encourages people to be lazy -- I know, because I'm guilty of it myself! Writing a few extra lines to store attributes away and format them in __str__ might not seem like much, but in most cases those lines are of no direct benefit to the person writing the code, so there's little motivation to do it right. -- Greg
On 4 July 2017 at 09:46, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
So isn't this a variant of the argument that defining well-behaved classes currently involves writing too much boilerplate code, and the fact that non-structured exceptions are significantly easier to define than structured ones is just an example of that more general problem? I personally don't think there's anything all *that* special about exceptions in this case - they're just a common example of something that would be better handled as a "data record" type, but is commonly handled as an opaque string because they're so much easier to define that way. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4 July 2017 at 06:08, Nick Coghlan <ncoghlan@gmail.com> wrote:
Yes, that's what I was (badly) trying to say. I agree that we could hide a lot of the boilerplate in BaseException (which is what Ken was suggesting) but I don't believe we yet know the best way to write that boilerplate, so I'm reluctant to put anything in the stdlib until we do know better. For now, experimenting with 3rd party "rich exception" base classes seems a sufficient option. It's possible that more advanced methods than simply using a base class may make writing good exception classes even easier, but I'm not sure I've seen any evidence of that yet. Paul
On Mon, Jul 03, 2017 at 10:44:20PM +0100, Paul Moore wrote:
Right -- and not only does that go against the exception PEP https://www.python.org/dev/peps/pep-0352/ but it's still fragile unless the callee guarantees to always pass the values you want in the order you expect. And even if they do make that promise, named attributes are simply better than positional arguments. Who wants to write: err.args[3] # or is it 4, I always have to look it up... when you could write: err.column instead? -- Steve
On Mon, Jul 03, 2017 at 01:46:14PM -0600, Jeff Walker wrote:
I disagree: you should be using a JSON library which offers the line and column number as part of its guaranteed, supported API for dealing with bad JSON. If those numbers aren't part of the library API, then it is just as much of a fragile hack to extract them from exception.args[] as to scrape them from the error message. Suppose you write this: print(err.args) which gives: [35, 42, 'Unterminated string starting at: line 42 column 36'] Without parsing the error string, how do you know whether 35 is the line number or the column number or the character offset or something else that you didn't think of? Even when the information is there, in the exception args, it is still a fragile hack to extract them if they aren't part of the API.
I don't know what this code smell is that people keep referring to, but to me, that code would smell.
This is a good example. I like this idea. I think that a good place to start would be setting the right example in the standard library: IndexError could have the offending index, KeyError the offending key, TypeError the offending type, etc. On Monday, July 3, 2017 at 3:49:23 PM UTC-4, Jeff Walker wrote:
On 3 July 2017 at 20:46, Jeff Walker <jeff.walker00@yandex.com> wrote:
Possibly. I hadn't reread the original email. Having done so, I'm confused as to how the proposal and the example are related. The proposal makes no difference unless the places where (for example) NameError are raised are changed. But the proposal doesn't suggest changing how the interpreter raises NameError. So how will the proposal make a difference? I'd understood from the example that Ken's need was to be able to find the name that triggered the NameError. His proposal doesn't do that (unless we are talking about user-raised NameError exceptions, as opposed to ones the interpreter raises - in which case why not just use a user-defined exception? So I'm -1 on his proposal, as I don't see anything in it that couldn't be done in user code for user-defined exceptions, and there's nothing in the proposal suggesting a change in how interpreter-raised exceptions are created.
Then yes, you need to know the API presented by the exception. Projects (and the core interpreter) are not particularly good at documenting (or designing) the API for their exceptions, but that doesn't alter the fact that exceptions are user-defined classes and as such do have an API. I'd be OK with arguments that the API of built in exceptions as raised by the interpreter could be improved. Indeed, I thought that was Ken's proposal. But his proposal seems to be that if we add a ___str__ method to BaseException, that will somehow automatically improve the API of all other exceptions. To quote Ken:
Alternatively, I could argue that code which uses print(str(e)) as its exception handling isn't very well written, and the fact that people do this is what leads to people passing only one argument to their exceptions when creating them. Look, I see that there might be something that could be improved here. But I don't see an explanation of how, if we implement just the proposed change to BaseException, the user code that Ken's quoting as having a problem could be improved. There seems to be an assumption of "and because of that change, people raising exceptions would change what they do". Frankly, no they wouldn't. There's no demonstrated benefit for them, and they'd have to maintain a mess of backward compatibility code. So why would they bother? Anyway, you were right that I'd replied to just the example, not the original proposal. I apologise for that, I should have read the thread more carefully. But if I had done so, it wouldn't have made much difference - I still don't see a justification for the proposed change. Paul
Paul, Indeed, nothing gets better until people change the way they do their exceptions. Ken's suggested enhancement to BaseException does not directly solve the problem, but it removes the roadblocks that discourage people from passing the components to the message. Seems to me that to address this problem, four things are needed: 1. Change BaseException. This allows people to pass the components to the message without ruining str(e). 2. A PEP that educates the Python community on this better way of writing exceptions. 3. Changes to the base language and standard library to employ the new approach. These would changes would be quite small and could be done opportunistically. 4. Time. Over time things will just get better as more people see the benefits of this new approach to exceptions and adopt it. And if they don't, we are no worse off than we are now. And frankly, I don't see any downside. The changes he is proposing are tiny and simple and backward compatible. -Jeff 03.07.2017, 14:23, "Paul Moore" <p.f.moore@gmail.com>:
On 3 July 2017 at 21:56, Jeff Walker <jeff.walker00@yandex.com> wrote:
As noted, I disagree that people are not passing components because str(e) displays them the way it does. But we're both just guessing at people's motivations, so there's little point in speculating.
I dispute this is the essential place to start. If nothing else, the proposed approach encourages people to use a position-based "args" attribute for exceptions, rather than properly named attributes.
2. A PEP that educates the Python community on this better way of writing exceptions.
Educating the community can be done right now, and doesn't need a PEP. Someone could write a blog post, or an article, that explains how to code exception classes, how to create the exceptions, and how client code can/should use the API. This can be done now, all you need to do is to start with "at the moment, BaseException doesn't implement these features, so you should create an application-specific base exception class to minimise duplication of code". If project authors take up the proposed approach, then that makes a good argument for moving the supporting code into the built in BaseException class.
And I've never said that there's a problem with these. Although I do dispute that using an args list is the best approach here - I'd much rather see NameError instances have a "name" attribute that had the name that couldn't be found as its value. Opportunistic changes to built in exceptions can implement the most appropriate API for the given exception - why constrain such changes to a "lowest common denominator" API that is ideal for no-one? class NameError(BaseException): def __init__(self, name): self.name = name def __str__(self): return f"name '{self.name}' is not defined" Of course, that's not backward compatible as it stands, but it could probably be made so, just as easily as implementing the proposed solution.
The same could be said of any improved practice. And I agree, let's encourage people to learn to write better code, and promote good practices. There's definitely no reason not to do this.
And frankly, I don't see any downside. The changes he is proposing are tiny and simple and backward compatible.
Well, the main downside I see is that I don't agree that the proposed changes are the best possible approach. Implementing them in the built in exceptions therefore makes it harder for people to choose better approaches (or at least encourages them not to look for better approaches). There's no way I'd consider that e.args[0] as a better way to get the name that triggered a NameError than e.name. This seems to me to be something that should be experimented with and proven outside of the stdlib, before we rush to change the language. I don't see anything that makes that impossible. Paul
On 2017-07-03 22:44, Paul Moore wrote:
Maybe exceptions could put any keyword arguments into the instance's __dict__: class BaseException: def __init__(self, *args, **kwargs): self.args = args self.__dict__.update(kwargs) You could then raise: raise NameError('name {!a} is not defined', name='foo') and catch: try: ... except NameError as e: print('{}: nicht gefunden.'.format(e.name))
All, My primary concern is gaining access to the components that make up the messages. I am not hung up on the implementation. I just proposed the minimum that I thought would resolve the issue and introduce the least amount of risk. Concerning MRAB's idea of making the named arguments attributes, I am good with it. I considered it, though I was thinking of using __getattr__(), but thought that perhaps it was a step to far for the BaseException. -Ken On Tue, Jul 04, 2017 at 12:04:16AM +0100, MRAB wrote:
Paul Moore wrote:
I've no doubt that the current situation encourages people to be lazy -- I know, because I'm guilty of it myself! Writing a few extra lines to store attributes away and format them in __str__ might not seem like much, but in most cases those lines are of no direct benefit to the person writing the code, so there's little motivation to do it right. -- Greg
On 4 July 2017 at 09:46, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
So isn't this a variant of the argument that defining well-behaved classes currently involves writing too much boilerplate code, and the fact that non-structured exceptions are significantly easier to define than structured ones is just an example of that more general problem? I personally don't think there's anything all *that* special about exceptions in this case - they're just a common example of something that would be better handled as a "data record" type, but is commonly handled as an opaque string because they're so much easier to define that way. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4 July 2017 at 06:08, Nick Coghlan <ncoghlan@gmail.com> wrote:
Yes, that's what I was (badly) trying to say. I agree that we could hide a lot of the boilerplate in BaseException (which is what Ken was suggesting) but I don't believe we yet know the best way to write that boilerplate, so I'm reluctant to put anything in the stdlib until we do know better. For now, experimenting with 3rd party "rich exception" base classes seems a sufficient option. It's possible that more advanced methods than simply using a base class may make writing good exception classes even easier, but I'm not sure I've seen any evidence of that yet. Paul
On Mon, Jul 03, 2017 at 10:44:20PM +0100, Paul Moore wrote:
Right -- and not only does that go against the exception PEP https://www.python.org/dev/peps/pep-0352/ but it's still fragile unless the callee guarantees to always pass the values you want in the order you expect. And even if they do make that promise, named attributes are simply better than positional arguments. Who wants to write: err.args[3] # or is it 4, I always have to look it up... when you could write: err.column instead? -- Steve
On Mon, Jul 03, 2017 at 01:46:14PM -0600, Jeff Walker wrote:
I disagree: you should be using a JSON library which offers the line and column number as part of its guaranteed, supported API for dealing with bad JSON. If those numbers aren't part of the library API, then it is just as much of a fragile hack to extract them from exception.args[] as to scrape them from the error message. Suppose you write this: print(err.args) which gives: [35, 42, 'Unterminated string starting at: line 42 column 36'] Without parsing the error string, how do you know whether 35 is the line number or the column number or the character offset or something else that you didn't think of? Even when the information is there, in the exception args, it is still a fragile hack to extract them if they aren't part of the API.
I don't know what this code smell is that people keep referring to, but to me, that code would smell.
This is a good example. I like this idea. I think that a good place to start would be setting the right example in the standard library: IndexError could have the offending index, KeyError the offending key, TypeError the offending type, etc. On Monday, July 3, 2017 at 3:49:23 PM UTC-4, Jeff Walker wrote:
participants (8)
-
Greg Ewing -
Jeff Walker -
Ken Kundert -
MRAB -
Neil Girdhar -
Nick Coghlan -
Paul Moore -
Steven D'Aprano