Mailman 3 Exceptions with Message Templates - Python-ideas

Exceptions with Message Templates

Ryan Fox

8 Aug 2019 8 Aug '19

3:52 p.m.

Exception definitions in Python are not great. There are two main ways they're used in the code that I've read: 1) By far the most common:

...

...
...
class MyException(Exception): ... pass raise MyException(f'Bad thing happened during {action} in {context}')

2) Much rarer:

...

...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in {context}') raise MyException(current_action, current_context)

Version 1 is quick and easy, but messages skew and become inconsistent as they're copied from place to place. The Python standard library isn't immune to this either. Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother. My proposal is a new exception class as the preferred base for user-defined exceptions:

...

...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

I have a reference implementation, implemented in an almost trivial amount of pure Python code: https://github.com/rcfox/exception-template/blob/master/exception_template/e... So why am I bothering you with this? I would like to see this become the preferred method of defining new exceptions as recommended by the Python documentation. This would also require adding ExceptionTemplate as a built-in exception type. I will grant that ExceptionTemplate seems a little bit magical, but all of the required arguments are explicitly laid out to the user in the 'message' field, and format strings should be a familiar concept to most users. I've also included some tests that show that you can still treat these exceptions as regular classes: https://github.com/rcfox/exception-template/blob/master/tests/test.py#L53

Attachments:

attachment.htm (text/html — 2.6 KB)

Show replies by date

Brett Cannon

8 Aug 8 Aug

6:02 p.m.

Three things. One, this isn't backwards-compatible as you are not passing any details down into Exception.__init__() to make sure that BaseException.args gets populated. Two, you can simplify your code by using str.format_map() instead of the string module. Three, I don't see enough overhead from version 2 to your version 3 since you're only saving a line of code so say there's that much boilerplate. But the best way to prove me wrong is to release this on PyPI and see if people use it. But without community use proving people want this we won't be up for adding a new built-in exception as you're asking every book on Python to be rewritten to cover this which is no small thing. On Thu, Aug 8, 2019 at 8:56 AM Ryan Fox wrote:

...

Exception definitions in Python are not great. There are two main ways they're used in the code that I've read:

1) By far the most common:

...
...
...
class MyException(Exception): ... pass raise MyException(f'Bad thing happened during {action} in {context}')

2) Much rarer:

...
...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in {context}') raise MyException(current_action, current_context)

Version 1 is quick and easy, but messages skew and become inconsistent as they're copied from place to place. The Python standard library isn't immune to this either.

Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother.

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

I have a reference implementation, implemented in an almost trivial amount of pure Python code: https://github.com/rcfox/exception-template/blob/master/exception_template/e...

So why am I bothering you with this? I would like to see this become the preferred method of defining new exceptions as recommended by the Python documentation. This would also require adding ExceptionTemplate as a built-in exception type.

I will grant that ExceptionTemplate seems a little bit magical, but all of the required arguments are explicitly laid out to the user in the 'message' field, and format strings should be a familiar concept to most users.

I've also included some tests that show that you can still treat these exceptions as regular classes: https://github.com/rcfox/exception-template/blob/master/tests/test.py#L53

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GEO2WZ... Code of Conduct: http://python.org/psf/codeofconduct/

Ryan Fox

6:37 p.m.

Thanks for the comments. I'm not really sure what you mean with regards to backwards compatibility. Would it suffice to have ExceptionTemplate.__init__ accept *args and pass that into super().__init__? I see that BaseException does something with args in __new__ and __init__, but I'm not very familiar with the Python internals. How does BaseException deal with kwargs? str.format_map() would make the formatting simpler, but I also use string.Formatter.parse() to in ExceptionTemplate._error_check() to find the required arguments. I don't see an alternative for this on str. You're right about the third thing. I hadn't even considered the books! I have released on PyPI as 'exception-template' and we'll see what happens. One person commented on Reddit that they liked the idea but would probably copy the code directly into their project rather than require an additional external dependency. I guess I was hoping to get a feel of whether this was something that appealed to this audience and/or could ever be accepted, as well as to refine the idea and implementation. On Thu, Aug 8, 2019 at 2:03 PM Brett Cannon wrote:

...

Three things. One, this isn't backwards-compatible as you are not passing any details down into Exception.__init__() to make sure that BaseException.args gets populated.

Two, you can simplify your code by using str.format_map() instead of the string module.

Three, I don't see enough overhead from version 2 to your version 3 since you're only saving a line of code so say there's that much boilerplate. But the best way to prove me wrong is to release this on PyPI and see if people use it. But without community use proving people want this we won't be up for adding a new built-in exception as you're asking every book on Python to be rewritten to cover this which is no small thing.

On Thu, Aug 8, 2019 at 8:56 AM Ryan Fox wrote:

...
Exception definitions in Python are not great. There are two main ways they're used in the code that I've read:

1) By far the most common:

...
...
...
class MyException(Exception): ... pass raise MyException(f'Bad thing happened during {action} in {context}')

2) Much rarer:

...
...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in {context}') raise MyException(current_action, current_context)

Version 1 is quick and easy, but messages skew and become inconsistent as they're copied from place to place. The Python standard library isn't immune to this either.

Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother.

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

I have a reference implementation, implemented in an almost trivial amount of pure Python code: https://github.com/rcfox/exception-template/blob/master/exception_template/e...

So why am I bothering you with this? I would like to see this become the preferred method of defining new exceptions as recommended by the Python documentation. This would also require adding ExceptionTemplate as a built-in exception type.

I will grant that ExceptionTemplate seems a little bit magical, but all of the required arguments are explicitly laid out to the user in the 'message' field, and format strings should be a familiar concept to most users.

I've also included some tests that show that you can still treat these exceptions as regular classes: https://github.com/rcfox/exception-template/blob/master/tests/test.py#L53

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GEO2WZ... Code of Conduct: http://python.org/psf/codeofconduct/

Brett Cannon

9 Aug 9 Aug

5 p.m.

On Thu, Aug 8, 2019 at 11:38 AM Ryan Fox wrote:

...

Thanks for the comments.

I'm not really sure what you mean with regards to backwards compatibility. Would it suffice to have ExceptionTemplate.__init__ accept *args and pass that into super().__init__? I see that BaseException does something with args in __new__ and __init__, but I'm not very familiar with the Python internals. How does BaseException deal with kwargs?

It doesn't deal with them because it won't accept them. :)

...

str.format_map() would make the formatting simpler, but I also use string.Formatter.parse() to in ExceptionTemplate._error_check() to find the required arguments. I don't see an alternative for this on str.

You can also use str.format(**kwargs) if you want to be strict about what you accept in terms of keyword arguments while being generic in your implementation. I also just realized that your class won't have any helpful introspection on the call signature either, so all users of your class will have to document in the docstring very clearly what is expected in the constructor call as their code editors won't be able to tell them and help()/inspect won't be able to help either.

...

You're right about the third thing. I hadn't even considered the books! I have released on PyPI as 'exception-template' and we'll see what happens. One person commented on Reddit that they liked the idea but would probably copy the code directly into their project rather than require an additional external dependency.

Yep, this is small enough to easily fit into a person's personal toolbox of handy code.

...

I guess I was hoping to get a feel of whether this was something that appealed to this audience and/or could ever be accepted, as well as to refine the idea and implementation.

On Thu, Aug 8, 2019 at 2:03 PM Brett Cannon wrote:

...
Three things. One, this isn't backwards-compatible as you are not passing any details down into Exception.__init__() to make sure that BaseException.args gets populated.

Two, you can simplify your code by using str.format_map() instead of the string module.

Three, I don't see enough overhead from version 2 to your version 3 since you're only saving a line of code so say there's that much boilerplate. But the best way to prove me wrong is to release this on PyPI and see if people use it. But without community use proving people want this we won't be up for adding a new built-in exception as you're asking every book on Python to be rewritten to cover this which is no small thing.

On Thu, Aug 8, 2019 at 8:56 AM Ryan Fox wrote:

...
Exception definitions in Python are not great. There are two main ways they're used in the code that I've read:

1) By far the most common:

...
...
...
class MyException(Exception): ... pass raise MyException(f'Bad thing happened during {action} in {context}')

2) Much rarer:

...
...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in {context}') raise MyException(current_action, current_context)

Version 1 is quick and easy, but messages skew and become inconsistent as they're copied from place to place. The Python standard library isn't immune to this either.

Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother.

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

I have a reference implementation, implemented in an almost trivial amount of pure Python code: https://github.com/rcfox/exception-template/blob/master/exception_template/e...

So why am I bothering you with this? I would like to see this become the preferred method of defining new exceptions as recommended by the Python documentation. This would also require adding ExceptionTemplate as a built-in exception type.

I will grant that ExceptionTemplate seems a little bit magical, but all of the required arguments are explicitly laid out to the user in the 'message' field, and format strings should be a familiar concept to most users.

I've also included some tests that show that you can still treat these exceptions as regular classes: https://github.com/rcfox/exception-template/blob/master/tests/test.py#L53

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GEO2WZ... Code of Conduct: http://python.org/psf/codeofconduct/

Andrew Barnert

8 Aug 8 Aug

8:12 p.m.

On Aug 8, 2019, at 08:52, Ryan Fox wrote:

...

...
...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in {context}') raise MyException(current_action, current_context)

...

Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother.

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

Does this really save boilerplate? Your magic template has apparently turned positional-or-keyword parameters into keyword-only. This means that every exception class that you’d normally construct with positional args today (which apparently includes even your example, since that’s what you did in version 1 and 2) now requires keyword boilerplate on every line where you construct one. And, since you generally construct and use a class many times, but only define it once, that’s adding a lot more boilerplate than it’s saving. Also, even in the definition, while you are saving one line, you’re doing it by replacing a standard idiom used all over Python that has an obvious meaning, which nobody ever forgets how to write, with magic involving a special name that’s unlike anything else in the stdlib, which many people will forget how to write, forcing them to look up the docs or find an example whenever they add the first new exception to a new module. Also, this doesn’t fill in the exception’s stored args. Is there a way to fix that without requiring more boilerplate? Remember, stored args are positional, not keywords. So, even if you can come up with some magic (e.g., parsing the format string to sort the keywords in the order of first appearance), that only helps the interpreter, not a human reader who sees exc.args[2] in an except clause for an exception created with three keyword args and has to guess which one is #2. Without explicitly listing the args in order somewhere, how do you solve that? And is there a way to list the named args of a function in order that’s less boilerplate than (but just as understandable as) a def statement?

Ryan Fox

10:01 p.m.

...

Your magic template has apparently turned positional-or-keyword parameters into keyword-only. This means that every exception class that you’d normally construct with positional args today (which apparently includes even your example, since that’s what you did in version 1 and 2) now requires keyword boilerplate on every line where you construct one. And, since you generally construct and use a class many times, but only define it once, that’s adding a lot more boilerplate than it’s saving.

In my mind, you're filling in a format string, so of course you're going to give the values by name. But I can see how that would seem like a step backwards compared to an exception with positional arguments. And yes, taking arguments in the order they appear in the string is too much magic and not intuitive.

...

Also, even in the definition, while you are saving one line, you’re doing it by replacing a standard idiom used all over Python that has an obvious meaning, which nobody ever forgets how to write, with magic involving a special name that’s unlike anything else in the stdlib, which many people will forget how to write, forcing them to look up the docs or find an example whenever they add the first new exception to a new module.

...

Also, this doesn’t fill in the exception’s stored args. Is there a way to fix that without requiring more boilerplate? Remember, stored args are

That seems overly pessimistic. If you can remember to extend a class, it's not too much of a stretch to remember what aspect of it that you're defining. positional, not keywords. So, even if you can come up with some magic (e.g., parsing the format string to sort the keywords in the order of first appearance), that only helps the interpreter, not a human reader who sees exc.args[2] in an except clause for an exception created with three keyword args and has to guess which one is #2. Without explicitly listing the args in order somewhere, how do you solve that? And is there a way to list the named args of a function in order that’s less boilerplate than (but just as understandable as) a def statement? I don't see why you would want to access arguments by their position. Why wouldn't you access them by name? ie: exc.action The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion. On Thu, Aug 8, 2019 at 4:12 PM Andrew Barnert wrote:

...

On Aug 8, 2019, at 08:52, Ryan Fox wrote:

...
...
...
...
class MyException(Exception): ... def __init__(self, action, context): ... super().__init__(f'Bad thing happened during {action} in

{context}')

...
...
...
...
raise MyException(current_action, current_context)

...

...
Version 2 looks simple enough to do, but all of the repetitious boilerplate adds up when several exception types need to be defined. (And it's even worse when you want all of your code to include typing annotations.) Most people don't bother.

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

Does this really save boilerplate?

Your magic template has apparently turned positional-or-keyword parameters into keyword-only. This means that every exception class that you’d normally construct with positional args today (which apparently includes even your example, since that’s what you did in version 1 and 2) now requires keyword boilerplate on every line where you construct one. And, since you generally construct and use a class many times, but only define it once, that’s adding a lot more boilerplate than it’s saving.

Also, even in the definition, while you are saving one line, you’re doing it by replacing a standard idiom used all over Python that has an obvious meaning, which nobody ever forgets how to write, with magic involving a special name that’s unlike anything else in the stdlib, which many people will forget how to write, forcing them to look up the docs or find an example whenever they add the first new exception to a new module.

Also, this doesn’t fill in the exception’s stored args. Is there a way to fix that without requiring more boilerplate? Remember, stored args are positional, not keywords. So, even if you can come up with some magic (e.g., parsing the format string to sort the keywords in the order of first appearance), that only helps the interpreter, not a human reader who sees exc.args[2] in an except clause for an exception created with three keyword args and has to guess which one is #2. Without explicitly listing the args in order somewhere, how do you solve that? And is there a way to list the named args of a function in order that’s less boilerplate than (but just as understandable as) a def statement?

Andrew Barnert

11:07 p.m.

On Aug 8, 2019, at 15:01, Ryan Fox wrote:

...

I don't see why you would want to access arguments by their position.

Because that’s the way it’s worked since Python 1.x, and there’s tons of existing code that expects it, including the default __str__ and __repr__ for exceptions and the code that formats tracebacks.

...

The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

Yes they do. Try it: >>> e = InputError('[2+3)', 'mismatched brackets') >>> e.args ('[2+3)', 'mismatched brackets') >>> e InputError('[2+3)', 'mismatched brackets') If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial. Also, notice that the tutorial examples don’t even try to create a formatted message; they expect that the type name and the args will be enough for debugging. I’m not sure that’s a great design, but it means that your intended fix only solves a problem they didn’t even have in the first place.

...

So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion.

I agree that it would be nice for more people to move their message formatting into the class, but you need a design that encourages that without fighting against the fact that exception args are positional, and I’m not sure what that looks like. And I don’t think it’s your implementation that’s bad (it seems to do what it says perfectly well), but that the design doesn’t work. Of course if you were designing a new language (or a new library from builtins up for the same language), this would be easy. Exceptions would look like (maybe be) @dataclasses, storing their arguments by name and generating repr/str/traceback that takes that into account, and all of them would actually store the relevant values instead of half of them making you parse it out of the first arg, and there would be a special message property or method instead of args[0] being sort of special but not special enough, and so on. And then, providing an easier way to create that message property would be an easy problem. But I think with the way Python is today, it’s not. Of course I’d be happy to be proven wrong on that. :)

Sebastian Kreft

9 Aug 9 Aug

12:21 a.m.

On Thu, Aug 8, 2019 at 7:09 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...

On Aug 8, 2019, at 15:01, Ryan Fox wrote:

I don't see why you would want to access arguments by their position.

Because that’s the way it’s worked since Python 1.x, and there’s tons of existing code that expects it, including the default __str__ and __repr__ for exceptions and the code that formats tracebacks.

I don't really understand what you mean here. This property was broken since ImportError started accepting keyword arguments. For example:

...

...
...
ImportError("message", name="name", path="path").args ('message',)

...

...
...
ImportError("message", "foo", name="name", path="path").args ('message', 'foo')

For the case of str and repr, one could just call super with the formatted message as the only positional argument. I suggest taking a look at PEP 473 https://www.python.org/dev/peps/pep-0473/ for ideas on why having structured arguments is a good idea.

...

The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

Yes they do. Try it:

>>> e = InputError('[2+3)', 'mismatched brackets') >>> e.args ('[2+3)', 'mismatched brackets') >>> e InputError('[2+3)', 'mismatched brackets')

If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

Also, notice that the tutorial examples don’t even try to create a formatted message; they expect that the type name and the args will be enough for debugging. I’m not sure that’s a great design, but it means that your intended fix only solves a problem they didn’t even have in the first place.

So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion.

I agree that it would be nice for more people to move their message formatting into the class, but you need a design that encourages that without fighting against the fact that exception args are positional, and I’m not sure what that looks like. And I don’t think it’s your implementation that’s bad (it seems to do what it says perfectly well), but that the design doesn’t work.

Of course if you were designing a new language (or a new library from builtins up for the same language), this would be easy. Exceptions would look like (maybe be) @dataclasses, storing their arguments by name and generating repr/str/traceback that takes that into account, and all of them would actually store the relevant values instead of half of them making you parse it out of the first arg, and there would be a special message property or method instead of args[0] being sort of special but not special enough, and so on. And then, providing an easier way to create that message property would be an easy problem. But I think with the way Python is today, it’s not.

Of course I’d be happy to be proven wrong on that. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5ULPVN... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft

Brett Cannon

4:54 p.m.

On Thu, Aug 8, 2019 at 5:24 PM Sebastian Kreft wrote:

...

On Thu, Aug 8, 2019 at 7:09 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...
On Aug 8, 2019, at 15:01, Ryan Fox wrote:

I don't see why you would want to access arguments by their position.

Because that’s the way it’s worked since Python 1.x, and there’s tons of existing code that expects it, including the default __str__ and __repr__ for exceptions and the code that formats tracebacks.

I don't really understand what you mean here. This property was broken since ImportError started accepting keyword arguments.

The property isn't broken for ImportError, it just isn't being given the keyword arguments because it didn't makes sense to pass them down with no information attached to it. The 'args' attribute still gets the message which is the key detail. -Brett

...

For example:

...
...
...
ImportError("message", name="name", path="path").args ('message',)

...
...
...
ImportError("message", "foo", name="name", path="path").args ('message', 'foo')

For the case of str and repr, one could just call super with the formatted message as the only positional argument.

I suggest taking a look at PEP 473 https://www.python.org/dev/peps/pep-0473/ for ideas on why having structured arguments is a good idea.

...
The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

Yes they do. Try it:

>>> e = InputError('[2+3)', 'mismatched brackets') >>> e.args ('[2+3)', 'mismatched brackets') >>> e InputError('[2+3)', 'mismatched brackets')

If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

Also, notice that the tutorial examples don’t even try to create a formatted message; they expect that the type name and the args will be enough for debugging. I’m not sure that’s a great design, but it means that your intended fix only solves a problem they didn’t even have in the first place.

So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion.

I agree that it would be nice for more people to move their message formatting into the class, but you need a design that encourages that without fighting against the fact that exception args are positional, and I’m not sure what that looks like. And I don’t think it’s your implementation that’s bad (it seems to do what it says perfectly well), but that the design doesn’t work.

Of course if you were designing a new language (or a new library from builtins up for the same language), this would be easy. Exceptions would look like (maybe be) @dataclasses, storing their arguments by name and generating repr/str/traceback that takes that into account, and all of them would actually store the relevant values instead of half of them making you parse it out of the first arg, and there would be a special message property or method instead of args[0] being sort of special but not special enough, and so on. And then, providing an easier way to create that message property would be an easy problem. But I think with the way Python is today, it’s not.

Of course I’d be happy to be proven wrong on that. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5ULPVN... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZOLTXV... Code of Conduct: http://python.org/psf/codeofconduct/

Sebastian Kreft

5:47 p.m.

On Fri, Aug 9, 2019 at 12:55 PM Brett Cannon wrote:

...

On Thu, Aug 8, 2019 at 5:24 PM Sebastian Kreft wrote:

...
On Thu, Aug 8, 2019 at 7:09 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...
On Aug 8, 2019, at 15:01, Ryan Fox wrote:

I don't see why you would want to access arguments by their position.

Because that’s the way it’s worked since Python 1.x, and there’s tons of existing code that expects it, including the default __str__ and __repr__ for exceptions and the code that formats tracebacks.

I don't really understand what you mean here. This property was broken since ImportError started accepting keyword arguments.

The property isn't broken for ImportError, it just isn't being given the keyword arguments because it didn't makes sense to pass them down with no information attached to it. The 'args' attribute still gets the message which is the key detail.

The "broken" property was alluding to Andrew's comment that exception arguments need to be positional and cannot/shouldn't be keywords and I was giving an example from the standard library in which we can pass keyword arguments which are not stored in the exception's arguments. Also note that my comment mentioned that passing the formatted message as the only argument to the super constructor would solve the compatibility problem.

...

-Brett

...
For example:

...
...
...
ImportError("message", name="name", path="path").args ('message',)

...
...
...
ImportError("message", "foo", name="name", path="path").args ('message', 'foo')

For the case of str and repr, one could just call super with the formatted message as the only positional argument.

I suggest taking a look at PEP 473 https://www.python.org/dev/peps/pep-0473/ for ideas on why having structured arguments is a good idea.

...
The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

Yes they do. Try it:

>>> e = InputError('[2+3)', 'mismatched brackets') >>> e.args ('[2+3)', 'mismatched brackets') >>> e InputError('[2+3)', 'mismatched brackets')

If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

Also, notice that the tutorial examples don’t even try to create a formatted message; they expect that the type name and the args will be enough for debugging. I’m not sure that’s a great design, but it means that your intended fix only solves a problem they didn’t even have in the first place.

So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion.

I agree that it would be nice for more people to move their message formatting into the class, but you need a design that encourages that without fighting against the fact that exception args are positional, and I’m not sure what that looks like. And I don’t think it’s your implementation that’s bad (it seems to do what it says perfectly well), but that the design doesn’t work.

Of course if you were designing a new language (or a new library from builtins up for the same language), this would be easy. Exceptions would look like (maybe be) @dataclasses, storing their arguments by name and generating repr/str/traceback that takes that into account, and all of them would actually store the relevant values instead of half of them making you parse it out of the first arg, and there would be a special message property or method instead of args[0] being sort of special but not special enough, and so on. And then, providing an easier way to create that message property would be an easy problem. But I think with the way Python is today, it’s not.

Of course I’d be happy to be proven wrong on that. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5ULPVN... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZOLTXV... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft

Ryan Fox

7 p.m.

...

If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

Between the links I've found, none of them refer to BaseException.__new__ setting up args, and the references to args make it sound like an optional thing or something the built-in exceptions handle themselves. https://docs.python.org/3/library/exceptions.html https://docs.python.org/3/c-api/exceptions.html https://docs.python.org/3/tutorial/errors.html Okay, so this __new__/__init__ divide is sort of a hack because you can't trust users to call super().__init__(...) on their user-defined exceptions? No judgement, just trying to wrap my head around this. It sounds like BaseException.args is mostly used for generating strings? Couldn't I just get away with overriding __str__, __repr__? I understand that existing built-in exception classes have some semantic value associated with arguments in given positions, but that wouldn't apply to new user-defined exceptions. And going forward, if you wanted to make your tool extract a value, it doesn't seem like it should make a big difference to access exc.arg[3] or exc.thing. I've also looked at giving the formatted message as args[0], but I haven't figured out how to handle a class with an explicit __init__ that defines a POSITIONAL_OR_KEYWORD argument, like this: https://github.com/rcfox/exception-template/blob/master/tests/test.py#L58 The 'key' argument ends up getting thrown in with kwargs. Maybe I just get rid of the error check for extra arguments given?

...

I also just realized that your class won't have any helpful introspection on the call signature either, so all users of your class will have to document in the docstring very clearly what is expected in the constructor call as their code editors won't be able to tell them and help()/inspect won't be able to help either.

I won't try to convince you that this is a good solution, but help()/inspect could at least be tricked with a metaclass that replaces the signature of __init__: class _Signaturizer(type): def __new__(cls, name, bases, classdict): result = super().__new__(cls, name, bases, classdict) args = set(name for _, name, _, _ in result.formatter.parse(result.message) if name) params = [inspect.Parameter(a, inspect.Parameter.KEYWORD_ONLY) for a in sorted(args)] result.__init__.__signature__ = inspect.signature(result.__init__).replace(parameters=params) return result class ExceptionTemplate(Exception, metaclass=_Signaturizer): # ... Of course, tools that rely on static analysis wouldn't be fooled. On Fri, Aug 9, 2019 at 1:57 PM Sebastian Kreft wrote:

...

On Fri, Aug 9, 2019 at 12:55 PM Brett Cannon wrote:

...
On Thu, Aug 8, 2019 at 5:24 PM Sebastian Kreft wrote:

...
On Thu, Aug 8, 2019 at 7:09 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...
On Aug 8, 2019, at 15:01, Ryan Fox wrote:

I don't see why you would want to access arguments by their position.

Because that’s the way it’s worked since Python 1.x, and there’s tons of existing code that expects it, including the default __str__ and __repr__ for exceptions and the code that formats tracebacks.

I don't really understand what you mean here. This property was broken since ImportError started accepting keyword arguments.

The property isn't broken for ImportError, it just isn't being given the keyword arguments because it didn't makes sense to pass them down with no information attached to it. The 'args' attribute still gets the message which is the key detail.

The "broken" property was alluding to Andrew's comment that exception arguments need to be positional and cannot/shouldn't be keywords and I was giving an example from the standard library in which we can pass keyword arguments which are not stored in the exception's arguments.

Also note that my comment mentioned that passing the formatted message as the only argument to the super constructor would solve the compatibility problem.

...
-Brett

...
For example:

...
...
...
ImportError("message", name="name", path="path").args ('message',)

...
...
...
ImportError("message", "foo", name="name", path="path").args ('message', 'foo')

For the case of str and repr, one could just call super with the formatted message as the only positional argument.

I suggest taking a look at PEP 473 https://www.python.org/dev/peps/pep-0473/ for ideas on why having structured arguments is a good idea.

...
The user-defined exceptions in the Python documentation don't pass arguments to the base class either: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

Yes they do. Try it:

>>> e = InputError('[2+3)', 'mismatched brackets') >>> e.args ('[2+3)', 'mismatched brackets') >>> e InputError('[2+3)', 'mismatched brackets')

If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

Also, notice that the tutorial examples don’t even try to create a formatted message; they expect that the type name and the args will be enough for debugging. I’m not sure that’s a great design, but it means that your intended fix only solves a problem they didn’t even have in the first place.

So let's go ahead and assume my implementation is flawed. The fact that people prefer to copy their format strings all over their projects implies that the current exception scheme is suboptimal. Can we agree on that? If not, there's no need to continue this discussion.

I agree that it would be nice for more people to move their message formatting into the class, but you need a design that encourages that without fighting against the fact that exception args are positional, and I’m not sure what that looks like. And I don’t think it’s your implementation that’s bad (it seems to do what it says perfectly well), but that the design doesn’t work.

Of course if you were designing a new language (or a new library from builtins up for the same language), this would be easy. Exceptions would look like (maybe be) @dataclasses, storing their arguments by name and generating repr/str/traceback that takes that into account, and all of them would actually store the relevant values instead of half of them making you parse it out of the first arg, and there would be a special message property or method instead of args[0] being sort of special but not special enough, and so on. And then, providing an easier way to create that message property would be an easy problem. But I think with the way Python is today, it’s not.

Of course I’d be happy to be proven wrong on that. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5ULPVN... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZOLTXV... Code of Conduct: http://python.org/psf/codeofconduct/

-- Sebastian Kreft _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/426DOB... Code of Conduct: http://python.org/psf/codeofconduct/

Andrew Barnert

11:31 p.m.

On Aug 9, 2019, at 12:00, Ryan Fox wrote:

...

...
If you’re wondering why this works, it’s because Error and InputError don’t override __new__. Which should make it obvious why a tutorial aimed at novices doesn’t get into the details, but that’s why Python has reference manuals instead of just a tutorial.

...

Okay, so this __new__/__init__ divide is sort of a hack because you can't trust users to call super().__init__(...) on their user-defined exceptions?

No, It’s not a hack; almost every immutable class that allows mutable subclasses in Python has a __new__ that the mutable subclasses rarely override, and a do-nothing __init__ that the mutable subclasses do. This does also save one remove one possibility for novices to make mistakes, and it also removes a line of boilerplate for most subclasses, but I don’t think either of those is the point; they’re just bonus. Anyway, you just define a new exception class the way the tutorial shows, and you get a round-trippable repr and proper tracebacks and args like magic; that’s all a novice needs to know—but it’s something we have to not break for them.

...

It sounds like BaseException.args is mostly used for generating strings? Couldn't I just get away with overriding __str__, __repr__?

Yes, but that means every class that doesn’t do the usual thing has to instead implement two or three extra methods, which is hardly reducing boilerplate, which was the goal.

...

I understand that existing built-in exception classes have some semantic value associated with arguments in given positions, but that wouldn't apply to new user-defined exceptions. And going forward, if you wanted to make your tool extract a value, it doesn't seem like it should make a big difference to access exc.arg[3] or exc.thing.

As I said, if we were designing a new language from scratch, we’d almost surely want exceptions to be more like dataclasses or namedtuples than things that store the positional args in a tuple. But we’re obviously not going to change all of the builtin exceptions, the hundreds of other exceptions in the stdlib and popular libraries, and the thousands in specific projects people are working on overnight. Adding named attributes, and structure in general, to exceptions is a great goal, but it’s a very long-term goal (that’s been going on since 3.0); a new feature that requires us to already be most of the way there before it’s useful isn’t as useful as it appears. Something that preserves the existing magic for args and repr and half of str, while adding the other half of str (and maybe named attributes too?), that would be a step forward. But adding the their half of str while breaking the existing magic (and making named attributes harder rather than easier) doesn’t seem like it is.

Kyle Lahnakoski

12 Aug 12 Aug

10 p.m.

On 2019-08-08 11:52, Ryan Fox wrote:

...

My proposal is a new exception class as the preferred base for user-defined exceptions:

...
...
...
class MyException(ExceptionTemplate): ... message = 'Bad thing happened during {action} in {context}' raise MyException(action=current_action, context=current_context)

The `message` string and the `MyException` class are 1-1; maybe you can remove more boilerplate and do it in one step:

...

raise ExceptionUsingTemplate( 'Bad thing happened during {action} in {context}', action=current_action, context=current_context )

Plus, give it a shorter name:

...

error( 'Bad thing happened during {action} in {context}', action=current_action, context=current_context )

You still have the "[templates] copied from place to place" problem; in those cases you raise the same type of error in many different locations, you can define a constant, and that constant represents the exception class:

...

BAD_THING_HAPPENED = 'Bad thing happened during {action} in {context}' ... error( BAD_THING_HAPPENED, action=current_action, context=current_context )

When catching exceptions, my code rarely cares about the specific type of exception, rather it only cares if an exception was raised. But in a few rare cases the exception handler is discerning:

...

except Exception as e: if BAD_THING_HAPPENED in e: # special case else: # all the other ways this failed

1718

Age (days ago)

1722

Last active (days ago)

List overview

Download

12 comments

5 participants

participants (5)

Andrew Barnert
Brett Cannon
Kyle Lahnakoski
Ryan Fox
Sebastian Kreft

Exceptions with Message Templates

tags

participants (5)