Re: [Python-ideas] Consistent programming error handling idiom
On Fri, Apr 08, 2016 at 11:24:15AM -0700, rian@thelig.ht wrote: I want to live in a world where I can do this:
while cbs: cb = cbs.pop() try: cb() except Exception as e: logging.exception("In main loop") if is_a_bug(e): raise SystemExit() from e
And I want a pony :-)
So long as Python allows people to write code like this:
# A deliberately silly example if len(seq) == 0: raise ImportError("assertion fails")
you cannot expect to automatically know what an exception means without any context of where it came from and why it happened. The above example is silly, but it doesn't take much effort to come up with more serious ones:
- an interpreter written in Python may raise SyntaxError, which is not a bug in the interpreter;
- a test framework may raise AssertionError for a failed test, which is not a bug in the framework;
- a function may raise MemoryError if the call *would* run out of memory, but without actually running out of memory; consequently it is not a fatal error, while another function may raise the same MemoryError because it actually did fatally run out of memory.
Effectively, you want the compiler to Do What I Mean when it comes to exceptions. DWIM may, occasionally, be a good idea in applications, but I maintain it is never a good idea in a programming language.
I think you're misinterpreting me. I don't want a pony and I don't want a sufficiently smart compiler. I want a consistent opt-in idiom with community consensus. I want a clear way to express that an exception is an error and not an internal bug. It doesn't have to catch 100% of cases, the idiom just needs to approach consistency across all Python libraries that I may import. If the programmer doesn't pay attention to the idiom, then is_a_bug() will never return true (or not True if it's is_not_a_bug()). AssertionError is already unambiguous, I'm sure there are other candidates as well. I'm not the first or only one to want something like this http://blog.tsunanet.net/2012/10/pythons-screwed-up-exception-hierarchy.html But I can see the pile on has begun and my point is lost. Maybe this will get added in ten or twenty years.
On 04/08/2016 02:25 PM, Rian Hunter wrote:
I want a consistent opt-in idiom with community consensus. I want a clear way to express that an exception is an error and not an internal bug. It doesn't have to catch 100% of cases, the idiom just needs to approach consistency across all Python libraries that I may import.
But I can see the pile on has begun and my point is lost. Maybe this will get added in ten or twenty years.
Then come up with one, test it out, and come back with "Here's a cool new idiom, and this is why it's better" post. -- ~Ethan~
On Sat, Apr 9, 2016 at 7:25 AM, Rian Hunter <rian@thelig.ht> wrote:
I want a consistent opt-in idiom with community consensus. I want a clear way to express that an exception is an error and not an internal bug. It doesn't have to catch 100% of cases, the idiom just needs to approach consistency across all Python libraries that I may import.
If the programmer doesn't pay attention to the idiom, then is_a_bug() will never return true (or not True if it's is_not_a_bug()). AssertionError is already unambiguous, I'm sure there are other candidates as well.
I'm not the first or only one to want something like this http://blog.tsunanet.net/2012/10/pythons-screwed-up-exception-hierarchy.html
The problem with posts like that is that it assumes there's some kind of defined category of "stuff you always want to catch" vs "stuff you never want to catch". This simply isn't the case. You ONLY catch the exceptions you truly understand. Everything else, you leave. There's seldom a need to catch a broad-but-specific subset of exceptions, and those needs aren't entirely consistent, so they fundamentally cannot be codified into the hierarchy. ChrisA
Rian Hunter wrote:
I want a consistent opt-in idiom with community consensus. I want a clear way to express that an exception is an error and not an internal bug.
There's already a convention for this. Exceptions derived from EnvironmentError (OSError in python 3) usually result from something the user did wrong. Anything else that escapes lower-level exception handling is probably a bug. So my top-level exception handlers usually look like this: try: ... except EnvironmentError as e: # Display an appropriate error message to the user # If it's something interactive, go back for another # request, otherwise exit Anything else is left to propagate and generate a traceback. It's not perfect, but it works well enough most of the time. If I find a case where some non-bug exception gets raised that doesn't derive from EnvironmentError, I fix things so that it gets caught somewhere lower down and re-raised as an EnvironmentError. So the rule you seem to be after is probably "If it's likely to be a user error, derive it from EnvironmentError, otherwise don't." I think that's about the best that can be done, considering that library code can't always know whether a given exception is a user error or a bug, because it depends on what the calling code is trying to accomplish. For example, some_library.read_file("confug.ini") will probably produce an OSError (file not found) even though it's the programmer's fault for misspelling "config". -- Greg
On Sun, Apr 10, 2016 at 05:38:41PM +1200, Greg Ewing wrote:
Rian Hunter wrote:
I want a consistent opt-in idiom with community consensus. I want a clear way to express that an exception is an error and not an internal bug.
There's already a convention for this. Exceptions derived from EnvironmentError (OSError in python 3) usually result from something the user did wrong. Anything else that escapes lower-level exception handling is probably a bug.
I don't think this is a valid description of EnvironmentError. I don't think you can legitimately say it "usually" comes from user error. Let's say I have an application which, on startup, looks for config files in various places. If they exist and are readable, the application uses them. If they don't exist or aren't readable, an EnvironmentError will be generated (say, IOError). This isn't a user error, and my app can and should just ignore such missing config files. Then the application goes to open a user-specified data file. If the file doesn't exist, or can't be read, that will generate an EnvironmentError, but it isn't one that should be logged. (Who wants to log every time the user mispells a file name, or tries to open a file they don't have permission for?) In an interactive application, the app should display an error message and then wait for more commands. In a batch or command-line app, the application should exit. So treatment of the error depends on *what sort of application* you have, not just the error itself. Then the application phones home, looking for updates to download, uploading the popularity statistics of the most commonly used commands, bug reports, etc. What if it can't contact the home server? That's most likely an EnvironmentError too, but it's not a user-error. Oops, I spelled the URL of my server "http://myserver.com.ua" instead of .au. So that specific EnvironmentError is a programming bug. (No wonder I haven't had any popularity stats uploaded...) So EnvironmentError can represent any of: - situation normal, not actually an error at all; - a non-fatal user-error; - a fatal user-error; - transient network errors that will go away on their own; - programming bugs.
It's not perfect, but it works well enough most of the time. If I find a case where some non-bug exception gets raised that doesn't derive from EnvironmentError, I fix things so that it gets caught somewhere lower down and re-raised as an EnvironmentError.
That's ... rather strange. As in: EnvironmentError("substring not found") for an unsuccessful search? -- Steve
Steven D'Aprano wrote:
Let's say I have an application which, on startup, looks for config files in various places. If they exist and are readable, the application uses them. If they don't exist or aren't readable, an EnvironmentError will be generated (say, IOError). This isn't a user error, and my app can and should just ignore such missing config files.
Of course you can't assume that an EnvironmentError *anywhere* in the program represents a user error. I was just suggesting a heuristic for the *top level* exception hander to use. Making that heuristic work well requires cooperation from the rest of the code. In this case, it means wrapping the code that reads config files to catch file-not-found errors. You're going to have to do that anyway if you want to carry on with the rest of the processing.
That's ... rather strange. As in:
EnvironmentError("substring not found")
for an unsuccessful search?
I might put a bit more information in there to help the user. The idea is that whatever I put in the EnvironmentError will end up getting displayed to the user as an error message by my top-level exception handler. I might also use a subclass of EnvironmentError if I want to be able to catch it specifically, but that's not strictly necessary. I don't show the user the exception class name, only its argument. -- Greg
participants (5)
-
Chris Angelico -
Ethan Furman -
Greg Ewing -
Rian Hunter -
Steven D'Aprano