[Tutor] try..except - what about that ton of **Error statements?

[Steven D'Aprano]

On 22/05/13 23:31, boB Stepp wrote:
> On Wed, May 22, 2013 at 7:50 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On 22/05/13 15:46, Jim Mooney wrote:
>>>
>
> [...]
>
>>
>> But don't do this in real code! In real code, the rules you should apply
>> are:
>>
>>
>> 1) never hide programming errors by catching exceptions;
>>
>> 2) errors should only be caught if you can recover from them;
>>
>
> I have not made it to the point of catching errors. I am still at the
> stage of doing my best to prevent them from occurring. However, I try
> to follow all of the posts as best I can and I am not sure I
> understand what you mean here. You seem to be saying that there may be
> errors that you cannot eliminate, but also cannot do anything about if
> they happen to occur. Am I interpreting this correctly? Please
> clarify.


Correct. Some errors can be recovered from, e.g. if a website is not available, you can wait a few seconds and try again; if a file cannot be read, you can ask the user for a different file name. Some errors cannot reasonably be recovered from. If you ask the user for a number between 1 and 10, and they enter "hello", what are you going to do?

If your program is interactive, you can ask them again. But if it is non-interactive, you have no choice but to fail gracefully and display an error.

Some errors are *expected* -- users make mistakes, they don't read instructions, or misunderstand them, they enter garbage, make typos, etc. Web sites and networks go down temporarily, or respond slowly; disks fill up; etc. Good programs should deal with them gracefully.

Some errors are *unexpected*, in which case they should be identified as either a bug that needs to be fixed, or as an error condition which *should have been expected* but wasn't. If you catch every exception in sight, you will never be able to identify those unexpected errors and fix them.



>> 3) your job as a programmer is *not* to stop your program from raising an
>> error, but to make it behave correctly -- sometimes an error is the right
>> thing to do;
>>
>
> Also, can you clarify this point as well? When would you want your
> program to crash?

You don't *want* it to crash, you would rather it exit gracefully. But a crash is better than silently doing the wrong thing.

Some years ago there was a very unfortunate incident where a computer-controlled medical x-ray irradiated a large number of people with fatal doses of radiation. The machine should have halted with an error, but due to a combination of bad design, operator error, and programming bugs, instead it blasted the poor doomed patients with thousands of times the safe limit of radiation.

That's an extreme example, but the same principle applies in general. Your program has a job to do. If it cannot do that job correctly, it should own up to it rather than do something else. If you tell your program to save your work to a file, and the disk is full, what would you rather happen?

- You get an error message, so you can make some space and try again.

- The program silently discards your data, so it is not saved but you don't know it.

- The program deletes other files at random until it has freed up enough space to save your data.


Pick one.


>> 4) catch the fewest possible errors that make sense;
>>
>> 5) nearly always, "the fewest" will mean *zero* -- 99% of your code should
>> not be inside a try...except block;
>>
>> 6) you should put the least amount of code as possible inside each try block
>> -- as a general rule, that doesn't just mean "one line of code", but *one
>> operation*.
>>
>
> Here by "one operation" do you mean the most efficient means to deal
> with that particular error condition?

No. I mean that you should identify (if you can) the smallest thing that can go wrong in the way you expect, and deal with it.

For example, suppose you are looking up a key in a dict, and process it in some way. If the key is not in the dict, you can deal with it quite easily:


try:
     value = my_dict[key]
     result = process(value)
except KeyError:
     result = something_else()


Looks reasonable, yes?

No. You're guarding too much with the try block. The problem is, there are two places where you *might* get a KeyError:

- when you call my_dict[key]

- inside process(value) -- who knows what is happening in there? You have to assume that it *could* raise KeyError.

But only the first case can be legitimately handled by the except clause. A KeyError in process(value) needs to be treated as a bug, and fixed, not covered up. So the above is better written as:

try:
     value = my_dict[key]
except KeyError:
     result = something_else()
else:
     # No exception occurred.
     result = process(value)


Now if you get KeyError inside the call to process(), you will see it, and can fix it.



>> In general, once I have decided that I need a try...except block around a
>> certain operation, I look up what exceptions it is documented to produce (if
>> any!). Then I decide, if such-and-such an exception occurs, can I recover
>> from it? If not, I don't catch it. If so, then I do. Then, I wait to see if
>> the operation produces some other undocumented exception. If so, then I
>> repeat the process.
>>
> Maybe I have been totally misunderstanding your comments. Is what you
> are saying above referring strictly to while you are still debugging
> the program? Or do the above comments that I asked for clarification
> refer to the finished product?

The non-trivial program that is 100% bug-free has never been written yet. Even those that are "finished" probably have bugs, it may be that they are so obscure that nobody has discovered them yet. In practice, programs will be available for use long before they are bug-free, but the rate at which bugs are reported will get slower. At first, you'll be discovering bugs all the time. Then only occasionally. Then rarely. Eventually, you will get to the point that the bugs that remain are so rare, or minor, or so hard to reproduce, that you will decide not to bother fixing them.

This is why pieces of software that are really old and stable will still occasionally get a new version.


-- 
Steven