[Python-Dev] PEP 348: Exception Reorganization for Python 3.0

[Nick Coghlan]

Guido van Rossum wrote:
> The point is not to avoid bare 'except:' from hiding programming
> errors. There's no hope to obtain that goal.
> 
> The point is to make *legitimate* uses of bare 'except:' easier -- the
> typical use case is an application that has some kind of main loop
> which uses bare 'except:' to catch gross programming errors in other
> parts of the app, or in code received from an imperfect source (like
> an end-user script) and recovers by logging the error and continuing.
> (I was going to say "or clean up and exit", but that use case is
> handled by 'finally:'.)
> 
> Those legitimate uses often need to make a special case of
> Keyboardinterrupt and SystemExit -- KeyboardInterrupt because it's not
> a bug in the code but a request from the user who is *running* the app
> (and the appropriate default response is to exit with a stack trace);
> SystemExit because it's not a bug but a deliberate attempt to exit the
> program -- logging an error would be a mistake.
> 
> I think the use cases for moving other exceptions out of the way are
> weak; MemoryError and SystemError are exceedingly rare and I've never
> felt the need to exclude them; when GeneratorExit or StopIteration
> reach the outer level of an app, it's a bug like all the others that
> bare 'except:' WANTS to catch.

To try to turn this idea into a concrete example, the idea would be to make 
the following code work correctly:

   for job in joblist:
     try:
        job.exec()
     except: # or "except Exception:"
        failed_jobs.append((job, sys.exc_info()))

Currently, this code will make a user swear, as Ctrl-C will cause the program 
to move onto the next job, instead of exiting as you would except (I have 
found Python scripts not exiting when I press Ctrl-C to be an all-too-common 
problem).

Additionally calling sys.exit() inside a job will fail. This may be deliberate 
(to prevent a job from exiting the whole application), but given only the code 
above, it looks like a bug.

The program will attempt to continue in the face of a MemoryError. This is 
actually reasonable, as memory may have been freed as the stack unwound to the 
level of the job execution loop, or the request that failed may have been for 
a ridicuolously large amount of memory.

The program will also attempt to continue in the face of a SystemError. This 
is reasonable too, as SystemError is only used when the VM thinks the current 
operation needs to be aborted due to an internal problem in the VM, but the VM 
itself is still safe to use. If the VM thinks something is seriously wrong 
with the internal data structures, it will kill the process with Py_FatalError 
(to ensure that no further Python code is executed), rather than raise 
SystemError.

As others have pointed out, GeneratorExit and StopIteration should never reach 
the job execution loop - if they do, there's a bug in the job, and they should 
be caught and logged.

That covers the six exceptions that have been proposed to be moved out from 
under "Exception", and, as I see it, only two of them end up making the grade 
- SystemExit and KeyboardInterrupt, for exactly the reasons Guido gives in his 
message above.

This suggests a Py3k exception hierarchy that looks like:

   BaseException
   +-- CriticalException
       +-- SystemExit
       +-- KeyboardInterrupt
   +-- Exception
       +-- GeneratorExit
       +-- (Remainder as for Python 2.4, other than KeyboardInterrupt)

With a transitional 2.x hierarchy that looks like:

   BaseException
   +-- CriticalException
       +-- SystemExit
       +-- KeyboardInterrupt
   +-- Exception
       +-- GeneratorExit
       +-- (Remainder exactly as for Python 2.4)

The reason for the CriticalException parent is that Python 2.x code can be 
made 'correct' by doing:

   try:
       # whatever
   except CriticalException:
       raise
   except: # or 'except Exception'
       # Handle everything non-critical

And, the hypothetical job execution loop above can be updated to:

   for job in joblist:
     try:
        job.exec()
     except CriticalException:
        failed_jobs.append((job, sys.exc_info()))
        job_idx = joblist.find(job)
        skipped_jobs.extend(joblist[job_idx+1:]
        raise
     except: # or "except Exception:"
        failed_jobs.append((job, sys.exc_info()))


To tell the truth, if base except is kept around for Py3k, I would prefer to 
see it catch BaseException rather than Exception. Failing that, I would prefer 
to see it removed. Having it catch something other than the root of the 
exception hierarchy would be just plain confusing.

Moving SystemExit and KeyboardInterrupt is the only change we've considered 
which seems to have a genuine motivating use case. The rest of the changes 
suggested don't seem to be solving an actual problem (or are solving a problem 
that is minor enough to be not worth any backward compatibility pain).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com