[Python-ideas] PEP 3151 - Reworking the OS and IO exception hierarchy (again)

Antoine Pitrou solipsis at pitrou.net
Fri Nov 12 01:35:14 CET 2010

On Fri, 12 Nov 2010 00:31:18 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> Nick Coghlan wrote:
> > On Fri, Nov 12, 2010 at 1:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >> There's nothing wrong with code such as this:
> >>
> >> try:
> >>  ...
> >> except socket.error:
> >>  print 'Network problem'
> >> except IOError:
> >>  print 'File I/O problem'
> >> except OSError:
> >>  print 'File system problem'
> >>
> >> You probably have a different use of try-except in mind. Those
> >> exception handling blocks are used at various levels in an
> >> application and the code run between try-except may well be dealing
> >> with sockets and file I/O, but require two different sets of
> >> problem resolution or reporting implementations.
> >>
> >> The PEP breaks such code.
> > 
> > No, that's the kind of code we're saying is already subtly broken, and
> > we're just changing the way it is broken to be far more obvious
> > (instead of *sometimes* printing the wrong message based on exactly
> > what went wrong, it will instead consistently print the message
> > associated with whichever of IOError and OSError is listed first). The
> > standard library may have been consistent about IOError vs OSError at
> > one point in time, but that time is in the past. Any code which
> > doesn't catch (or ignore) them both and treat them as equivalent to
> > each other is either already broken (if the try block contains code
> > which may raise either exception) or won't be affected by the proposal
> > in the PEP (if the try block is guaranteed to raise only the exception
> > type already caught by the try block).
> I don't follow you.
> Example: If I run code that e.g. uses os.fork()
> and get an IOError, I don't want to catch that, since this is
> likely a programming error of some sort which I don't want to
> mask. However, if I get an OSError (the error raised by os.fork()),
> I can report this as a resource problem to higher level code
> which can then take appropriate action.

If you're writing code such as
(which I'm sure is frowned upon in the stdlib):

        ... 99 lines of code ...
        ... 99 other lines of code ...
    except OSError:
        # oh, os.fork() must have failed

you're already living dangerously, and even without this PEP.
OSError is not "the error raised by os.fork()", it is the error raised
by *many* stdlib functions including os.fork().  It could be raised by
many of the other 198 lines of code in the try ... except block above.

So, the goal of the exception category is to tell you what the problem
is, not where it happened.  How you know where it happened is by having
fine-grained try ... except blocks.
(and if you're a human, you can also read the traceback)

> Antoine is arguing that simply by relying on the errno you
> can write "careful" code. That's not correct. If I write
> code that only checks the errno without knowing in which context
> this was set, I cannot always make assumptions on what that
> errno really means, e.g. EACCES, EBADF or EPERM are used by lots
> of C lib and OS functions, so there's no implicit hint to
> the subsystem where the error originated.

Checking exceptions without knowing in which context they are raised
*is* doomed to failure. Your OSError could come from a lot of places
already, so you can't do anything meaningful with it.

On the other hand, if using a fine-grained try ... except block you
know that e.g. EACCES (or PermissionError) was issued by os.rename(),
you know precisely what it means.

> If we go down that route, we might as well merge TypeError
> into ValueError as well, since the distinction between those
> has been blurred long before OSError was even introduced.

Well, no, on the contrary.

The distinction between TypeError and ValueError (and KeyError and
SyntaxError and MemoryError... etc.) is *precisely* the kind of
distinction which is advocated by this PEP, that is: a distinction
between different kinds of problems. Not a distinction based on the
location where a problem happens.

(otherwise we would have DictError, StrError, BytesError, IntError...

And, indeed, the PEP helps improve the consistency of the exception
hierarchy in that respect.

(if you prefer the other kind of consistency, you can propose a PEP to
introduce DictError, ForkError, RenameError, etc.)

> Besides, there are lots of functions in the os module that don't do
> any I/O in the standard sense - why should those raise an IOError ?
> Most of the functions deal with OS services, so the name OSError
> is a lot more appropriate.

Well, you were part of the earlier discussion where this was discussed.
I'll let you browse through the archives.

> Sooner or later
> people are going to use ConnectionError for all kinds of connections,
> not only socket-related ones.

I'm not sure what the problem with reusing exceptions is. We do it all
the time.



More information about the Python-ideas mailing list