[Python-Dev] PEP 3151 from the BDFOP

[Antoine Pitrou]

Hi,

> One guiding principle for me is that we should keep the abstraction as thin as
> possible.  In particular, I'm concerned about mapping multiple errnos into a
> single Error.  For example both EPIPE and ESHUTDOWN mapping to BrokePipeError,
> or EACESS or EPERM to PermissionError.  I think we should resist this, so that
> one errno maps to exactly one Error.  Where grouping is desired, Python
> already has mechanisms to deal with that, e.g. superclasses and multiple
> inheritance.  Therefore, I think it would be better to have
> 
> + FileSystemPermissionError
>   + AccessError (EACCES)
>   + PermissionError (EPERM)

I'm not sure that's a good idea:

- EPERM is not only about filesystem permissions, see for example
  http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_cond_timedwait.html

- EACCES and EPERM as a low-level distinction makes sense, but at the
  Python programmer's high-level point of view, the AccessError /
  PermissionError distinction does not seem to convey any useful
  meaning.
  (or perhaps that's just my bad understanding of English)

- the "errno" attribute is still there (and still displayed - see below)
  for people who know their system calls and want to inspect the
  original error code

> Also, some of the artificial hierarchy introduced in the PEP may
> not be necessary (e.g. the faux superclass FileSystemPermissionError above).
> This might lead to the elimination of FileSystemError as some have suggested
> (I too question its utility).

Yes, FileSystemError might be removed. I thought that it would be
useful, in some library routines, to catch all filesystem-related
errors indistinctly, but it's not a complete catchall actually (for
example, AccessError is outside of the FileSystemError subtree).

> Similarly, I think it would be helpful to have the errno name (e.g. ENOENT) in
> the error message string.  That way, it won't get in the way for most code,
> but would be usefully printed out for uncaught exceptions.

Agreed, but I think that's a feature request quite orthogonal from the
PEP. The errno *number* is still printed as it was before:

>>> open("foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'foo'

(see e.g. http://bugs.python.org/issue12762)

> A second guiding principle should be that careful code that works in Python
> 3.2 must continue to work in Python 3.3 once PEP 3151 is accepted, but also
> for Python 2 code ported straight to Python 3.3.

I don't porting straight to 3.3 would make a difference, especially now
that the idea of deprecating old exception names has been abandoned.

> Do be prepared for
> complaints about compatibility for careless code though - there's a ton of
> that out in the wild, and people will always complain with their "working"
> code breaks due to an upgrade.  Be *very* explicit about this in the release
> notes and NEWS file, and put your asbestos underoos on.

I'll take care about that :)

> Have you considered the impact of this PEP on other Python implementations?
> My hazy memory of Jython tells me that errnos don't really leak into Java and
> thus Jython much, but what about PyPy and IronPython?  E.g. step 1's
> deprecation strategy seems pretty CPython-centric.

Alternative implementations already have to implement errno codes in a
way or another if they want to have a chance of running existing code.
So I don't think the PEP makes much of a difference for them.
But their implementors can give their opinion on this.

> As for step 1 (coalescing the errors).  This makes sense and I'm generally
> agreeable, but I'm wondering whether it's best to re-use IOError for this
> rather than introduce a new exception.  Not that I can think of a good name
> for that.  I'm just not totally convinced that existing code when upgrading to
> Python 3.3 won't introduce silent failures.  If an existing error is to be
> re-used for this, I'm torn on whether IOError or OSError is a better choice.
> Popularity aside, OSError *feels* more right.

I don't have any personal preference. Previous discussions seemed to
indicate people preferred IOError. But changing the implementation to
OSError would be simple. I agree OSError feels slightly more right, as
in more generic.

> What is the impact of the PEP on tools such as 2to3 and 3to2?

I'd say none for 2to3. For 3to2 I'm not sure. Obviously if you write
code taking advantage of new features, it will be difficultly
back-portable to 2.x. But that's not specific to PEP 3151. Python 3.2
has lot such stuff already:
http://docs.python.org/py3k/whatsnew/3.2.html

> Just to be clear, am I right that (on POSIX systems at least) IOError and its
> subclasses will always have an errno attribute still?

Yes!

> And that anything
> raising an exception (e.g. via PyErr_SetFromErrno) other than the new ones
> will raise IOError?

I'm not sure I understand the question precisely. The errno mapping
mechanism is implemented in IOError.__new__, but it gets called only if
the class is exactly IOError, not a subclass:

>>> IOError(errno.EPERM, "foo")
PermissionError(1, 'foo')
>>> class MyIOError(IOError): pass
... 
>>> MyIOError(errno.EPERM, "foo")
MyIOError(1, 'foo')

Using IOError.__new__ is the easiest way to ensure that all code
raising IO errors takes advantage of the errno mapping. Otherwise you
may get APIs raising the proper subclasses, and other APIs always
raising base IOError (it doesn't happen often, but some Python
library code raises an IOError with an explicit errno).

> I also think that rather than transforming exception when raised from Python,
> i.e. via __new__ hackery, perhaps it should be a ValueError in its own right
> to raise IOError with an error represented by one of the subclasses.

That would make it harder to keep compatibility while adding new
subclasses in future Python versions. Imagine a lot of people lobby for
a dedicated EBADF subclass and obtain it, then IOError(EBADF, "some
message") would suddenly raise a ValueError. Or do I misunderstand your
proposal?

> I surveyed some of my own code and observed (as others have) that EISDIR and
> ENOTDIR are pretty rare.

Yes, I think they are common in shutil-like code.

> I found more examples of ECHILD and ESRCH than the
> former two.  How'd you like to add those two to make your BDFOP happy? :)

Wow, I didn't know ESRCH.
How would you call the respective exceptions?
- ChildProcessError for ECHILD?
- ProcessLookupError for ESRCH?

> What if all the errno symbolic names were mapped as attributes on IOError?
> The only advantage of that would be to eliminate the need to import errno, or
> for the ugly `e.errno == errno.ENOENT` stuff.  That would then be rewritten as
> `e.errno == IOError.ENOENT`.  A mild savings to be sure, but still.

Hmm, I guess that's explorable as an orthogonal idea.

> How dumb/useless/unworkable would it be to add an __future__ to switch from
> the old hierarchy to the new one?  Probably pretty. ;)

Well, the hierarchy is built-in, since it's about standard exceptions.
Also, you usually get the exception from some library API, so a
__future__ in your own module would not achieve much.

> What about an api that applications/libraries could use to add additional
> exceptions based on other errnos they cared about?  This could be consulted in
> PyErr_SetFromErrno() and raised instead of IOError.  Okay, yeah, that's
> probably pretty dumb too.

The problem is that behaviour becomes inconsistent accross libraries.
I'm not sure that's very helpful to the user.

Regards

Antoine.