On Wed, 10 Nov 2004, John P Speno wrote:
Hi, sorry for the delayed response.
> While using subprocess (aka popen5), I came across one potential gotcha. I've had
> exceptions ending like this:
>
> File "test.py", line 5, in test
> cmd = popen5.Popen(args, stdout=PIPE)
> File "popen5.py", line 577, in __init__
> data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
> OSError: [Errno 4] Interrupted system call
>
> (on Solaris 9)
>
> Would it make sense for subprocess to use a more robust read() function
> which can handle these cases, i.e. when the parent's read on the pipe
> to the child's stderr is interrupted by a system call, and returns EINTR?
> I imagine it could catch EINTR and EAGAIN and retry the failed read().
I assume you are using signals in your application? The os.read above is
not the only system call that can fail with EINTR. subprocess.py is full
of other system calls that can fail, and I suspect that many other Python
modules are as well.
I've made a patch (attached) to subprocess.py (and test_subprocess.py)
that should guard against EINTR, but I haven't committed it yet. It's
quite large.
Are Python modules supposed to handle EINTR? Why not let the C code handle
this? Or, perhaps the signal module should provide a sigaction function,
so that users can use SA_RESTART.
Index: subprocess.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/subprocess.py,v
retrieving revision 1.8
diff -u -r1.8 subprocess.py
--- subprocess.py 7 Nov 2004 14:30:34 -0000 1.8
+++ subprocess.py 17 Nov 2004 19:42:30 -0000
@@ -888,6 +888,50 @@
pass
+ def _read_no_intr(self, fd, buffersize):
+ """Like os.read, but retries on EINTR"""
+ while True:
+ try:
+ return os.read(fd, buffersize)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
+
+ def _read_all(self, fd, buffersize):
+ """Like os.read, but retries on EINTR, and reads until EOF"""
+ all = ""
+ while True:
+ data = self._read_no_intr(fd, buffersize)
+ all += data
+ if data == "":
+ return all
+
+
+ def _write_no_intr(self, fd, s):
+ """Like os.write, but retries on EINTR"""
+ while True:
+ try:
+ return os.write(fd, s)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
+ def _waitpid_no_intr(self, pid, options):
+ """Like os.waitpid, but retries on EINTR"""
+ while True:
+ try:
+ return os.waitpid(pid, options)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
def _execute_child(self, args, executable, preexec_fn, close_fds,
cwd, env, universal_newlines,
startupinfo, creationflags, shell,
@@ -963,7 +1007,7 @@
exc_value,
tb)
exc_value.child_traceback = ''.join(exc_lines)
- os.write(errpipe_write, pickle.dumps(exc_value))
+ self._write_no_intr(errpipe_write, pickle.dumps(exc_value))
# This exitcode won't be reported to applications, so it
# really doesn't matter what we return.
@@ -979,7 +1023,7 @@
os.close(errwrite)
# Wait for exec to fail or succeed; possibly raising exception
- data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
+ data = self._read_all(errpipe_read, 1048576) # Exceptions limited to 1 MB
os.close(errpipe_read)
if data != "":
child_exception = pickle.loads(data)
@@ -1003,7 +1047,7 @@
attribute."""
if self.returncode == None:
try:
- pid, sts = os.waitpid(self.pid, os.WNOHANG)
+ pid, sts = self._waitpid_no_intr(self.pid, os.WNOHANG)
if pid == self.pid:
self._handle_exitstatus(sts)
except os.error:
@@ -1015,7 +1059,7 @@
"""Wait for child process to terminate. Returns returncode
attribute."""
if self.returncode == None:
- pid, sts = os.waitpid(self.pid, 0)
+ pid, sts = self._waitpid_no_intr(self.pid, 0)
self._handle_exitstatus(sts)
return self.returncode
@@ -1049,27 +1093,33 @@
stderr = []
while read_set or write_set:
- rlist, wlist, xlist = select.select(read_set, write_set, [])
+ try:
+ rlist, wlist, xlist = select.select(read_set, write_set, [])
+ except select.error, e:
+ if e[0] == errno.EINTR:
+ continue
+ else:
+ raise
if self.stdin in wlist:
# When select has indicated that the file is writable,
# we can write up to PIPE_BUF bytes without risk
# blocking. POSIX defines PIPE_BUF >= 512
- bytes_written = os.write(self.stdin.fileno(), input[:512])
+ bytes_written = self._write_no_intr(self.stdin.fileno(), input[:512])
input = input[bytes_written:]
if not input:
self.stdin.close()
write_set.remove(self.stdin)
if self.stdout in rlist:
- data = os.read(self.stdout.fileno(), 1024)
+ data = self._read_no_intr(self.stdout.fileno(), 1024)
if data == "":
self.stdout.close()
read_set.remove(self.stdout)
stdout.append(data)
if self.stderr in rlist:
- data = os.read(self.stderr.fileno(), 1024)
+ data = self._read_no_intr(self.stderr.fileno(), 1024)
if data == "":
self.stderr.close()
read_set.remove(self.stderr)
Index: test/test_subprocess.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v
retrieving revision 1.14
diff -u -r1.14 test_subprocess.py
--- test/test_subprocess.py 12 Nov 2004 15:51:48 -0000 1.14
+++ test/test_subprocess.py 17 Nov 2004 19:42:30 -0000
@@ -7,6 +7,7 @@
import tempfile
import time
import re
+import errno
mswindows = (sys.platform == "win32")
@@ -35,6 +36,16 @@
fname = tempfile.mktemp()
return os.open(fname, os.O_RDWR|os.O_CREAT), fname
+ def read_no_intr(self, obj):
+ while True:
+ try:
+ return obj.read()
+ except IOError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
#
# Generic tests
#
@@ -123,7 +134,7 @@
p = subprocess.Popen([sys.executable, "-c",
'import sys; sys.stdout.write("orange")'],
stdout=subprocess.PIPE)
- self.assertEqual(p.stdout.read(), "orange")
+ self.assertEqual(self.read_no_intr(p.stdout), "orange")
def test_stdout_filedes(self):
# stdout is set to open file descriptor
@@ -151,7 +162,7 @@
p = subprocess.Popen([sys.executable, "-c",
'import sys; sys.stderr.write("strawberry")'],
stderr=subprocess.PIPE)
- self.assertEqual(remove_stderr_debug_decorations(p.stderr.read()),
+ self.assertEqual(remove_stderr_debug_decorations(self.read_no_intr(p.stderr)),
"strawberry")
def test_stderr_filedes(self):
@@ -186,7 +197,7 @@
'sys.stderr.write("orange")'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
- output = p.stdout.read()
+ output = self.read_no_intr(p.stdout)
stripped = remove_stderr_debug_decorations(output)
self.assertEqual(stripped, "appleorange")
@@ -220,7 +231,7 @@
stdout=subprocess.PIPE,
cwd=tmpdir)
normcase = os.path.normcase
- self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir))
+ self.assertEqual(normcase(self.read_no_intr(p.stdout)), normcase(tmpdir))
def test_env(self):
newenv = os.environ.copy()
@@ -230,7 +241,7 @@
'sys.stdout.write(os.getenv("FRUIT"))'],
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read(), "orange")
+ self.assertEqual(self.read_no_intr(p.stdout), "orange")
def test_communicate(self):
p = subprocess.Popen([sys.executable, "-c",
@@ -305,7 +316,8 @@
'sys.stdout.write("\\nline6");'],
stdout=subprocess.PIPE,
universal_newlines=1)
- stdout = p.stdout.read()
+
+ stdout = self.read_no_intr(p.stdout)
if hasattr(open, 'newlines'):
# Interpreter with universal newline support
self.assertEqual(stdout,
@@ -343,7 +355,7 @@
def test_no_leaking(self):
# Make sure we leak no resources
- max_handles = 1026 # too much for most UNIX systems
+ max_handles = 10 # too much for most UNIX systems
if mswindows:
max_handles = 65 # a full test is too slow on Windows
for i in range(max_handles):
@@ -424,7 +436,7 @@
'sys.stdout.write(os.getenv("FRUIT"))'],
stdout=subprocess.PIPE,
preexec_fn=lambda: os.putenv("FRUIT", "apple"))
- self.assertEqual(p.stdout.read(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout), "apple")
def test_args_string(self):
# args is a string
@@ -457,7 +469,7 @@
p = subprocess.Popen(["echo $FRUIT"], shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read().strip(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple")
def test_shell_string(self):
# Run command through the shell (string)
@@ -466,7 +478,7 @@
p = subprocess.Popen("echo $FRUIT", shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read().strip(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple")
def test_call_string(self):
# call() function with string argument on UNIX
@@ -525,7 +537,7 @@
p = subprocess.Popen(["set"], shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertNotEqual(p.stdout.read().find("physalis"), -1)
+ self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1)
def test_shell_string(self):
# Run command through the shell (string)
@@ -534,7 +546,7 @@
p = subprocess.Popen("set", shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertNotEqual(p.stdout.read().find("physalis"), -1)
+ self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1)
def test_call_string(self):
# call() function with string argument on Windows
/Peter Åstrand <astrand(a)lysator.liu.se>
Hi,
I'm having 2 problems with the current cvs :
During compilation this warning occurs:
*** WARNING: renaming "dbm" since importing it failed: build/lib.linux-i686-2.5/
dbm.so: undefined symbol: dbm_firstkey
and the 'dbm' module is unavailable.
I'm running MandrakeLinux 2005 (10.2) gcc 3.4.3
(I'm also having this problem when compiling python 2.3.5 or 2.4.1)
furthermore the 'make install' of current cvs fails halfway trough
with the following errors:
.....
.....
Compiling /opt/python25/lib/python2.5/bsddb/test/test_associate.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation',
('/opt/python25/lib/python2.5/bsddb/test/test_associate.py', 97, 23, '\t
os.mkdir(homeDir)\n'))
Compiling /opt/python25/lib/python2.5/bsddb/test/test_basics.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation',
('/opt/python25/lib/python2.5/bsddb/test/test_basics.py', 400, 26, '\t if
get_raises_error:\n'))
Compiling /opt/python25/lib/python2.5/bsddb/test/test_compare.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation',
('/opt/python25/lib/python2.5/bsddb/test/test_compare.py', 167, 5, '\t"""\n'))
.....
.....
Compiling /opt/python25/lib/python2.5/bsddb/test/test_recno.py ...
Sorry: TabError: ('inconsistent use of tabs and spaces in indentation',
('/opt/python25/lib/python2.5/bsddb/test/test_recno.py', 38, 46, '\tget_returns_none =
d.set_get_returns_none(2)\n'))
.....
.....
make: *** [libinstall] Error 1
$
And then it quits.
Fixing the tab indentation errors locally makes the problem go away.
Regards,
Irmen de Jong
[Christoph, please keep the python-dev list in the loop here, at least
until they get annoyed and decide we're off-topic. I think this is
crucial to the way they package and deliver Python]
Christoph Ludwig <cludwig(a)cdc.informatik.tu-darmstadt.de> writes:
> On Thu, Jul 07, 2005 at 06:27:46PM -0400, David Abrahams wrote:
>> "Martin v. Löwis" <martin(a)v.loewis.de> writes:
>>
>> > David Abrahams wrote:
>> >> I'm wondering if there has been a well-known recent change either in Python
>> >> or GCC that would account for these new reports. Any relevant
>> >> information would be appreciated.
> [...]
>> > Python is linked with g++ if configure thinks this is necessary
>>
>> Right. The question is, when should configure "think it's necessary?"
>
> Just to add to the confusion... I encountered the case that configure decided
> to use gcc for linking but it should have used g++. (It is Python
> PR #1189330 <http://tinyurl.com/dlheb>. This was on a x86 Linux system with
> g++ 3.4.2.)
>
> Background: The description of --with-cxx in the README of the
> Python 2.4.1 source distribution made me think that I need to
> configure my Python installation with
> --with-configure=/opt/gcc/gcc-3.4.2/bin/g++ if I plan to use C++
> extensions built with this compiler. (That was possibly a
> misunderstanding on my part,
AFAICT, yes.
> but Python should build with this option anyway.)
>
> configure set `LINKCC=$(PURIFY) $(CC)'. The result was that make failed when
> linking the python executable due to an unresolved reference to
> __gxx_personality_v0. I had to replace CC by CXX in the definition of LINKCC
> to finish the build of Python.
>
> When I looked into this problem I saw that configure in fact builds a test
> executable that included an object file compiled with g++. If the link step
> with gcc succeeds then LINKCC is set as above, otherwise CXX is
> used. Obviously, on my system this test was successful so configure decided
> to link with gcc. However, minimal changes to the source of the test program
> caused the link step to fail. It was not obvious to me at all why the latter
> source code should cause a dependency on the C++ runtime if the original
> code does not. My conclusion was that this test is fragile and should be
> skipped.
Sounds like it. I have never understood what the test was really
checking for since the moment it was first described to me, FWIW.
> If Python is built with --with-cxx then it should be linked with CXX
> as well.
U betcha.
> I gather from posts on the Boost mailing lists that you can import
> Boost.Python extensions even if Python was configured
> --without-cxx.
Yes, all the tests are passing that way.
> (On ELF based Linux/x86, at least.) That leaves me wondering
>
> * when is --with-cxx really necessary?
I think it's plausible that if you set sys.dlopenflags to share
symbols it *might* end up being necessary, but IIRC Ralf does use
sys.dlopenflags with a standard build of Python (no
--with-cxx)... right, Ralf?
> * what happens if I import extensions built with different g++ versions? Will
> there be a conflict between the different versions of libstdc++ those
> extensions depend on?
Not unless you set sys.dlopenflags to share symbols.
It's conceivable that they might conflict through their shared use of
libboost_python.so, but I think you have to accept that an extension
module and the libboost_python.so it is linked with have to be built
with compatible ABIs anyway. So in that case you may need to make
sure each group of extensions built with a given ABI use their own
libboost_python.so (or just link statically to libboost_python.a if
you don't need cross-module conversions).
HTH,
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
I'd like to see the Python source be stored in Subversion instead
of CVS, and on python.org instead of sf.net. To facilitate discussion,
I have drafted a PEP describing the rationale for doing so, and
the technical procedure to be performed.
This is for discussion on python-dev and eventual BDFL pronouncement;
if you see a reason not to make such a change, or if you would prefer
a different procedure, please speak up. Encouragement and support is
welcome, as well :-)
Regards,
Martin
PEP: XXX
Title: Migrating the Python CVS to Subversion
Version: $Revision $
Last-Modified: $Date$
Author: Martin v. Löwis <martin(a)v.loewis.de>
Discussions-To: <python-dev(a)python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 14-Jul-2004
Post-History: 14-Jul-2004
Abstract
========
The Python source code is currently managed in a CVS repository on
sourceforge.net. This PEP proposes to move it to a subversion repository
on svn.python.org.
Rationale
=========
This change has two aspects: moving from CVS to subversion, and
moving from SourceForge to python.org. For each, a rationale will
be given.
Moving to Subversion
--------------------
CVS has a number of limitations that have been elimintation by
Subversion. For the development of Python, the most notable improvements
are:
- ability to rename files and directories, and to remove directories,
while keeping the history of these files.
- support for change sets (sets of correlated changes to multiple
files) through global revision numbers.
- support for offline diffs, which is useful when creating patches.
Moving to python.org
--------------------
SourceForge has kindly provided an important infrastructure for the
past years. Unfortunately, the attention that SF received has also
caused repeated overload situations in the past, to which the SF
operators could not always respond in a timely manner. In particular,
for CVS, they had to reduce the load on the primary CVS server by
introducing a second, read-only CVS server for anonymous access.
This server is regularly synchronized, but behind the the read-write
CVS repository between synchronizations. As a result, users without
commit access can see recent changes to the repository only with
a delay.
On python.org, it would be possible to make the repository accessible
for anonymous access.
Migration Procedure
===================
To move the Python CVS repository, the following steps need to be
executed. The steps are elaborated in more detail in the next
sections.
1. Assign passwords for all current committers for use on svn.python.org.
User names on SF and svn.python.org should be identical, unless some
committer requests a different user name.
2. At the beginning of the migration, announce that the repository on
SourceForge closed.
3. 24 hours after the last commit, download the CVS repository.
4. Convert the CVS repository into two subversion repositories,
one for distutils and one for Python.
5. Publish the repositories for write access for all committers,
and anonymous read access.
6. Disable CVS access on SF.
Assign Passwords
================
Currently, access to Subversion on svn.python.org uses WebDAV over
https, using basic authentication. For this to work, authorized
users need to provide a password. This mechanism should be used,
atleast initially, for the Python CVS as well, since various committers
also have a username/password pair for the www SVN repository already.
Alternatives to password-based access include:
- Subversion over SSH, using SSH key pairs. This would require
to give committers accounts on the machine, which currently is
ruled out by the administration policy of svn.python.org.
- Subversion over WebDAV, using SSL client certificates. This
would work, but would require to administrate a certificate
authority.
Downloading the CVS Repository
==============================
The CVS repository can be downloaded from
http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2
Since this tarball is generated only once a day, some time after
the repository freeze must pass before the tarball can be picked
up. It should be verified that the last commit, as recorded on
the python-commits mailing list, is indeed included in the tarball.
Converting the CVS Repository
=============================
The Python CVS repository contains two modules: distutils and
python. Keeping them together will produce quite long repository
URLs, so it is more convenient if the Python CVS and the distutils
CVS are converted into two separate repositories.
As the repository format, fsfs should be used (requires Subversion 1.1).
fsfs has the advantage of being more backup-friendly, as it allows to
backup a repository incrementally, without requiring to run any dump
commands.
The conversion should be done using cvs2svn utility, available e.g.
in the cvs2svn Debian package. The command for converting the Python
repository is
cvs2svn -q --encoding=latin1 --force-branch=cnri-16-start
--force-branch=descr-branch --force-branch=release152p1-patches
--force-tag=r16b1 --fs-type=fsfs -s py.svn.new python/python
The command to convert the distutils repository is
cvs2svn -q --encoding=latin1 --fs-type=fsfs -s dist.svn.new python/distutils
Sample results of this conversion are available at
http://www.dcl.hpi.uni-potsdam.de/python/http://www.dcl.hpi.uni-potsdam.de/distutils/
Publish the Repositories
========================
The repositories should be published at https://svn.python.org/python
and https://svn.python.org/distutils. Read-write should be granted
through basic authentication to all current SF committers; read-only
access should be granted anonymously. As an option, websvn (available
e.g. from the Debian websvn package) could be provided.
The current SF project admins should get write access to the password
file, in order to create or delete new users.
Disable CVS
===========
It appears that CVS cannot be disabled entirely. Only the user interface
can be removed from the project page; the repository itself remains
available. If desired, write access to the python and distutils modules
can be disabled through a commitinfo entry.
With 343 accepted, we can now add __enter__() and __exit__() methods to
objects.
What term should describe those objects in the documentation?
For instance, if the new magic methods are added to decimal.Context(),
do we then describe Context objects as "withable" ;-)
The PEP itself doesn't provide much of a clue. The terms "template",
"wrapper", and "block" are over-broad and do not provide a metaphor for
setup/teardown, r entry/exit, or initialization/finalization.
Whatever term is selected, it should be well thought-out and added to
the glossary. The choice of words will likely have a great effect on
learnability and on how people think about the new tool.
Raymond
Well, it has been discussed at multiple times in the past and I have
promised to write this PEP several times, so I finally found enough
time to write a PEP on reorganizing exceptions for Python 3.0 .
Key points in this PEP is the reworking the hierarchy, requiring
anything raised to inherit from a certain superclass, and to change
bare 'except' clauses to catch a specific superclass. The first and
last points I expect some contention, but the middle point I expect
people are okay with (Guido liked the idea when Paul Prescod brought
it up and the only person who didn't like it, Holger, ended up being
okay with it when the superclass had a reasonable name).
One thing people might not notice is that I have some minor ideas
listed in the tree in parentheses. If people have an opinion on the
ideas please speak up.
Otherwise the other major points of contention are covered in the Open
Issues section or will be brought up in the usual trashing of PEPs
that cover contraversial changes.
And please realize this is for Python 3.0! None of this is being
proposed for any version before then (they could be, but that is
another PEP entirely). Hopefully this PEP along with Ping's PEP 344
will cover the major ideas for exceptions for Python 3.0 .
-Brett
--------------------------------------------------------------
PEP: XXX
Title: Exception Reorganization for Python 3.0
Version: $Revision: 1.5 $
Last-Modified: $Date: 2005/06/07 13:17:37 $
Author: Brett Cannon <brett(a)python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Jul-2005
Post-History: XX-XXX-XXX
Abstract
========
Python, as of version 2.4, has 38 exceptions (including warnings) in
the built-in namespace in a rather shallow hierarchy.
This list of classes has grown over the years without a chance to
learn from mistakes and cleaning up the hierarchy.
This PEP proposes doing a reorganization for Python 3000 when
backwards-compatibility is not an issue.
It also proposes changing bare ``except`` clauses to catch only
exceptions inheriting from a specific superclass.
Lastly, this PEP proposes, in Python 3000, that all objects to be
passed to a ``raise`` statement must inherit from a specific
superclass.
Rationale
=========
Exceptions are a critical part of Python.
While exceptions are traditionally used to signal errors in a program,
they have also grown to be used for flow control for things such as
iterators.
There importance is great.
But the organization of the exception hierarchy has not been maintained.
Mostly for backwards-compatibility reasons, the hierarchy has stayed
very flat and old exceptions who usefulness have not been proven have
been left in.
Making exceptions more hierarchical would help facilitate exception
handling by making catching exceptions much more logical with use of
superclasses.
This should also help lead to less errors from being too broad in what
exceptions are caught in an ``except`` clause.
A required superclass for all exceptions is also being proposed
[Summary2004-08-01]_.
By requiring any object that is used in a ``raise`` statement to
inherit from a specific superclass, certain attributes (such as those
laid out in PEP 344 [PEP344]_) can be guaranteed to exist.
This also will lead to the planned removal of string exceptions.
Lastly, bare ``except`` clauses can be made less error-prone
[Summary2004-09-01]_.
Often people use a bare ``except`` when what they really wanted were
non-critical exceptions to be caught while more system-specific ones,
such as MemoryError, to pass through and to halt the interpreter.
With a hierarchy reorganization, bare ``except`` clauses can be
changed to only catch exceptions that subclass a non-critical
exception superclass, allowing more critical exceptions to propagate
to the top of the execution stack as was most likely intended.
Philosophy of Reorganization
============================
There are several goals in this reorganization that defined the
philosophy used to guide the work.
One goal was to prune out unneeded exceptions.
Extraneous exceptions should not be left in since it just serves to
clutter the built-in namespace.
Unneeded exceptions also dilute the importance of other exceptions by
splitting uses between several exceptions when all uses should have
been under a single exception.
Another goal was to introduce any exceptions that were deemed needed
to fill any holes in the hierarchy.
Most new exceptions were done to flesh out the inheritance hierarchy
to make it easier to catch a category of exceptions with a simpler
``except`` clause.
Changing inheritance to make it more reasonable was a goal.
As stated above, having proper inheritance allows for more accurate
``except`` statements when catching exceptions based on the
inheritance tree.
Lastly, any renaming to make an exception's use more obvious from its
name was done.
Having to look up what an exception is meant to be used for because
the name does not proper reflect its usage is annoying and slows down
debugging.
Having a proper name also makes debugging easier on new programmers.
New Hierarchy
=============
Raisable (new; rename BaseException?)
+-- CriticalException (new)
+-- KeyboardInterrupt
+-- MemoryError
+-- SystemExit
+-- SystemError (subclass SystemExit?)
+-- AssertionError
+-- SyntaxError
+-- IndentationError
+-- TabError
+-- Exception (replaces StandardError)
+-- ArithmeticError
+-- FloatingPointError
+-- DivideByZeroError
+-- OverflowError
+-- ControlFlowException (new)
+-- StopIteration
+-- GeneratorExit (introduced by PEP 342 [PEP342]_; subclass
StopIteration?)
+-- UnicodeError
+-- UnicodeDecodeError
+-- UnicodeEncodeError
+-- UnicodeTranslateError (subclass UnicodeEncodeError and
UnicodeDecodeError?)
+-- LookupError (better name?)
+-- IndexError
+-- KeyError
+-- TypeError
+-- AttributeError (subclassing new)
+-- OSError (does not inherit from EnvironmentError)
+-- WindowsError
+-- MacError (new)
+-- UnixError (new)
+-- IOError (does not inherit from EnvironmentError)
+-- EOFError (subclassing new)
+-- ImportError
+-- NotImplementedError
+-- NamespaceException (new)
+-- UnboundGlobalError (new name for NameError)
+-- UnboundLocalError (no longer subclasses UnboundGlobalError
which replaces NameError)
+-- WeakReferenceError (rename for ReferenceError)
+-- ValueError
+-- Warning
+-- UserWarning
+-- PendingDeprecationWarning
+-- DeprecationWarning (subclassing new)
+-- SyntaxWarning
+-- SemanticsWarning (new name for RuntimeWarning)
+-- FutureWarning
Differences Compared to Python 2.4
==================================
Changes to exceptions from Python 2.4 can take shape in three forms:
removal, renaming, or change in their superclass.
There are also new exceptions introduced in the proposed hierarchy.
New Exceptions
--------------
Raisable
''''''''
The base exception **all** exceptions inherit from.
The name "Raisable" has been chosen to reflect that it is not meant to
be treated as an exception directly, but as the common object that all
things to base passed to ``raise`` must inherit from.
CriticalException
'''''''''''''''''
The superclass for exceptions for which a severe error has occurred
that one would not want to catch accidentally.
The name is meant to reflect the point that these exceptions are
usually raised only when the interpreter should most likely be
terminated.
MacError
''''''''
Introduced for symmertry with WindowsError.
UnixError
'''''''''
Introduced for symmetry with WindowsError.
NamespaceException
''''''''''''''''''
To provide a common superclass for exceptions dealing with namespace
issues, this exception is introduced.
Both UnboundLocalError and UnboundGlobalError (the new name for
NameError) inherit from this class.
Removed Exceptions
------------------
EnvironmentError
''''''''''''''''
Originally meant as an exception for when an event outside of the
interpreter needed to raise an exception, its use has been deemed
unneeded.
While both IOError and OSError inherited this class, the distinction
between OS and I/O is significant enough to not warrant having a
common subclass that one might base an ``except`` clause on.
StandardError
'''''''''''''
Originally meant to act as a superclass for exceptions not considered
critical, its subclassing was rampant and partially negated its
usefulness.
The need for StandardError is negated thanks to the
CriticalException/Exception dichotomy where Exception takes its place.
RuntimeError
''''''''''''
Meant to be used when an existing exception does not fit, its
usefulness is consider meager in Python 2.4 [exceptionsmodule]_.
Also, with the CriticalException/Exception dichotomy, Exception or
CriticalException can be raised directly for the same use.
Renamed Exceptions
------------------
ReferenceError
''''''''''''''
Renamed WeakReferenceError.
ReferenceError was added to the built-in exception hierarchy in Python
2.2 [exceptionsmodule]_.
Taken directly from the ``weakref`` module, its name comes directly
from its original name when it resided in the module.
Unfortunately its name does not suggest its connection to weak
references and thus deserves a renaming.
NameError
'''''''''
Renamed UnboundGlobalError.
While NameError suggests its use, the name does not properly restrict its scope.
With UnboundLocalError already in existence, it seems reasonable to
change NameError to UnboundGlobalError to brings its name more in
line.
RuntimeWarning
''''''''''''''
Renamed SemanticsWarning.
RuntimeWarning is to represent semantic changes coming in the future.
But while saying that affects "runtime" is true, flat-out stating it
is a semantic change is much clearer, eliminating any possible
association of "runtime" with the virtual machine specifically.
Changed Inheritance
-------------------
StopIteration and GeneratorExit
'''''''''''''''''''''''''''''''
Inherit from ControlFlowException.
Collecting all control flow-related exceptions under a common
superclass continues with the theme of maintaining a hierarchy.
AttributeError
''''''''''''''
Inherits TypeError.
Since most attribute access errors can be attributed to an object not
being the type that one expects, it makes sense for AttributeError to
be considered a type error.
IOError
'''''''
No longer subclasses EnvironmentError.
While IOError does fall under the umbrella of EnvironmentError, the
need for EnvironmentError has been deemed wanting, and thus has been
removed.
Thus IOError now subclasses Exception directly.
EOFError
''''''''
Subclasses IOError.
Since an EOF comes from I/O it only makes sense that it be considered
an I/O error.
UnboundGlobalError and UnboundLocalError
''''''''''''''''''''''''''''''''''''''''
Both subclass NamespaceException.
Originally UnboundLocalError subclassed NameError (original name of
UnboundGlobalError), but with both exceptions specifying a certain
namespace, that subclassing has been deemed inappropriate.
Instead, a common class between the two of them has been introduced.
DeprecationWarning
'''''''''''''''''''
Subclasses PendingDeprecationWarning.
Since a DeprecationWarning is a PendingDeprecationWarning that is
happening sooner, if you care about PendingDeprecationWarnings you are
defintely going to care about DeprecationWarnings.
Required Base Class for ``raise``
=================================
By requiring all objects passed to a ``raise`` statement, one is
guaranteed that all exceptions will have certain attributes.
If PEP 342 [PEP344]_ is accepted, the attributes outlined there will
be guaranteed to be on all exceptions raised.
This should help facilitate debugging by making the querying of
information from exceptions much easier.
Technically speaking, this can be easily implemented by modifying
``RAISE_VARARGS`` to do an inheritance check and raise TypeError if it
does not inherit from Raisable.
Bare ``except`` Clauses
=======================
Bare ``except`` clauses typically are not meant to catch all exceptions.
Often times catching KeyboardInterrupt is not intended.
Unfortunately the exception hierarchy in Python 2.4 does not support a
simple way to control what exceptions are caught except to be very
explicit with a list of exceptions.
The reorganization changes this by adding more of a hierarchy.
If bare ``except`` statements only catch classes that inherit from
Exception, its use will be more in line with what people expect.
An implementation can be handled at the bytecode level by modifying
the compiler to emit bytecode to do a check for Exception when a bare
``except`` is used.
Open Issues
===========
Change the Name of Raisable?
----------------------------
It has been argued that Raisable is a bad name to use since it is not
an actual word [python-dev1]_.
At issue is choosing a name that clearly explains the class' usage.
"BaseException" might be acceptable although the name does not quite
make it as explicit as Raisable that the class is not meant to be
raised directly.
Should Bare ``except`` Clauses be Removed?
------------------------------------------
It has been argued that bare ``except`` clauses should be removed
entirely [python-dev2]_.
The train of thought is that it will force people to specify what they
want to catch and not be too broad.
One issue with this, though, is whether people will make the proper
choices of what to catch or go too broadly.
Would a new Python programmer make the right decision and not catch
CriticalExceptions or would they go overboard and catch Raisable?
It would seem the issue boils down to whether we think people will
make proper decisions or make a reasonable solution available for the
simplest case assuming they might make a bad one.
Change the name of Exception?
-----------------------------
Some have suggested names other than "Exception" for the superclass to
inherit from for bare ``except`` clauses to match against.
The issue with going with a name that is not simplistic is that it
raises the chance of people subclassing the wrong exception for what
they want.
By using the most generic name for the common case it raises the
chances that people will use this class for subclassing their own
exceptions rather than Raisable or CriticalException.
Acknowledgements
================
XXX
References
==========
.. [PEP342] PEP 342 (Coroutines via Enhanced Generators)
(http://www.python.org/peps/pep-0342.html)
.. [PEP344] PEP 344 (Exception Chaining and Embedded Tracebacks)
(http://www.python.org/peps/pep-0344.html)
.. [exceptionsmodules] 'exceptions' module
(http://docs.python.org/lib/module-exceptions.html)
.. [Summary2004-08-01] python-dev Summary (An exception is an
exception, unless it doesn't inherit from Exception)
(http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-i…)
.. [Summary2004-09-01] python-dev Summary (Cleaning the Exception House)
(http://www.python.org/dev/summary/2004-09-01_2004-09-15.html#cleaning-the-e…)
.. [python-dev1] python-dev email (Exception hierarchy)
(http://mail.python.org/pipermail/python-dev/2004-August/047908.html)
.. [python-dev2] python-dev email (Dangerous exceptions)
(http://mail.python.org/pipermail/python-dev/2004-September/048681.html)
Copyright
=========
This document has been placed in the public domain.
Hello python-list,
As I Understood, semantic may be next:
def qwerty(a,a.i,b,b.i,f.j):
pass
Would work like:
def qwerty(anonymous1,anonymous2,anonymous3,anonymous4,anonymous5):
(a,a.i,b,b.i,f.j)=(anonymous1,anonymous2,anonymous3,anonymous4,anonymous5)
del anonymous1
del anonymous2
del anonymous3
del anonymous4
del anonymous5
If so then I like it, because it more explicit than anything other
and it is working right now. I just tryed:
>>> class Stub:
... pass
>>> def qwer(a1,a2,a3,a4,a5):
... (a,a.i,b,b.i,f['i'])=(a1,a2,a3,a4,a5)
... del a1
... del a2
... del a3
... del a4
... del a5
... print (a,a.i,b,b.i,f['i'])
>>> f={}
>>> qwer(Stub(),1,Stub(),2,3)
(<__main__.Stub instance at 0x00C49530>, 1, <__main__.Stub instance at 0x00C498C8>, 2, 3)
So there are not too much for implement.
Another question - named arguments. How I can assign to them?
May be so:
f={}
def qwerty(a,a.i,f['i']):
print (a,a.i,f['i'])
qwerty(f['i']=3,a=Stub(),a.i=4) - ?
and for __init__:
class MyClass:
def __init__(self,self.i,self.j,self.k):
pass
MyObject = MyClass(self.i=1,self.j=2,self.k=3)
?
or may be there can be an aliase syntax?
class MyClass:
def __init__(self, self.i by i,self.j by j, self.k by k):
pass
MyObject = MyClass(i=1,j=2,k=3)
--
Sokolov Yura
falcon(a)intercable.ru
On 7/30/05, Josiah Carlson <jcarlson(a)uci.edu> wrote:
>
> Brett Cannon <bcannon(a)gmail.com> wrote:
>
> > > +-- Warning
> > > +-- DeprecationWarning
> > > +-- FutureWarning
> > > +-- PendingDeprecationWarning
> >
> > Don't like the idea of having DeprecationWarning inherit from
> > PendingDeprecationWarning?
>
> Not all DeprecationWarnings are Pending, but all
> PendingDeprecationWarnings are DeprecationWarnings.
>
See, I don't agree with that logic. DeprecationWarning means
something has been deprecated, while PendingDeprecationWarning means
something will be deprecated in the future. I am say that the for
DeprecationWarning, the future is now and thus is a
PendingDeprecationWarning as well.
It also just makes sense from the standpoint of catching warnings. If
you care about catching PendingDeprecationWarning you are going to
care about catching a DeprecationWarning since if you are worrying
about the less severe version you are definitely going to care about
the most severe case.
-Brett
Brett Cannon wrote:
> New Hierarchy
> =============
>
> Raisable (new; rename BaseException?)
> +-- CriticalException (new)
> +-- KeyboardInterrupt
> +-- MemoryError
> +-- SystemExit
> +-- SystemError (subclass SystemExit?)
I'd recommend not subclassing SystemExit--there are too many programs
out there which expect the argument (e.g. sys.exit(3)) to mean something
specific, but that expectation doesn't apply at all to SystemError.
> +-- Exception (replaces StandardError)
>...
> +-- ControlFlowException (new)
I'd definitely appreciate ControlFlowException--there are a number of
exceptions in CherryPy which should subclass from that. Um, CherryPy
3.0, that is. ;)
> +-- LookupError (better name?)
SubscriptError, perhaps? But LookupError could become the "I didn't find
obj X in the container you specified" superclass, in which case
LookupError is perfect.
> +-- TypeError
> +-- AttributeError (subclassing new)
"Since most attribute access errors can be attributed to an object not
being the type that one expects, it makes sense for AttributeError to
be considered a type error."
Very hmmm. I would have thought it would be a LookupError instead, only
because I favor the duck-typing parts of Python. Making AttributeError
subclass from TypeError leans toward stronger typing models a bit too
much, IMO.
> +-- WeakReferenceError (rename for ReferenceError)
This also has a LookupError feel to it.
> It has been argued that Raisable is a bad name to use since it is not
> an actual word [python-dev1]_.
Perhaps, but compare http://www.onelook.com/?w=raisable to
http://www.onelook.com/?w=raiseable. The only sources "raiseable" has
going for it are Dictionary.com (which aggregates lots of questionable
sources), Rhymezone, and LookWAYup. I think "raisable" is the clear
winner.
Robert Brewer
System Architect
Amor Ministries
fumanchu(a)amor.org
