I've received some enthusiastic emails from someone who wants to
revive restricted mode. He started out with a bunch of patches to the
CPython runtime using ctypes, which he attached to an App Engine bug:
http://code.google.com/p/googleappengine/issues/detail?id=671
Based on his code (the file secure.py is all you need, included in
secure.tar.gz) it seems he believes the only security leaks are
__subclasses__, gi_frame and gi_code. (I have since convinced him that
if we add "restricted" guards to these attributes, he doesn't need the
functions added to sys.)
I don't recall the exploits that Samuele once posted that caused the
death of rexec.py -- does anyone recall, or have a pointer to the
threads?
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
Hello everyone!
We have been encountering several deadlocks in a threaded Python
application which calls subprocess.Popen (i.e. fork()) in some of its
threads.
This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel.
Preliminary analysis of the hang shows that the child process blocks
upon entering the execvp function, in which the import_lock is acquired
due to the following line:
def _ execvpe(file, args, env=None):
from errno import ENOENT, ENOTDIR
...
It is known that when forking from a pthreaded application, acquisition
attempts on locks which were already locked by other threads while
fork() was called will deadlock.
Due to these oddities we were wondering if it would be better to extract
the above import line from the execvpe call, to prevent lock
acquisition attempts in such cases.
Another workaround could be re-assigning a new lock to import_lock
(such a thing is done with the global interpreter lock) at PyOS_AfterFork or
pthread_atfork.
We'd appreciate any opinions you might have on the subject.
Thanks in advance,
Yair and Rotem
hey, has anyone investigated compiling python2.5 using winegcc, under wine?
i'm presently working my way through it, just for kicks, and was
wondering if anyone would like to pitch in or stare at the mess under
a microscope.
it's not as crazed as it sounds. cross-compiling python2.5 for win32
with mingw32 is an absolute miserable bitch of a job that goes
horribly wrong when you actually try to use the minimalist compiler to
do any real work.
so i figured that it would be easier to get python compiled using
wine. i _have_ got some success - a python script and a python.exe.so
(which is winegcc's friendly way of telling you you have something
that stands a chance of working) as well as a libpython25.dll.so.
what i _don't_ yet have is an _md5.dll (or should it be _md5.lib?)
i.e. the standard modules are a bit... iffy. the _winreg.o is
compiled; the _md5.o is compiled; the winreg.lib is not. whoops.
plus, it's necessary to enable nt_dl.c which is in PC/ _not_ in
Modules/.
one of the key issues that's a bit of a bitch is that python is
compiled up for win32 with a hard-coded pyconfig.h which someone went
to a _lot_ of trouble to create by hand instead of using autoconf. oh
- and it uses visualstudio so there's not even a Makefile. ignoring
that for the time-being was what allowed me to get as far as actually
having a python interpreter (with no c-based modules).
so there's a whole _stack_ of stuff that needs dragging kicking and
screaming into the 21st century.
there _is_ a reason why i want to do this. actually, there's two.
firstly, i sure as shit do _not_ want to buy, download, install _or_
run visual studio. i flat-out refuse to run an MS os and visual
studio runs like a dog under wine.
secondly, i want a python25.lib which i can use to cross-compile
modules for poor windows users _despite_ sticking to my principles and
keeping my integrity as a free software developer.
thirdly i'd like to cross-compile pywebkitgtk for win32
fourthly i'd like to compile and link applications to the extremely
successful and well wicked MSHTML.DLL... in the _wine_ project :) not
the one in windows (!) i want to experiment with DOM model
manipulation - from python - similar to the OLPC HulaHop project -
_but_ i want to compile or cross-compile everything from linux, not
windows (see 1 above)
fifthly i'd like to see COM (DCOM) working and pywin32 compiled and
useable under wine, even if it means having to get a license to use
dcom98 and oleauth.lib and oleauth.h etc. and all the developer files
needed to link DCOM applications under windows. actually what i'd
_really_ like to see is FreeDCE's DCOM work actually damn well
finished, it's only been eight years since wez committed the first
versions of the IDL and header files, and it's only been over fifteen
years since microsoft began its world domination using COM and DCOM.
... but that's another story :)
so that's ... five reasons not two. if anyone would like to
collaborate on a crazed project with someone who can't count, i'm
happy to make available what i've got up to so far, on github.org.
l.
Hi,
I'd like to bring up the general idea of using a PSF slot in GSoC2009
to improve the Python development infrastructure. I also happen to
have two concrete proposals for that (such a coincidence!). But I
assure you the general idea is more important than my proposals :)
General:
Solving issues that get in core devs' way when they work on Python
development could be a nice PSF GSoC project.
I believe there are enough code related tasks that would greatly
improve Python development workflow but lack manpower to complete.
E.g., ISTM that a student working on svnmerge in past SoC editions
would've been able to mitigate some painful shortcomings. If the core
developers could come up with a list of peeves that would be solvable
in code (in existing tools), that would allow for a very useful GSoC
proposal.
Proposals:
These might fit the description above, and they're both tracker
related (yep, one-trick-pony here). The upside is that at least one
potential mentor is available for each, and I'd be able to offer
support to the student.
First, the PSF could invest a slot on integrating different tools of
the development workflow. Variations of 'file issue at bug tracker,
submit code for review' or 'branch locally to virtualenv, upload
failing testcase to tracker, upload patch to tracker' command line
utils. If these could be developed as general tools (i.e., not deeply
tied to Python dev infrastructure), Ali Afshar from PIDA is willing to
mentor it. I'd be available to help with Roundup and Rietveld
(integration in progress), but would like to see it work with
Launchpad, Bugzilla, Google Code and Review Board :)
The other proposal is to use a slot in Roundup proper and the Python
tracker instance. This could have a core Roundup developer as mentor,
under the condition it's focused on Roundup itself. IMO, are some
missing/broken core features in Roundup that would have a positive
impact on our tracker, mostly those relating to searches, security and
UI. I'd have a lot to contribute to the Python tracker part, based on
ongoing work.
Even if neither is considered worthy, I'll keep them on my to-do list
and hope to slowly and hackishly work towards both proposals' goals.
Barring feedback saying that they're out of scope, stupid and
downright offensive, I'll post details on each to this thread very
soon.
So, if the PSF was to use a slot on improving the way you work on
Python development, what would you like to see fixed or implemented?
Best regards,
Daniel
On Wed, 10 Nov 2004, John P Speno wrote:
Hi, sorry for the delayed response.
> While using subprocess (aka popen5), I came across one potential gotcha. I've had
> exceptions ending like this:
>
> File "test.py", line 5, in test
> cmd = popen5.Popen(args, stdout=PIPE)
> File "popen5.py", line 577, in __init__
> data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
> OSError: [Errno 4] Interrupted system call
>
> (on Solaris 9)
>
> Would it make sense for subprocess to use a more robust read() function
> which can handle these cases, i.e. when the parent's read on the pipe
> to the child's stderr is interrupted by a system call, and returns EINTR?
> I imagine it could catch EINTR and EAGAIN and retry the failed read().
I assume you are using signals in your application? The os.read above is
not the only system call that can fail with EINTR. subprocess.py is full
of other system calls that can fail, and I suspect that many other Python
modules are as well.
I've made a patch (attached) to subprocess.py (and test_subprocess.py)
that should guard against EINTR, but I haven't committed it yet. It's
quite large.
Are Python modules supposed to handle EINTR? Why not let the C code handle
this? Or, perhaps the signal module should provide a sigaction function,
so that users can use SA_RESTART.
Index: subprocess.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/subprocess.py,v
retrieving revision 1.8
diff -u -r1.8 subprocess.py
--- subprocess.py 7 Nov 2004 14:30:34 -0000 1.8
+++ subprocess.py 17 Nov 2004 19:42:30 -0000
@@ -888,6 +888,50 @@
pass
+ def _read_no_intr(self, fd, buffersize):
+ """Like os.read, but retries on EINTR"""
+ while True:
+ try:
+ return os.read(fd, buffersize)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
+
+ def _read_all(self, fd, buffersize):
+ """Like os.read, but retries on EINTR, and reads until EOF"""
+ all = ""
+ while True:
+ data = self._read_no_intr(fd, buffersize)
+ all += data
+ if data == "":
+ return all
+
+
+ def _write_no_intr(self, fd, s):
+ """Like os.write, but retries on EINTR"""
+ while True:
+ try:
+ return os.write(fd, s)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
+ def _waitpid_no_intr(self, pid, options):
+ """Like os.waitpid, but retries on EINTR"""
+ while True:
+ try:
+ return os.waitpid(pid, options)
+ except OSError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
def _execute_child(self, args, executable, preexec_fn, close_fds,
cwd, env, universal_newlines,
startupinfo, creationflags, shell,
@@ -963,7 +1007,7 @@
exc_value,
tb)
exc_value.child_traceback = ''.join(exc_lines)
- os.write(errpipe_write, pickle.dumps(exc_value))
+ self._write_no_intr(errpipe_write, pickle.dumps(exc_value))
# This exitcode won't be reported to applications, so it
# really doesn't matter what we return.
@@ -979,7 +1023,7 @@
os.close(errwrite)
# Wait for exec to fail or succeed; possibly raising exception
- data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
+ data = self._read_all(errpipe_read, 1048576) # Exceptions limited to 1 MB
os.close(errpipe_read)
if data != "":
child_exception = pickle.loads(data)
@@ -1003,7 +1047,7 @@
attribute."""
if self.returncode == None:
try:
- pid, sts = os.waitpid(self.pid, os.WNOHANG)
+ pid, sts = self._waitpid_no_intr(self.pid, os.WNOHANG)
if pid == self.pid:
self._handle_exitstatus(sts)
except os.error:
@@ -1015,7 +1059,7 @@
"""Wait for child process to terminate. Returns returncode
attribute."""
if self.returncode == None:
- pid, sts = os.waitpid(self.pid, 0)
+ pid, sts = self._waitpid_no_intr(self.pid, 0)
self._handle_exitstatus(sts)
return self.returncode
@@ -1049,27 +1093,33 @@
stderr = []
while read_set or write_set:
- rlist, wlist, xlist = select.select(read_set, write_set, [])
+ try:
+ rlist, wlist, xlist = select.select(read_set, write_set, [])
+ except select.error, e:
+ if e[0] == errno.EINTR:
+ continue
+ else:
+ raise
if self.stdin in wlist:
# When select has indicated that the file is writable,
# we can write up to PIPE_BUF bytes without risk
# blocking. POSIX defines PIPE_BUF >= 512
- bytes_written = os.write(self.stdin.fileno(), input[:512])
+ bytes_written = self._write_no_intr(self.stdin.fileno(), input[:512])
input = input[bytes_written:]
if not input:
self.stdin.close()
write_set.remove(self.stdin)
if self.stdout in rlist:
- data = os.read(self.stdout.fileno(), 1024)
+ data = self._read_no_intr(self.stdout.fileno(), 1024)
if data == "":
self.stdout.close()
read_set.remove(self.stdout)
stdout.append(data)
if self.stderr in rlist:
- data = os.read(self.stderr.fileno(), 1024)
+ data = self._read_no_intr(self.stderr.fileno(), 1024)
if data == "":
self.stderr.close()
read_set.remove(self.stderr)
Index: test/test_subprocess.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v
retrieving revision 1.14
diff -u -r1.14 test_subprocess.py
--- test/test_subprocess.py 12 Nov 2004 15:51:48 -0000 1.14
+++ test/test_subprocess.py 17 Nov 2004 19:42:30 -0000
@@ -7,6 +7,7 @@
import tempfile
import time
import re
+import errno
mswindows = (sys.platform == "win32")
@@ -35,6 +36,16 @@
fname = tempfile.mktemp()
return os.open(fname, os.O_RDWR|os.O_CREAT), fname
+ def read_no_intr(self, obj):
+ while True:
+ try:
+ return obj.read()
+ except IOError, e:
+ if e.errno == errno.EINTR:
+ continue
+ else:
+ raise
+
#
# Generic tests
#
@@ -123,7 +134,7 @@
p = subprocess.Popen([sys.executable, "-c",
'import sys; sys.stdout.write("orange")'],
stdout=subprocess.PIPE)
- self.assertEqual(p.stdout.read(), "orange")
+ self.assertEqual(self.read_no_intr(p.stdout), "orange")
def test_stdout_filedes(self):
# stdout is set to open file descriptor
@@ -151,7 +162,7 @@
p = subprocess.Popen([sys.executable, "-c",
'import sys; sys.stderr.write("strawberry")'],
stderr=subprocess.PIPE)
- self.assertEqual(remove_stderr_debug_decorations(p.stderr.read()),
+ self.assertEqual(remove_stderr_debug_decorations(self.read_no_intr(p.stderr)),
"strawberry")
def test_stderr_filedes(self):
@@ -186,7 +197,7 @@
'sys.stderr.write("orange")'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
- output = p.stdout.read()
+ output = self.read_no_intr(p.stdout)
stripped = remove_stderr_debug_decorations(output)
self.assertEqual(stripped, "appleorange")
@@ -220,7 +231,7 @@
stdout=subprocess.PIPE,
cwd=tmpdir)
normcase = os.path.normcase
- self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir))
+ self.assertEqual(normcase(self.read_no_intr(p.stdout)), normcase(tmpdir))
def test_env(self):
newenv = os.environ.copy()
@@ -230,7 +241,7 @@
'sys.stdout.write(os.getenv("FRUIT"))'],
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read(), "orange")
+ self.assertEqual(self.read_no_intr(p.stdout), "orange")
def test_communicate(self):
p = subprocess.Popen([sys.executable, "-c",
@@ -305,7 +316,8 @@
'sys.stdout.write("\\nline6");'],
stdout=subprocess.PIPE,
universal_newlines=1)
- stdout = p.stdout.read()
+
+ stdout = self.read_no_intr(p.stdout)
if hasattr(open, 'newlines'):
# Interpreter with universal newline support
self.assertEqual(stdout,
@@ -343,7 +355,7 @@
def test_no_leaking(self):
# Make sure we leak no resources
- max_handles = 1026 # too much for most UNIX systems
+ max_handles = 10 # too much for most UNIX systems
if mswindows:
max_handles = 65 # a full test is too slow on Windows
for i in range(max_handles):
@@ -424,7 +436,7 @@
'sys.stdout.write(os.getenv("FRUIT"))'],
stdout=subprocess.PIPE,
preexec_fn=lambda: os.putenv("FRUIT", "apple"))
- self.assertEqual(p.stdout.read(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout), "apple")
def test_args_string(self):
# args is a string
@@ -457,7 +469,7 @@
p = subprocess.Popen(["echo $FRUIT"], shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read().strip(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple")
def test_shell_string(self):
# Run command through the shell (string)
@@ -466,7 +478,7 @@
p = subprocess.Popen("echo $FRUIT", shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertEqual(p.stdout.read().strip(), "apple")
+ self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple")
def test_call_string(self):
# call() function with string argument on UNIX
@@ -525,7 +537,7 @@
p = subprocess.Popen(["set"], shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertNotEqual(p.stdout.read().find("physalis"), -1)
+ self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1)
def test_shell_string(self):
# Run command through the shell (string)
@@ -534,7 +546,7 @@
p = subprocess.Popen("set", shell=1,
stdout=subprocess.PIPE,
env=newenv)
- self.assertNotEqual(p.stdout.read().find("physalis"), -1)
+ self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1)
def test_call_string(self):
# call() function with string argument on Windows
/Peter Åstrand <astrand(a)lysator.liu.se>
I've heard some good things about cmake — LLVM, googletest, and Boost
are all looking at switching to it — so I wanted to see if we could
simplify our autoconf+makefile system by using it. The biggest wins I
see from going to cmake are:
1. It can autogenerate the Visual Studio project files instead of
needing them to be maintained separately
2. It lets you write functions and modules without understanding
autoconf's mix of shell and M4.
3. Its generated Makefiles track header dependencies accurately so we
might be able to add private headers efficiently.
However, after trying it out during PyCon and looking over the
previous attempt at
<http://mail.python.org/pipermail/python-dev/2007-July/073912.html>, I
can't recommend switching to it.
A. It has no equivalent of autoheader, so we'd have to maintain
pyconfig.h.in by hand.
B. It does not allow the CMakeLists.txt file control the --help
output. This appears to be an intentional decision
(http://www.cmake.org/pipermail/cmake-promote/2006-May/000095.html).
To replace it, they have an interactive mode (which asks you about
each possible option) and a -LH flag (which runs the whole configure
process and then tells you about the flags you could have set if you
knew about them).
C. It does not allow the CMakeLists.txt file to see the command line,
so we can't stay backward compatible with the configure script, and
we'd have to replace flags like --with-pydebug with
-Dpydebug:bool=true. We could write a separate configure script to
mitigate this and the previous problem, but we'd have to keep that in
sync with CMakeLists.txt manually.
D. It doesn't have an expression language, so to replace
ac_md_system=`echo $ac_sys_system |
tr -d '[/ ]' | tr '[[A-Z]]' '[[a-z]]'`
you have to write:
string(REGEX REPLACE "[/ ]" "" ac_md_system ${ac_sys_system})
string(TOLOWER ${ac_md_system} ac_md_system)
So, at the very least, it doesn't look like a big enough win to
justify switching, and may not be a win at all.
The other popular configure+make replacement is scons. The biggest
objection to scons before even looking at it is that it requires a
working Python interpreter in order to build Python, causing a
bootstrap problem. However, Brett Cannon and I talked, and we think
this is surmountable. First, nearly every desktop system comes with a
Python interpreter, which would avoid the bootstrap for ordinary
development. Second, we need a cross compilation story anyway for
embedded systems, and the same mechanism could be used to get Python
onto a new desktop platform. Third, Jython and IronPython are pretty
mature and either can run scons now or should be able to run it after
some relatively easy tweaks. They could be used to build CPython on a
new platform. I don't intend to look into scons myself any time soon,
but I'd be interested in anyone's experiences who does try it.
Jeffrey
Hello,
There are a couple of ancillary portability concerns due to optimizations which
store system-dependent results of operations between constants in pyc files:
- Issue #5057: code like '\U00012345'[0] is optimized away and its result stored
as a constant in the pyc file, but the result should be different in UCS-2 and
UCS-4 builds.
- Issue #5593: code like 1e16+2.9999 is optimized away and its result stored as
a constant (again), but the result can vary slightly depending on the internal
FPU precision.
These problems have probably been there for a long time and almost no one seems
to complain, but I thought I'd report them here just in case.
Regards
Antoine.
Hi,
just saw that os.defpath for Windows is defined as
Lib/ntpath.py:30:defpath = '.;C:\\bin'
Most Windows machines I saw has no c:\bin directory.
Any reason why it was defined this way ?
Thanks,
Yinon
During the PyCon sprint I tried to make BaseException accept only a single
argument and bind it to BaseException.message . I was successful (see the
p3yk_no_args_on_exc branch), but it was very painful to pull off as anyone
who sat around me the last three days of the sprint will tell you as they
had to listen to me curse incessantly.
Because of the pain that I went through in the transition and thus the
lessons learned, Guido and I discussed it and we think it would be best to
give up on forcing BaseException to accept only a single argument. I think
it is still doable, but requires a multi-release transition period and not
the one that 2.6 -> 3.0 is offering. And so Guido and I plan on deprecating
BaseException.message as its entire point in existence was to help
transition to what we are not going to have happen. =)
Now that means BaseException.message might hold the record for shortest
lived feature as it was only introduced in 2.5 and is now to be deprecated
in 2.6 and removed in 2.7/3.0. =)
Below is PEP 352, revised to reflect the removal of
BaseException.messageand for letting the 'args' attribute stay (along
with suggesting one should
only pass a single argument to BaseException). Basically the interface for
exceptions doesn't really change in 3.0 except for the removal of
__getitem__.
--------------------------------------------------------------------------
PEP: 352
Title: Required Superclass for Exceptions
Version: $Revision: 53592 $
Last-Modified: $Date: 2007-01-28 21:54:11 -0800 (Sun, 28 Jan 2007) $
Author: Brett Cannon <brett(a)python.org>
Guido van Rossum <guido(a)python.org>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Oct-2005
Post-History:
Abstract
========
In Python 2.4 and before, any (classic) class can be raised as an
exception. The plan for 2.5 was to allow new-style classes, but this
makes the problem worse -- it would mean *any* class (or
instance) can be raised! This is a problem as it prevents any
guarantees from being made about the interface of exceptions.
This PEP proposes introducing a new superclass that all raised objects
must inherit from. Imposing the restriction will allow a standard
interface for exceptions to exist that can be relied upon. It also
leads to a known hierarchy for all exceptions to adhere to.
One might counter that requiring a specific base class for a
particular interface is unPythonic. However, in the specific case of
exceptions there's a good reason (which has generally been agreed to
on python-dev): requiring hierarchy helps code that wants to *catch*
exceptions by making it possible to catch *all* exceptions explicitly
by writing ``except BaseException:`` instead of
``except *:``. [#hierarchy-good]_
Introducing a new superclass for exceptions also gives us the chance
to rearrange the exception hierarchy slightly for the better. As it
currently stands, all exceptions in the built-in namespace inherit
from Exception. This is a problem since this includes two exceptions
(KeyboardInterrupt and SystemExit) that often need to be excepted from
the application's exception handling: the default behavior of shutting
the interpreter down without a traceback is usually more desirable than
whatever the application might do (with the possible exception of
applications that emulate Python's interactive command loop with
``>>>`` prompt). Changing it so that these two exceptions inherit
from the common superclass instead of Exception will make it easy for
people to write ``except`` clauses that are not overreaching and not
catch exceptions that should propagate up.
This PEP is based on previous work done for PEP 348 [#pep348]_.
Requiring a Common Superclass
=============================
This PEP proposes introducing a new exception named BaseException that
is a new-style class and has a single attribute, ``args``. Below
is the code as the exception will work in Python 3.0 (how it will
work in Python 2.x is covered in the `Transition Plan`_ section)::
class BaseException(object):
"""Superclass representing the base of the exception hierarchy.
Provides a 'message' attribute that contains either the single
argument to the constructor or the empty string. This attribute
is used in the string representation for the
exception. This is so that it provides the extra details in the
traceback.
"""
def __init__(self, *args):
"""Set the 'message' attribute'"""
self.args = args
def __str__(self):
"""Return the str of 'message'"""
if len(self.args) == 1:
return str(self.args[0])
else:
return str(self.args)
def __repr__(self):
return "%s(*%s)" % (self.__class__.__name__, repr(self.args))
No restriction is placed upon what may be passed in for ``args``
for backwards-compatibility reasons. In practice, though, only
a single string argument should be used. This keeps the string
representation of the exception to be a useful message about the
exception that is human-readable; this is why the ``__str__`` method
special-cases on length-1 ``args`` value. Including programmatic
information (e.g., an error code number) should be stored as a
separate attribute in a subclass.
The ``raise`` statement will be changed to require that any object
passed to it must inherit from BaseException. This will make sure
that all exceptions fall within a single hierarchy that is anchored at
BaseException [#hierarchy-good]_. This also guarantees a basic
interface that is inherited from BaseException. The change to
``raise`` will be enforced starting in Python 3.0 (see the `Transition
Plan`_ below).
With BaseException being the root of the exception hierarchy,
Exception will now inherit from it.
Exception Hierarchy Changes
===========================
With the exception hierarchy now even more important since it has a
basic root, a change to the existing hierarchy is called for. As it
stands now, if one wants to catch all exceptions that signal an error
*and* do not mean the interpreter should be allowed to exit, you must
specify all but two exceptions specifically in an ``except`` clause
or catch the two exceptions separately and then re-raise them and
have all other exceptions fall through to a bare ``except`` clause::
except (KeyboardInterrupt, SystemExit):
raise
except:
...
That is needlessly explicit. This PEP proposes moving
KeyboardInterrupt and SystemExit to inherit directly from
BaseException.
::
- BaseException
|- KeyboardInterrupt
|- SystemExit
|- Exception
|- (all other current built-in exceptions)
Doing this makes catching Exception more reasonable. It would catch
only exceptions that signify errors. Exceptions that signal that the
interpreter should exit will not be caught and thus be allowed to
propagate up and allow the interpreter to terminate.
KeyboardInterrupt has been moved since users typically expect an
application to exit when the press the interrupt key (usually Ctrl-C).
If people have overly broad ``except`` clauses the expected behaviour
does not occur.
SystemExit has been moved for similar reasons. Since the exception is
raised when ``sys.exit()`` is called the interpreter should normally
be allowed to terminate. Unfortunately overly broad ``except``
clauses can prevent the explicitly requested exit from occurring.
To make sure that people catch Exception most of the time, various
parts of the documentation and tutorials will need to be updated to
strongly suggest that Exception be what programmers want to use. Bare
``except`` clauses or catching BaseException directly should be
discouraged based on the fact that KeyboardInterrupt and SystemExit
almost always should be allowed to propagate up.
Transition Plan
===============
Since semantic changes to Python are being proposed, a transition plan
is needed. The goal is to end up with the new semantics being used in
Python 3.0 while providing a smooth transition for 2.x code. All
deprecations mentioned in the plan will lead to the removal of the
semantics starting in the version following the initial deprecation.
Here is BaseException as implemented in the 2.x series::
class BaseException(object):
"""Superclass representing the base of the exception hierarchy.
The __getitem__ method is provided for backwards-compatibility
and will be deprecated at some point.
"""
def __init__(self, *args):
"""Set the 'args' attribute."""
self.args = args
def __str__(self):
"""Return the str of args[0] or args, depending on length."""
return str(self.args[0]
if len(self.args) <= 1
else self.args)
def __repr__(self):
func_args = repr(self.args) if self.args else "()"
return self.__class__.__name__ + func_args
def __getitem__(self, index):
"""Index into arguments passed in during instantiation.
Provided for backwards-compatibility and will be
deprecated.
"""
return self.args[index]
Deprecation of features in Python 2.9 is optional. This is because it
is not known at this time if Python 2.9 (which is slated to be the
last version in the 2.x series) will actively deprecate features that
will not be in 3.0 . It is conceivable that no deprecation warnings
will be used in 2.9 since there could be such a difference between 2.9
and 3.0 that it would make 2.9 too "noisy" in terms of warnings. Thus
the proposed deprecation warnings for Python 2.9 will be revisited
when development of that version begins to determine if they are still
desired.
* Python 2.5 [done]
- all standard exceptions become new-style classes
- introduce BaseException
- Exception, KeyboardInterrupt, and SystemExit inherit from BaseException
- deprecate raising string exceptions
* Python 2.6
- deprecate catching string exceptions
- deprecate ``message`` attribute (see `Retracted Ideas`_)
* Python 2.7
- deprecate raising exceptions that do not inherit from BaseException
- remove ``message`` attribute
* Python 2.8
- deprecate catching exceptions that do not inherit from BaseException
* Python 2.9
- deprecate ``__getitem__`` (optional)
* Python 3.0 [done]
- drop everything that was deprecated above:
+ string exceptions (both raising and catching)
+ all exceptions must inherit from BaseException
+ drop ``__getitem__``
Retracted Ideas
===============
A previous version of this PEP that was implemented in Python 2.5
included a 'message' attribute on BaseException. Its purpose was to
begin a transition to BaseException accepting only a single argument.
This was to tighten the interface and to force people to use
attributes in subclasses to carry arbitrary information with an
exception instead of cramming it all into ``args``.
Unfortunately, while implementing the removal of the ``args``
attribute in Python 3.0 at the PyCon 2007 sprint
[#pycon2007-sprint-email]_, it was discovered that the transition was
very painful, especially for C extension modules. It was decided that
it would be better to deprecate the ``message`` attribute in
Python 2.6 (and remove in Python 2.7 and Python 3.0) and consider a
more long-term transition strategy in Python 3.0 to remove
multiple-argument support in BaseException in preference of accepting
only a single argument. Thus the introduction of ``message`` and the
original deprecation of ``args`` has been retracted.
References
==========
.. [#pep348] PEP 348 (Exception Reorganization for Python 3.0)
http://www.python.org/peps/pep-0348.html
.. [#hierarchy-good] python-dev Summary for 2004-08-01 through 2004-08-15
http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception...
.. [#SF_1104669] SF patch #1104669 (new-style exceptions)
http://www.python.org/sf/1104669
.. [#pycon2007-sprint-email] python-3000 email ("How far to go with
cleaning up exceptions")
http://mail.python.org/pipermail/python-3000/2007-March/005911.html
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:
Michael Foord wrote:
> Chris Withers wrote:
>> ... love to see ... but ... not optimistic
>> - python to grow a decent, cross platform, package management system
As stated, this may be impossible, because of the difference in what a
package should mean on Windows vs Unix.
If you just mean a way to add python packages from pypi as with
EasyInstall, then maybe.
>> - the standard library to actually shrink to a point where only
>> libraries that are not released elsewhere are included
In some environments, each new component must be approved. Once
python is approved, the standard library is OK, but adding 7 packages
from pypi requires 7 more sets of approvals.
On the other hand, if there were a way to say "The PSF explicitly
endorses Packages X, Y, and Z as worthy of the stdlib; they are
distributed separately for administrative reasons", then the initial
request could be for "Python plus officially endorsed addons"
That said, it may make sense to just give greater prominence to
existing repackagers, such as ActiveState or Enthought.
> If a library is well maintained then there seems to be little point in
> moving it into the standard library
The official endorsement is in many cases more important than shared
distribution.
-jJ
