Mailman 3 June 2009 - Python-checkins

r73711 - in python/branches/py3k: Lib/tempfile.py
by benjamin.peterson June 30, 2009

June 30, 2009

Author: benjamin.peterson Date: Wed Jul 1 00:27:25 2009 New Revision: 73711 Log: Merged revisions 73710 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r73710 | benjamin.peterson | 2009-06-30 17:14:33 -0500 (Tue, 30 Jun 2009) | 1 line provide a dummy __exit__ on windows ........ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Lib/tempfile.py Modified: python/branches/py3k/Lib/tempfile.py ============================================================================== --- python/branches/py3k/Lib/tempfile.py (original) +++ python/branches/py3k/Lib/tempfile.py Wed Jul 1 00:27:25 2009 @@ -408,6 +408,9 @@ result = self.file.__exit__(exc, value, tb) self.close() return result + else: + def __exit__(self, exc, value, tb): + pass def NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None,

1 0

r73710 - python/trunk/Lib/tempfile.py
by benjamin.peterson June 30, 2009

June 30, 2009

Author: benjamin.peterson Date: Wed Jul 1 00:14:33 2009 New Revision: 73710 Log: provide a dummy __exit__ on windows Modified: python/trunk/Lib/tempfile.py Modified: python/trunk/Lib/tempfile.py ============================================================================== --- python/trunk/Lib/tempfile.py (original) +++ python/trunk/Lib/tempfile.py Wed Jul 1 00:14:33 2009 @@ -411,6 +411,9 @@ result = self.file.__exit__(exc, value, tb) self.close() return result + else: + def __exit__(self, exc, value, tb): + pass def NamedTemporaryFile(mode='w+b', bufsize=-1, suffix="",

1 0

r73709 - peps/trunk/pep-0376.txt
by guido.van.rossum June 30, 2009

June 30, 2009

Author: guido.van.rossum Date: Wed Jul 1 00:10:55 2009 New Revision: 73709 Log: Minor fixes of typos and grammar in the first third of the file. Modified: peps/trunk/pep-0376.txt Modified: peps/trunk/pep-0376.txt ============================================================================== --- peps/trunk/pep-0376.txt (original) +++ peps/trunk/pep-0376.txt Wed Jul 1 00:10:55 2009 @@ -60,12 +60,12 @@ How distributions are installed ------------------------------- -Right now, when a distribution is installed in Python, every element it -contains is installed in various directories. +Right now, when a distribution is installed in Python, the elements it +contains are installed in various directories. The pure Python code, for instance, is installed in the `purelib` directory -which is located in the Python installation at ``lib\python2.6\site-packages`` -for example under Unix-like systems or Mac OS X, and in ``Lib/site-packages`` +which is located in the Python installation at ``lib/python2.6/site-packages`` +for example under Unix-like systems or Mac OS X, and in ``Lib\site-packages`` under Windows. This is done with the Distutils `install` command, which calls various subcommands. @@ -174,12 +174,13 @@ The egg-info directory name is created using a new function called ``egginfo_dirname(name, version)`` added to ``pkgutil``. ``name`` is -converted to a standard distribution name any runs of non-alphanumeric -characters are replaced with a single '-'. ``version`` is converted +converted to a standard distribution name by replacing any runs of +non-alphanumeric characters with a single '-'. ``version`` is converted to a standard version string. Spaces become dots, and all other -non-alphanumeric characters become dashes, with runs of multiple dashes -condensed to a single dash. Both attributes are then converted into their -filename-escaped form. Any '-' characters are currently replaced with '_'. +non-alphanumeric characters (except dots) become dashes, with runs of +multiple dashes condensed to a single dash. Both attributes are then +converted into their filename-escaped form, i.e. any '-' characters are +replaced with '_'. Examples:: @@ -198,10 +199,10 @@ A `RECORD` file will be added inside the `.egg-info` directory at installation time. The `RECORD` file will hold the list of installed files. These correspond to the files listed by the `record` option of the `install` command, and will -be generated by default. This will allow uninstallation, as explained later in this -PEP. The `install` command will also provide an option to prevent the `RECORD` -file from being written and this option should be used when creating system -packages. +be generated by default. This will allow uninstallation, as explained later in +this PEP. The `install` command will also provide an option to prevent the +`RECORD` file from being written and this option should be used when creating +system packages. Third-party installation tools also should not overwrite or delete files that are not in a RECORD file without prompting or warning. @@ -217,7 +218,7 @@ - field delimiter : `,` - quoting char : `"`. -- line terminator : ``os.linesep`` (so ``\r\n`` or ``\r``) +- line terminator : ``os.linesep`` (so ``\r\n`` or ``\n``) Each record is composed of three elements. @@ -226,7 +227,7 @@ - if the installed file is located in the directory where the `.egg-info` directory of the package is located, it will be a '/'-separated relative path, no matter what the target system is. This makes this information - cross-compatible and allows simple installation to be relocatable. + cross-compatible and allows simple installations to be relocatable. - if the installed file is located elsewhere in the system, a '/'-separated absolute path is used.

1 0

buildbot failure in x86 XP-4 trunk
by buildbot＠python.org June 30, 2009

June 30, 2009

The Buildbot has detected a new failure of x86 XP-4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20XP-4%20trunk/builds/2284 Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: bolen-windows Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl,jesse.noller BUILD FAILED: failed test Excerpt from the test logfile: 1 test failed: test_distutils ====================================================================== FAIL: test_build_ext_inplace (distutils.tests.test_build_ext.BuildExtTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\distutils\tests\test_build_ext.py", line 371, in test_build_ext_inplace self.assertEquals(wanted, path) AssertionError: 'E:\\cygwin\\home\\db3l\\buildarea\\trunk.bolen-windows\\build\\PCbuild\\src\\lxml\\etree.so' != 'E:\\cygwin\\home\\db3l\\buildarea\\trunk.bolen-windows\\build\\PCbuild\\src\\lxml\\etree.pyd' sincerely, -The Buildbot

1 0

r73708 - in python/trunk: Doc/library/multiprocessing.rst Lib/multiprocessing/process.py Lib/test/test_multiprocessing.py Misc/ACKS Misc/NEWS
by jesse.noller June 30, 2009

June 30, 2009

Author: jesse.noller Date: Tue Jun 30 19:11:52 2009 New Revision: 73708 Log: Resolves issues 5155, 5313, 5331 - bad file descriptor error with processes in processes Modified: python/trunk/Doc/library/multiprocessing.rst python/trunk/Lib/multiprocessing/process.py python/trunk/Lib/test/test_multiprocessing.py python/trunk/Misc/ACKS python/trunk/Misc/NEWS Modified: python/trunk/Doc/library/multiprocessing.rst ============================================================================== --- python/trunk/Doc/library/multiprocessing.rst (original) +++ python/trunk/Doc/library/multiprocessing.rst Tue Jun 30 19:11:52 2009 @@ -2101,6 +2101,38 @@ for i in range(10): Process(target=f, args=(lock,)).start() +Beware replacing sys.stdin with a "file like object" + + :mod:`multiprocessing` originally unconditionally called:: + + os.close(sys.stdin.fileno()) + + In the :meth:`multiprocessing.Process._bootstrap` method of - this resulted + in issues with processes-in-processes. This has been changed to:: + + sys.stdin.close() + sys.stdin = open(os.devnull) + + Which solves the fundamental issue of processes colliding with each other + resulting in a bad file descriptor error, but introduces a potential danger + to applications which replace :func:`sys.stdin` with a "file-like object" + with output buffering, this danger is that if multiple processes call + :func:`close()` on this file-like object, it could result in the same + data being flushed to the object multiple times, resulting in corruption. + + If you write a file-like object and implement your own caching, you can + make it fork-safe by storing the pid whenever you append to the cache, + and discarding the cache when the pid changes. For example:: + + @property + def cache(self): + pid = os.getpid() + if pid != self._pid: + self._pid = pid + self._cache = [] + return self._cache + + For more information, see :issue:`5155`, :issue:`5313` and :issue:`5331` Windows ~~~~~~~ Modified: python/trunk/Lib/multiprocessing/process.py ============================================================================== --- python/trunk/Lib/multiprocessing/process.py (original) +++ python/trunk/Lib/multiprocessing/process.py Tue Jun 30 19:11:52 2009 @@ -220,7 +220,8 @@ self._children = set() self._counter = itertools.count(1) try: - os.close(sys.stdin.fileno()) + sys.stdin.close() + sys.stdin = open(os.devnull) except (OSError, ValueError): pass _current_process = self Modified: python/trunk/Lib/test/test_multiprocessing.py ============================================================================== --- python/trunk/Lib/test/test_multiprocessing.py (original) +++ python/trunk/Lib/test/test_multiprocessing.py Tue Jun 30 19:11:52 2009 @@ -18,6 +18,7 @@ import random import logging import test_support +from StringIO import StringIO _multiprocessing = test_support.import_module('_multiprocessing') @@ -1861,7 +1862,74 @@ p.join() self.assertEqual(self.ns.test, 1) -testcases_other = [OtherTest, TestInvalidHandle, TestInitializers] +# +# Issue 5155, 5313, 5331: Test process in processes +# Verifies os.close(sys.stdin.fileno) vs. sys.stdin.close() behavior +# + +def _ThisSubProcess(q): + try: + item = q.get(block=False) + except Queue.Empty: + pass + +def _TestProcess(q): + queue = multiprocessing.Queue() + subProc = multiprocessing.Process(target=_ThisSubProcess, args=(queue,)) + subProc.start() + subProc.join() + +def _afunc(x): + return x*x + +def pool_in_process(): + pool = multiprocessing.Pool(processes=4) + x = pool.map(_afunc, [1, 2, 3, 4, 5, 6, 7]) + +class _file_like(object): + def __init__(self, delegate): + self._delegate = delegate + self._pid = None + + @property + def cache(self): + pid = os.getpid() + # There are no race conditions since fork keeps only the running thread + if pid != self._pid: + self._pid = pid + self._cache = [] + return self._cache + + def write(self, data): + self.cache.append(data) + + def flush(self): + self._delegate.write(''.join(self.cache)) + self._cache = [] + +class TestStdinBadfiledescriptor(unittest.TestCase): + + def test_queue_in_process(self): + queue = multiprocessing.Queue() + proc = multiprocessing.Process(target=_TestProcess, args=(queue,)) + proc.start() + proc.join() + + def test_pool_in_process(self): + p = multiprocessing.Process(target=pool_in_process) + p.start() + p.join() + + def test_flushing(self): + sio = StringIO() + flike = _file_like(sio) + flike.write('foo') + proc = multiprocessing.Process(target=lambda: flike.flush()) + flike.flush() + assert sio.getvalue() == 'foo' + +testcases_other = [OtherTest, TestInvalidHandle, TestInitializers, + TestStdinBadfiledescriptor] # # Modified: python/trunk/Misc/ACKS ============================================================================== --- python/trunk/Misc/ACKS (original) +++ python/trunk/Misc/ACKS Tue Jun 30 19:11:52 2009 @@ -46,6 +46,7 @@ Ulf Bartelt Nick Bastin Jeff Bauer +Mike Bayer Michael R Bax Anthony Baxter Samuel L. Bayer @@ -183,6 +184,7 @@ Dean Draayer John DuBois Paul Dubois +Graham Dumpleton Quinn Dunkan Robin Dunn Luke Dunstan @@ -553,6 +555,7 @@ Santiago Peres�n Mark Perrego Trevor Perrin +Gabriel de Perthuis Tim Peters Benjamin Peterson Chris Petrilli Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Tue Jun 30 19:11:52 2009 @@ -341,6 +341,10 @@ Library ------- +- Issues #5155, 5313, 5331: multiprocessing.Process._bootstrap was + unconditionally calling "os.close(sys.stdin.fileno())" resulting in file + descriptor errors + - Issue #6365: Distutils build_ext inplace mode was copying the compiled extension in a subdirectory if the extension name had dots.

1 0

buildbot failure in x86 FreeBSD trunk
by buildbot＠python.org June 30, 2009

June 30, 2009

The Buildbot has detected a new failure of x86 FreeBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20FreeBSD%20trunk/builds/2386 Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: bolen-freebsd Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: mark.dickinson BUILD FAILED: failed test Excerpt from the test logfile: Traceback (most recent call last): File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/threading.py", line 524, in __bootstrap_inner self.run() File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/threading.py", line 477, in run self.__target(*self.__args, **self.__kwargs) File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/SocketServer.py", line 224, in serve_forever r, w, e = select.select([self], [], [], poll_interval) error: (4, 'Interrupted system call') Traceback (most recent call last): File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/threading.py", line 524, in __bootstrap_inner self.run() File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/threading.py", line 477, in run self.__target(*self.__args, **self.__kwargs) File "/usr/home/db3l/buildarea/trunk.bolen-freebsd/build/Lib/SocketServer.py", line 224, in serve_forever r, w, e = select.select([self], [], [], poll_interval) error: (4, 'Interrupted system call') sincerely, -The Buildbot

1 0

buildbot failure in sparc solaris10 gcc trunk
by buildbot＠python.org June 30, 2009

June 30, 2009

The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/buil… Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: loewis-sun Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: mark.dickinson BUILD FAILED: failed test Excerpt from the test logfile: Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/threading.py", line 524, in __bootstrap_inner self.run() File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/threading.py", line 477, in run self.__target(*self.__args, **self.__kwargs) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/SocketServer.py", line 224, in serve_forever r, w, e = select.select([self], [], [], poll_interval) error: (4, 'Interrupted system call') 3 tests failed: test_posix test_socket test_socketserver ====================================================================== ERROR: test_getcwd_long_pathnames (test.test_posix.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_posix.py", line 270, in test_getcwd_long_pathnames shutil.rmtree(base_path) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/shutil.py", line 239, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/shutil.py", line 237, in rmtree os.rmdir(path) OSError: [Errno 22] Invalid argument: '/home2/buildbot/slave/trunk.loewis-sun/build/@test_13456_tmp.getcwd/@test_13456_tmp.getcwd' ====================================================================== ERROR: test_getcwd_long_pathnames (test.test_posix.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_posix.py", line 27, in tearDown os.unlink(test_support.TESTFN) OSError: [Errno 2] No such file or directory: '@test_13456_tmp' ====================================================================== FAIL: test_ForkingTCPServer (test.test_socketserver.SocketServerTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 189, in test_ForkingTCPServer self.stream_examine) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 147, in run_server testfunc(svrcls.address_family, addr) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 161, in stream_examine self.assertEquals(buf, TEST_STR) AssertionError: '' != 'hello world\n' ====================================================================== FAIL: test_ForkingUnixStreamServer (test.test_socketserver.SocketServerTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 207, in test_ForkingUnixStreamServer self.stream_examine) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 147, in run_server testfunc(svrcls.address_family, addr) File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 161, in stream_examine self.assertEquals(buf, TEST_STR) AssertionError: '' != 'hello world\n' Traceback (most recent call last): Unhandled exception in thread started by <bound method Thread.__bootstrap of <Thread(test.test_socketserver.ForkingUnixStreamServer serving, stopped daemon 1337)>> File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/threading.py", line 497, in __bootstrap Traceback (most recent call last): [761092 refs] *** Error code 1 sincerely, -The Buildbot

1 0

r73707 - python/trunk/Doc/library/optparse.rst
by georg.brandl June 30, 2009

June 30, 2009

Author: georg.brandl Date: Tue Jun 30 18:35:11 2009 New Revision: 73707 Log: #6371: fix link targets. Modified: python/trunk/Doc/library/optparse.rst Modified: python/trunk/Doc/library/optparse.rst ============================================================================== --- python/trunk/Doc/library/optparse.rst (original) +++ python/trunk/Doc/library/optparse.rst Tue Jun 30 18:35:11 2009 @@ -642,8 +642,8 @@ ``parser.error()`` from your application code. If :mod:`optparse`'s default error-handling behaviour does not suit your needs, -you'll need to subclass OptionParser and override its :meth:`exit` and/or -:meth:`error` methods. +you'll need to subclass OptionParser and override its :meth:`~OptionParser.exit` +and/or :meth:`~OptionParser.error` methods. .. _optparse-putting-it-all-together:

1 0

buildbot failure in x86 osx.5 3.x
by buildbot＠python.org June 30, 2009

June 30, 2009

The Buildbot has detected a new failure of x86 osx.5 3.x. Full details are available at: http://www.python.org/dev/buildbot/all/x86%20osx.5%203.x/builds/1100 Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: heller-x86-osx5 Build Reason: Build Source Stamp: [branch branches/py3k] HEAD Blamelist: mark.dickinson BUILD FAILED: failed test Excerpt from the test logfile: 1 test failed: test_binhex ====================================================================== ERROR: test_binhex (test.test_binhex.BinHexTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/test/test_binhex.py", line 32, in test_binhex binhex.hexbin(self.fname2, self.fname1) File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 445, in hexbin ifp = HexBin(inp) File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 364, in __init__ self._readheader() File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 384, in _readheader rest = self._read(1 + 4 + 4 + 2 + 4 + 4) File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 367, in _read data = self.ifp.read(len) File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 300, in read self._fill(wtd - len(self.post_buffer)) File "/Users/buildbot/buildarea/3.x.heller-x86-osx5/build/Lib/binhex.py", line 337, in _fill binascii.rledecode_hqx(self.pre_buffer[:mark]) binascii.Error: Orphaned RLE code at start make: *** [buildbottest] Error 1 sincerely, -The Buildbot

1 0

r73706 - python/trunk/Doc/library/exceptions.rst
by georg.brandl June 30, 2009

June 30, 2009

Author: georg.brandl Date: Tue Jun 30 18:18:55 2009 New Revision: 73706 Log: #6384: Add a heading for the exception hierarchy. Modified: python/trunk/Doc/library/exceptions.rst Modified: python/trunk/Doc/library/exceptions.rst ============================================================================== --- python/trunk/Doc/library/exceptions.rst (original) +++ python/trunk/Doc/library/exceptions.rst Tue Jun 30 18:18:55 2009 @@ -477,7 +477,10 @@ .. versionadded:: 2.5 -The class hierarchy for built-in exceptions is: +Exception hierarchy +------------------- + +The class hierarchy for built-in exceptions is: .. literalinclude:: ../../Lib/test/exception_hierarchy.txt

1 0