Mailman 3 July 2011 - Python-checkins

cpython (2.7): Fixes #10639: reindent.py should not convert newlines
by jason.coombs July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/f9cf55bbe9b9 changeset: 71536:f9cf55bbe9b9 branch: 2.7 parent: 71533:2229b0422369 user: Jason R. Coombs <jaraco(a)jaraco.com> date: Wed Jul 27 14:05:37 2011 -0400 summary: Fixes #10639: reindent.py should not convert newlines Backport of changeset 070dc6e359fb, reindent.py now will use the newline detected in the original file and will report an error if mixed newlines are encountered. files: Misc/NEWS | 6 ++++++ Tools/scripts/reindent.py | 17 ++++++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,12 @@ functions (*BSD and OS X). Also add new stat file flags for OS X (UF_HIDDEN and UF_COMPRESSED). +Tools/Demos +----------- + +- Issue #10639: reindent.py no longer converts newlines and will raise + an error if attempting to convert a file with mixed newlines. + Tests ----- diff --git a/Tools/scripts/reindent.py b/Tools/scripts/reindent.py --- a/Tools/scripts/reindent.py +++ b/Tools/scripts/reindent.py @@ -35,7 +35,7 @@ The backup file is a copy of the one that is being reindented. The ".bak" file is generated with shutil.copy(), but some corner cases regarding -user/group and permissions could leave the backup file more readable that +user/group and permissions could leave the backup file more readable than you'd prefer. You can always use the --nobackup option to prevent this. """ @@ -44,6 +44,7 @@ import tokenize import os, shutil import sys +import io verbose = 0 recurse = 0 @@ -108,13 +109,19 @@ if verbose: print "checking", file, "...", try: - f = open(file) + f = io.open(file) except IOError, msg: errprint("%s: I/O Error: %s" % (file, str(msg))) return r = Reindenter(f) f.close() + + newline = r.newlines + if isinstance(newline, tuple): + errprint("%s: mixed newlines detected; cannot process file" % file) + return + if r.run(): if verbose: print "changed." @@ -126,7 +133,7 @@ shutil.copyfile(file, bak) if verbose: print "backed up", file, "to", bak - f = open(file, "w") + f = io.open(file, "w", newline=newline) r.write(f) f.close() if verbose: @@ -173,6 +180,10 @@ # indeed, they're our headache! self.stats = [] + # Save the newlines found in the file so they can be used to + # create output without mutating the newlines. + self.newlines = f.newlines + def run(self): tokenize.tokenize(self.getline, self.tokeneater) # Remove trailing empty lines. -- Repository URL: http://hg.python.org/cpython

1 0

cpython: fix doc typo for library/test.rst
by eli.bendersky July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/8989aa5b357c changeset: 71532:8989aa5b357c user: Eli Bendersky <eliben(a)gmail.com> date: Wed Jul 27 20:29:59 2011 +0300 summary: fix doc typo for library/test.rst files: Doc/library/test.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -447,7 +447,7 @@ Module and package deprecation messages are suppressed during this import if *deprecated* is ``True``. - This function will raise :exc:`unittest.SkipTest` is the named module + This function will raise :exc:`unittest.SkipTest` if the named module cannot be imported. Example use:: -- Repository URL: http://hg.python.org/cpython

2 1

cpython (merge 3.2 -> default): Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime.
by charles-francois.natali July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/5f003d725619 changeset: 71535:5f003d725619 parent: 71532:8989aa5b357c parent: 71534:2bc740a83010 user: Charles-François Natali <neologix(a)free.fr> date: Wed Jul 27 19:42:05 2011 +0200 summary: Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. files: Lib/pydoc.py | 4 ++-- Misc/NEWS | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -218,8 +218,8 @@ def synopsis(filename, cache={}): """Get the one-line summary out of a module file.""" mtime = os.stat(filename).st_mtime - lastupdate, result = cache.get(filename, (0, None)) - if lastupdate < mtime: + lastupdate, result = cache.get(filename, (None, None)) + if lastupdate is None or lastupdate < mtime: info = inspect.getmoduleinfo(filename) try: file = tokenize.open(filename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,8 @@ Library ------- +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. + - Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is given as a low fd, it gets overwritten. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime.
by charles-francois.natali July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/2bc740a83010 changeset: 71534:2bc740a83010 branch: 3.2 parent: 71527:c3aebd01a033 user: Charles-François Natali <neologix(a)free.fr> date: Wed Jul 27 19:40:02 2011 +0200 summary: Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. files: Lib/pydoc.py | 4 ++-- Misc/NEWS | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -224,8 +224,8 @@ def synopsis(filename, cache={}): """Get the one-line summary out of a module file.""" mtime = os.stat(filename).st_mtime - lastupdate, result = cache.get(filename, (0, None)) - if lastupdate < mtime: + lastupdate, result = cache.get(filename, (None, None)) + if lastupdate is None or lastupdate < mtime: info = inspect.getmoduleinfo(filename) try: file = tokenize.open(filename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,8 @@ Library ------- +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. + - Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is given as a low fd, it gets overwritten. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): - Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime.
by charles-francois.natali July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/2229b0422369 changeset: 71533:2229b0422369 branch: 2.7 parent: 71530:6119440bae30 user: Charles-François Natali <neologix(a)free.fr> date: Wed Jul 27 19:36:40 2011 +0200 summary: - Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. files: Lib/pydoc.py | 4 ++-- Misc/NEWS | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -212,8 +212,8 @@ def synopsis(filename, cache={}): """Get the one-line summary out of a module file.""" mtime = os.stat(filename).st_mtime - lastupdate, result = cache.get(filename, (0, None)) - if lastupdate < mtime: + lastupdate, result = cache.get(filename, (None, None)) + if lastupdate is None or lastupdate < mtime: info = inspect.getmoduleinfo(filename) try: file = open(filename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,8 @@ Library ------- +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. + - Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is given as a low fd, it gets overwritten. -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #11049: added test_support to regrtest.STDTESTS list
by eli.bendersky July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/0f09b7f90060 changeset: 71531:0f09b7f90060 parent: 71528:842494d73f69 user: Eli Bendersky <eliben(a)gmail.com> date: Wed Jul 27 20:21:45 2011 +0300 summary: Issue #11049: added test_support to regrtest.STDTESTS list files: Lib/test/regrtest.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -800,6 +800,7 @@ 'test_unittest', 'test_doctest', 'test_doctest2', + 'test_support' ] # set of tests that we don't want to be executed when using regrtest -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is
by ross.lagerwall July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/6119440bae30 changeset: 71530:6119440bae30 branch: 2.7 user: Ross Lagerwall <rosslagerwall(a)gmail.com> date: Wed Jul 27 18:54:53 2011 +0200 summary: Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is given as a low fd, it gets overwritten. files: Lib/subprocess.py | 8 +++ Lib/test/test_subprocess.py | 58 +++++++++++++++++++++++++ Misc/NEWS | 3 + 3 files changed, 69 insertions(+), 0 deletions(-) diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1148,6 +1148,14 @@ os.close(errread) os.close(errpipe_read) + # When duping fds, if there arises a situation + # where one of the fds is either 0, 1 or 2, it + # is possible that it is overwritten (#12607). + if c2pwrite == 0: + c2pwrite = os.dup(c2pwrite) + if errwrite == 0 or errwrite == 1: + errwrite = os.dup(errwrite) + # Dup fds for child def _dup2(a, b): # dup2() removes the CLOEXEC flag but diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -877,6 +877,64 @@ # all standard fds closed. self.check_close_std_fds([0, 1, 2]) + def check_swap_fds(self, stdin_no, stdout_no, stderr_no): + # open up some temporary files + temps = [mkstemp() for i in range(3)] + temp_fds = [fd for fd, fname in temps] + try: + # unlink the files -- we won't need to reopen them + for fd, fname in temps: + os.unlink(fname) + + # save a copy of the standard file descriptors + saved_fds = [os.dup(fd) for fd in range(3)] + try: + # duplicate the temp files over the standard fd's 0, 1, 2 + for fd, temp_fd in enumerate(temp_fds): + os.dup2(temp_fd, fd) + + # write some data to what will become stdin, and rewind + os.write(stdin_no, b"STDIN") + os.lseek(stdin_no, 0, 0) + + # now use those files in the given order, so that subprocess + # has to rearrange them in the child + p = subprocess.Popen([sys.executable, "-c", + 'import sys; got = sys.stdin.read();' + 'sys.stdout.write("got %s"%got); sys.stderr.write("err")'], + stdin=stdin_no, + stdout=stdout_no, + stderr=stderr_no) + p.wait() + + for fd in temp_fds: + os.lseek(fd, 0, 0) + + out = os.read(stdout_no, 1024) + err = test_support.strip_python_stderr(os.read(stderr_no, 1024)) + finally: + for std, saved in enumerate(saved_fds): + os.dup2(saved, std) + os.close(saved) + + self.assertEqual(out, b"got STDIN") + self.assertEqual(err, b"err") + + finally: + for fd in temp_fds: + os.close(fd) + + # When duping fds, if there arises a situation where one of the fds is + # either 0, 1 or 2, it is possible that it is overwritten (#12607). + # This tests all combinations of this. + def test_swap_fds(self): + self.check_swap_fds(0, 1, 2) + self.check_swap_fds(0, 2, 1) + self.check_swap_fds(1, 0, 2) + self.check_swap_fds(1, 2, 0) + self.check_swap_fds(2, 0, 1) + self.check_swap_fds(2, 1, 0) + def test_wait_when_sigchild_ignored(self): # NOTE: sigchild_ignore.py may not be an effective test on all OSes. sigchild_ignore = test_support.findfile("sigchild_ignore.py", diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is + given as a low fd, it gets overwritten. + - Issue #12102: Document that buffered files must be flushed before being used with mmap. Patch by Steffen Daode Nurpmeso. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): merge from 3.2 - Fix closes Issue12621 - Fix docstrings of find and rfind
by senthil.kumaran July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/e266415cf42c changeset: 71529:e266415cf42c branch: 2.7 parent: 71526:5da7924c4775 user: Senthil Kumaran <senthil(a)uthcode.com> date: Wed Jul 27 23:36:51 2011 +0800 summary: merge from 3.2 - Fix closes Issue12621 - Fix docstrings of find and rfind methods of bytes/bytearry/unicodeobject. files: Objects/bytearrayobject.c | 4 ++-- Objects/stringobject.c | 4 ++-- Objects/unicodeobject.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1170,7 +1170,7 @@ "B.find(sub [,start [,end]]) -> int\n\ \n\ Return the lowest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1240,7 +1240,7 @@ "B.rfind(sub [,start [,end]]) -> int\n\ \n\ Return the highest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -1727,7 +1727,7 @@ "S.find(sub [,start [,end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1766,7 +1766,7 @@ "S.rfind(sub [,start [,end]]) -> int\n\ \n\ Return the highest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6491,7 +6491,7 @@ "S.find(sub [,start [,end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -7226,7 +7226,7 @@ "S.rfind(sub [,start [,end]]) -> int\n\ \n\ Return the highest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> default): merge from 3.2 - Fix closes Issue12621 - Fix docstrings of find and rfind
by senthil.kumaran July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/842494d73f69 changeset: 71528:842494d73f69 parent: 71525:943d323cb4b8 parent: 71527:c3aebd01a033 user: Senthil Kumaran <senthil(a)uthcode.com> date: Wed Jul 27 23:34:29 2011 +0800 summary: merge from 3.2 - Fix closes Issue12621 - Fix docstrings of find and rfind methods of bytes/bytearry/unicodeobject. files: Objects/bytearrayobject.c | 4 ++-- Objects/bytesobject.c | 4 ++-- Objects/unicodeobject.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1102,7 +1102,7 @@ "B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1196,7 +1196,7 @@ "B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1269,7 +1269,7 @@ "B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in B where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within B[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1308,7 +1308,7 @@ "B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within B[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -7660,7 +7660,7 @@ "S.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -8587,7 +8587,7 @@ "S.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Fix closes Issue12621 - Fix docstrings of find and rfind methods of
by senthil.kumaran July 27, 2011

July 27, 2011

http://hg.python.org/cpython/rev/c3aebd01a033 changeset: 71527:c3aebd01a033 branch: 3.2 parent: 71524:1140b32747c9 user: Senthil Kumaran <senthil(a)uthcode.com> date: Wed Jul 27 23:33:54 2011 +0800 summary: Fix closes Issue12621 - Fix docstrings of find and rfind methods of bytes/bytearry/unicodeobject. files: Objects/bytearrayobject.c | 4 ++-- Objects/bytesobject.c | 4 ++-- Objects/unicodeobject.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1102,7 +1102,7 @@ "B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1172,7 +1172,7 @@ "B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where subsection sub is found,\n\ -such that sub is contained within s[start,end]. Optional\n\ +such that sub is contained within B[start,end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1273,7 +1273,7 @@ "B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in B where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within B[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -1312,7 +1312,7 @@ "B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within B[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -7587,7 +7587,7 @@ "S.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); @@ -8513,7 +8513,7 @@ "S.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in S where substring sub is found,\n\ -such that sub is contained within s[start:end]. Optional\n\ +such that sub is contained within S[start:end]. Optional\n\ arguments start and end are interpreted as in slice notation.\n\ \n\ Return -1 on failure."); -- Repository URL: http://hg.python.org/cpython

1 0