Mailman 3 May 2013 - Python-checkins

cpython: Clarify some documentation
by brett.cannon May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/8483ff12c3e5 changeset: 83962:8483ff12c3e5 user: Brett Cannon <brett(a)python.org> date: Tue May 28 17:50:14 2013 -0400 summary: Clarify some documentation files: Doc/library/importlib.rst | 48 +++++++++----------------- 1 files changed, 17 insertions(+), 31 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -800,7 +800,7 @@ .. decorator:: module_for_loader - A :term:`decorator` for a :term:`loader` method, + A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to handle selecting the proper module object to load with. The decorated method is expected to have a call signature taking two positional arguments @@ -813,20 +813,16 @@ as expected for a :term:`loader`. If the module is not found in :data:`sys.modules` then a new one is constructed with its :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to - **self**, and :attr:`__package__` set if - :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and - does not raise :exc:`ImportError` for **name**. If a new module is not - needed then the module found in :data:`sys.modules` will be passed into the - method. + **self**, and :attr:`__package__` set based on what + :meth:`importlib.abc.InspectLoader.is_package` returns (if available). If a + new module is not needed then the module found in :data:`sys.modules` will + be passed into the method. If an exception is raised by the decorated method and a module was added to :data:`sys.modules` it will be removed to prevent a partially initialized module from being in left in :data:`sys.modules`. If the module was already in :data:`sys.modules` then it is left alone. - Use of this decorator handles all the details of which module object a - loader should initialize as specified by :pep:`302` as best as possible. - .. note:: :class:`ModuleManager` subsumes the module management aspect of this decorator. @@ -837,16 +833,16 @@ .. decorator:: set_loader - A :term:`decorator` for a :term:`loader` method, - to set the :attr:`__loader__` - attribute on loaded modules. If the attribute is already set the decorator - does nothing. It is assumed that the first positional argument to the - wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to. + A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` + to set the :attr:`__loader__` + attribute on the returned module. If the attribute is already set the + decorator does nothing. It is assumed that the first positional argument to + the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set + to. .. note:: - - It is recommended that :func:`module_for_loader` be used over this - decorator as it subsumes this functionality. + As this decorator sets :attr:`__loader__` after loading the module, it is + recommended to use :func:`module_for_loader` instead when appropriate. .. versionchanged:: 3.4 Set ``__loader__`` if set to ``None``, as if the attribute does not @@ -855,19 +851,9 @@ .. decorator:: set_package - A :term:`decorator` for a :term:`loader` to set the :attr:`__package__` - attribute on the module returned by the loader. If :attr:`__package__` is - set and has a value other than ``None`` it will not be changed. - Note that the module returned by the loader is what has the attribute - set on and not the module found in :data:`sys.modules`. - - Reliance on this decorator is discouraged when it is possible to set - :attr:`__package__` before importing. By - setting it beforehand the code for the module is executed with the - attribute set and thus can be used by global level code during - initialization. + A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the :attr:`__package__` attribute on the returned module. If :attr:`__package__` + is set and has a value other than ``None`` it will not be changed. .. note:: - - It is recommended that :func:`module_for_loader` be used over this - decorator as it subsumes this functionality. + As this decorator sets :attr:`__package__` after loading the module, it is + recommended to use :func:`module_for_loader` instead when appropriate. -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Update importlib.h
by brett.cannon May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/a49c22a1bfbd changeset: 83961:a49c22a1bfbd user: Brett Cannon <brett(a)python.org> date: Tue May 28 17:50:04 2013 -0400 summary: Update importlib.h files: Python/importlib.h | 5369 +++++++++++++++---------------- 1 files changed, 2684 insertions(+), 2685 deletions(-) diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Introduce importlib.util.ModuleManager which is a context manager to
by brett.cannon May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/3a1976680717 changeset: 83960:3a1976680717 user: Brett Cannon <brett(a)python.org> date: Tue May 28 17:29:34 2013 -0400 summary: Introduce importlib.util.ModuleManager which is a context manager to handle providing (and cleaning up if needed) the module to be loaded. A future commit will use the context manager in Lib/importlib/_bootstrap.py and thus why the code is placed there instead of in Lib/importlib/util.py. files: Doc/library/importlib.rst | 13 + Lib/importlib/_bootstrap.py | 38 +- Lib/importlib/util.py | 1 + Lib/test/test_importlib/test_util.py | 50 + Misc/NEWS | 3 + Python/importlib.h | 6748 +++++++------ 6 files changed, 3510 insertions(+), 3343 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -789,6 +789,15 @@ .. versionadded:: 3.3 +.. class:: ModuleManager(name) + + A :term:`context manager` which provides the module to load. The module will + either come from :attr:`sys.modules` in the case of reloading or a fresh + module if loading a new module. Proper cleanup of :attr:`sys.modules` occurs + if the module was new and an exception was raised. + + .. versionadded:: 3.4 + .. decorator:: module_for_loader A :term:`decorator` for a :term:`loader` method, @@ -818,6 +827,10 @@ Use of this decorator handles all the details of which module object a loader should initialize as specified by :pep:`302` as best as possible. + .. note:: + :class:`ModuleManager` subsumes the module management aspect of this + decorator. + .. versionchanged:: 3.3 :attr:`__loader__` and :attr:`__package__` are automatically set (when possible). diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -9,7 +9,7 @@ # # IMPORTANT: Whenever making changes to this module, be sure to run # a top-level make in order to get the frozen version of the module -# update. Not doing so, will result in the Makefile to fail for +# update. Not doing so will result in the Makefile to fail for # all others who don't have a ./python around to freeze the module # in the early stages of compilation. # @@ -20,10 +20,6 @@ # reference any injected objects! This includes not only global code but also # anything specified at the class level. -# XXX Make sure all public names have no single leading underscore and all -# others do. - - # Bootstrap-related code ###################################################### _CASE_INSENSITIVE_PLATFORMS = 'win', 'cygwin', 'darwin' @@ -498,6 +494,38 @@ print(message.format(*args), file=sys.stderr) +class ModuleManager: + + """Context manager which returns the module to be loaded. + + Does the proper unloading from sys.modules upon failure. + + """ + + def __init__(self, name): + self._name = name + + def __enter__(self): + self._module = sys.modules.get(self._name) + self._is_reload = self._module is not None + if not self._is_reload: + # This must be done before open() is called as the 'io' module + # implicitly imports 'locale' and would otherwise trigger an + # infinite loop. + self._module = new_module(self._name) + # This must be done before putting the module in sys.modules + # (otherwise an optimization shortcut in import.c becomes wrong) + self._module.__initializing__ = True + sys.modules[self._name] = self._module + return self._module + + def __exit__(self, *args): + self._module.__initializing__ = False + del self._module + if any(arg is not None for arg in args) and not self._is_reload: + del sys.modules[self._name] + + def set_package(fxn): """Set __package__ on the returned module.""" def set_package_wrapper(*args, **kwargs): diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py --- a/Lib/importlib/util.py +++ b/Lib/importlib/util.py @@ -1,5 +1,6 @@ """Utility code for constructing importers, etc.""" +from ._bootstrap import ModuleManager from ._bootstrap import module_for_loader from ._bootstrap import set_loader from ._bootstrap import set_package diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -2,10 +2,60 @@ from . import util as test_util import imp import sys +from test import support import types import unittest +class ModuleManagerTests(unittest.TestCase): + + module_name = 'ModuleManagerTest_module' + + def setUp(self): + support.unload(self.module_name) + self.addCleanup(support.unload, self.module_name) + + def test_new_module(self): + # Test a new module is created, inserted into sys.modules, has + # __initializing__ set to True after entering the context manager, + # and __initializing__ set to False after exiting. + with util.ModuleManager(self.module_name) as module: + self.assertIn(self.module_name, sys.modules) + self.assertIs(sys.modules[self.module_name], module) + self.assertTrue(module.__initializing__) + self.assertFalse(module.__initializing__) + + def test_new_module_failed(self): + # Test the module is removed from sys.modules. + try: + with util.ModuleManager(self.module_name) as module: + self.assertIn(self.module_name, sys.modules) + raise exception + except Exception: + self.assertNotIn(self.module_name, sys.modules) + else: + self.fail('importlib.util.ModuleManager swallowed an exception') + + def test_reload(self): + # Test that the same module is in sys.modules. + created_module = imp.new_module(self.module_name) + sys.modules[self.module_name] = created_module + with util.ModuleManager(self.module_name) as module: + self.assertIs(module, created_module) + + def test_reload_failed(self): + # Test that the module was left in sys.modules. + created_module = imp.new_module(self.module_name) + sys.modules[self.module_name] = created_module + try: + with util.ModuleManager(self.module_name) as module: + raise Exception + except Exception: + self.assertIn(self.module_name, sys.modules) + else: + self.fail('importlib.util.ModuleManager swallowed an exception') + + class ModuleForLoaderTests(unittest.TestCase): """Tests for importlib.util.module_for_loader.""" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -96,6 +96,9 @@ Library ------- +- Add importlib.util.ModuleManager as a context manager to provide the proper + module object to load. + - Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw stream's read() returns more bytes than requested. diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.3 -> default): Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/6d0fd113a2e4 changeset: 83959:6d0fd113a2e4 parent: 83956:ad56dff3602f parent: 83957:0889ab0d0da1 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 22:47:36 2013 +0300 summary: Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. files: Doc/data/refcounts.dat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -932,7 +932,7 @@ PyObject_CallMethodObjArgs:PyObject*::+1: PyObject_CallMethodObjArgs:PyObject*:o:0: -PyObject_CallMethodObjArgs:char*:name:: +PyObject_CallMethodObjArgs:PyObject*:name:0: PyObject_CallMethodObjArgs::...:: PyObject_CallObject:PyObject*::+1: -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/ef9d42b98a3d changeset: 83958:ef9d42b98a3d branch: 2.7 parent: 83936:687295c6c8f2 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 22:46:51 2013 +0300 summary: Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. files: Doc/data/refcounts.dat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -932,7 +932,7 @@ PyObject_CallMethodObjArgs:PyObject*::+1: PyObject_CallMethodObjArgs:PyObject*:o:0: -PyObject_CallMethodObjArgs:char*:name:: +PyObject_CallMethodObjArgs:PyObject*:name:0: PyObject_CallMethodObjArgs::...:: PyObject_CallObject:PyObject*::+1: -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.3): Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat.
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/0889ab0d0da1 changeset: 83957:0889ab0d0da1 branch: 3.3 parent: 83955:b864f4056b78 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 22:46:15 2013 +0300 summary: Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. files: Doc/data/refcounts.dat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -927,7 +927,7 @@ PyObject_CallMethodObjArgs:PyObject*::+1: PyObject_CallMethodObjArgs:PyObject*:o:0: -PyObject_CallMethodObjArgs:char*:name:: +PyObject_CallMethodObjArgs:PyObject*:name:0: PyObject_CallMethodObjArgs::...:: PyObject_CallObject:PyObject*::+1: -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.3 -> default): Merge with 3.3
by terry.reedy May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/c5d4c041ab47 changeset: 83942:c5d4c041ab47 parent: 83940:2ea849fde22b parent: 83941:24c3e7e08168 user: Terry Jan Reedy <tjreedy(a)udel.edu> date: Mon May 27 21:33:40 2013 -0400 summary: Merge with 3.3 files: Lib/idlelib/CallTips.py | 4 +- Lib/idlelib/PathBrowser.py | 3 +- Lib/idlelib/idle_test/(a)README.txt | 63 +++++++++++ Lib/idlelib/idle_test/__init__.py | 9 + Lib/idlelib/idle_test/test_calltips.py | 11 + Lib/idlelib/idle_test/test_pathbrowser.py | 12 ++ Lib/test/test_idle.py | 14 ++ Misc/ACKS | 1 + Misc/NEWS | 3 + 9 files changed, 118 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/CallTips.py b/Lib/idlelib/CallTips.py --- a/Lib/idlelib/CallTips.py +++ b/Lib/idlelib/CallTips.py @@ -264,4 +264,6 @@ print("%d of %d tests failed" % (num_fail, num_tests)) if __name__ == '__main__': - main() + #main() + from unittest import main + main('idlelib.idle_test.test_calltips', verbosity=2, exit=False) diff --git a/Lib/idlelib/PathBrowser.py b/Lib/idlelib/PathBrowser.py --- a/Lib/idlelib/PathBrowser.py +++ b/Lib/idlelib/PathBrowser.py @@ -95,4 +95,5 @@ mainloop() if __name__ == "__main__": - main() + from unittest import main + main('idlelib.idle_test.test_pathbrowser', verbosity=2, exit=False) diff --git a/Lib/idlelib/idle_test/(a)README.txt b/Lib/idlelib/idle_test/(a)README.txt new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle_test/(a)README.txt @@ -0,0 +1,63 @@ +README FOR IDLE TESTS IN IDLELIB.IDLE_TEST + +The idle directory, idlelib, has over 60 xyz.py files. The idle_test +subdirectory should contain a test_xyy.py for each one. (For test modules, +make 'xyz' lower case.) Each should start with the following cut-paste +template, with the blanks after after '.'. 'as', and '_' filled in. +--- +import unittest +import idlelib. as + +class Test_(unittest.TestCase): + + def test_(self): + +if __name__ == '__main__': + unittest.main(verbosity=2, exit=2) +--- +Idle tests are run with unittest; do not use regrtest's test_main. + +Once test_xyy is written, the following should go at the end of xyy.py, +with xyz (lowercased) added after 'test_'. +--- +if __name__ == "__main__": + import unittest + unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False) +--- + +In Idle, pressing F5 in an editor window with either xyz.py or test_xyz.py +loaded will then run the test with the version of Python running Idle and +tracebacks will appear in the Shell window. The options are appropriate for +developers running (as opposed to importing) either type of file during +development: verbosity=2 lists all test_y methods; exit=False avoids a +spurious sys.exit traceback when running in Idle. The following command +lines also run test_xyz.py + +python -m idlelib.xyz # With the capitalization of the xyz module +python -m unittest -v idlelib.idle_test.test_xyz + +To run all idle tests either interactively ('>>>', with unittest imported) +or from a command line, use one of the following. + +>>> unittest.main('idlelib.idle_test', verbosity=2, exit=False) +python -m unittest -v idlelib.idle_test +python -m test.test_idle +python -m test test_idle + +The idle tests are 'discovered' in idlelib.idle_test.__init__.load_tests, +which is also imported into test.test_idle. Normally, neither file should be +changed when working on individual test modules. The last command runs runs +unittest indirectly through regrtest. The same happens when the entire test +suite is run with 'python -m test'. So it must work for buildbots to stay green. + +To run an individual Testcase or test method, extend the +dotted name given to unittest on the command line. + +python -m unittest -v idlelib.idle_test.text_xyz.Test_case.test_meth + +To disable test/test_idle.py, there are at least two choices. +a. Comment out 'load_tests' line, no no tests are discovered (simple and safe); +Running no tests passes, so there is no indication that nothing was run. +b.Before that line, make module an unexpected skip for regrtest with +import unittest; raise unittest.SkipTest('skip for buildbots') +When run directly with unittest, this causes a normal exit and traceback. \ No newline at end of file diff --git a/Lib/idlelib/idle_test/__init__.py b/Lib/idlelib/idle_test/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle_test/__init__.py @@ -0,0 +1,9 @@ +from os.path import dirname + +def load_tests(loader, standard_tests, pattern): + this_dir = dirname(__file__) + top_dir = dirname(dirname(this_dir)) + package_tests = loader.discover(start_dir=this_dir, pattern='test*.py', + top_level_dir=top_dir) + standard_tests.addTests(package_tests) + return standard_tests diff --git a/Lib/idlelib/idle_test/test_calltips.py b/Lib/idlelib/idle_test/test_calltips.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle_test/test_calltips.py @@ -0,0 +1,11 @@ +import unittest +import idlelib.CallTips as ct + +class Test_get_entity(unittest.TestCase): + def test_bad_entity(self): + self.assertIsNone(ct.get_entity('1/0')) + def test_good_entity(self): + self.assertIs(ct.get_entity('int'), int) + +if __name__ == '__main__': + unittest.main(verbosity=2, exit=False) diff --git a/Lib/idlelib/idle_test/test_pathbrowser.py b/Lib/idlelib/idle_test/test_pathbrowser.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle_test/test_pathbrowser.py @@ -0,0 +1,12 @@ +import unittest +import idlelib.PathBrowser as PathBrowser + +class PathBrowserTest(unittest.TestCase): + + def test_DirBrowserTreeItem(self): + # Issue16226 - make sure that getting a sublist works + d = PathBrowser.DirBrowserTreeItem('') + d.GetSubList() + +if __name__ == '__main__': + unittest.main(verbosity=2, exit=False) diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_idle.py @@ -0,0 +1,14 @@ +# Skip test if tkinter wasn't built or idlelib was deleted. +from test.support import import_module +import_module('tkinter') # discard return +itdir = import_module('idlelib.idle_test') + +# Without test_main present, regrtest.runtest_inner (line1219) +# imitates unittest.main by calling +# unittest.TestLoader().loadTestsFromModule(this_module) +# which look for load_tests and uses it if found. +load_tests = itdir.load_tests + +if __name__ == '__main__': + import unittest + unittest.main(verbosity=2, exit=False) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -584,6 +584,7 @@ Bill Janssen Thomas Jarosch Juhana Jauhiainen +Rajagopalasarma Jayakrishnan Zbigniew Jędrzejewski-Szmek Julien Jehannet Drew Jenkins diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ - Issue #17206: Py_CLEAR(), Py_DECREF(), Py_XINCREF() and Py_XDECREF() now expand their arguments once instead of multiple times. Patch written by Illia Polosukhin. +- Issue #15392: Create a unittest framework for IDLE. + Rajagopalasarma Jayakrishnan + - Issue #17937: Try harder to collect cyclic garbage at shutdown. -- Repository URL: http://hg.python.org/cpython

3 2

cpython (merge 3.3 -> default): Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/ad56dff3602f changeset: 83956:ad56dff3602f parent: 83954:96e543ba96a4 parent: 83955:b864f4056b78 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 16:27:08 2013 +0300 summary: Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw stream's read() returns more bytes than requested. files: Lib/test/test_io.py | 9 +++++++++ Misc/NEWS | 3 +++ Modules/_io/bufferedio.c | 8 ++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3027,6 +3027,15 @@ class CMiscIOTest(MiscIOTest): io = io + def test_readinto_buffer_overflow(self): + # Issue #18025 + class BadReader(self.io.BufferedIOBase): + def read(self, n=-1): + return b'x' * 10**6 + bufio = BadReader() + b = bytearray(2) + self.assertRaises(ValueError, bufio.readinto, b) + class PyMiscIOTest(MiscIOTest): io = pyio diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -96,6 +96,9 @@ Library ------- +- Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw + stream's read() returns more bytes than requested. + - Issue #18011: base64.b32decode() now raises a binascii.Error if there are non-alphabet characters present in the input string to conform a docstring. Updated the module documentation. diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -69,6 +69,14 @@ } len = Py_SIZE(data); + if (len > buf.len) { + PyErr_Format(PyExc_ValueError, + "read() returned too much data: " + "%zd bytes requested, %zd returned", + buf.len, len); + Py_DECREF(data); + goto error; + } memcpy(buf.buf, PyBytes_AS_STRING(data), len); PyBuffer_Release(&buf); -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.3): Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/b864f4056b78 changeset: 83955:b864f4056b78 branch: 3.3 parent: 83953:b98380a1d979 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 16:24:45 2013 +0300 summary: Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw stream's read() returns more bytes than requested. files: Lib/test/test_io.py | 9 +++++++++ Misc/NEWS | 3 +++ Modules/_io/bufferedio.c | 8 ++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3019,6 +3019,15 @@ class CMiscIOTest(MiscIOTest): io = io + def test_readinto_buffer_overflow(self): + # Issue #18025 + class BadReader(self.io.BufferedIOBase): + def read(self, n=-1): + return b'x' * 10**6 + bufio = BadReader() + b = bytearray(2) + self.assertRaises(ValueError, bufio.readinto, b) + class PyMiscIOTest(MiscIOTest): io = pyio diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -24,6 +24,9 @@ Library ------- +- Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw + stream's read() returns more bytes than requested. + - Issue #18011: base64.b32decode() now raises a binascii.Error if there are non-alphabet characters present in the input string to conform a docstring. Updated the module documentation. diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -69,6 +69,14 @@ } len = Py_SIZE(data); + if (len > buf.len) { + PyErr_Format(PyExc_ValueError, + "read() returned too much data: " + "%zd bytes requested, %zd returned", + buf.len, len); + Py_DECREF(data); + goto error; + } memcpy(buf.buf, PyBytes_AS_STRING(data), len); PyBuffer_Release(&buf); -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.3 -> default): Issue #17746: Skip test_shutil.test_non_matching_mode when run as root or
by serhiy.storchaka May 28, 2013

May 28, 2013

http://hg.python.org/cpython/rev/96e543ba96a4 changeset: 83954:96e543ba96a4 parent: 83952:29a823f31465 parent: 83953:b98380a1d979 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Tue May 28 15:53:46 2013 +0300 summary: Issue #17746: Skip test_shutil.test_non_matching_mode when run as root or on unsuitable platform/environment. files: Lib/test/test_shutil.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1338,9 +1338,13 @@ # Other platforms: shouldn't match in the current directory. self.assertIsNone(rv) + @unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0, + 'non-root user required') def test_non_matching_mode(self): # Set the file read-only and ask for writeable files. os.chmod(self.temp_file.name, stat.S_IREAD) + if os.access(self.temp_file.name, os.W_OK): + self.skipTest("can't set the file read-only") rv = shutil.which(self.file, path=self.dir, mode=os.W_OK) self.assertIsNone(rv) -- Repository URL: http://hg.python.org/cpython

1 0