Mailman 3 December 2015 - Python-checkins

cpython (merge 3.5 -> default): Merge for issue #25500
by brett.cannon Dec. 4, 2015

Dec. 4, 2015

https://hg.python.org/cpython/rev/0259c2c555fb changeset: 99432:0259c2c555fb parent: 99430:ad855c779bf3 parent: 99431:567baf74ebad user: Brett Cannon <brett(a)python.org> date: Fri Dec 04 14:52:07 2015 -0800 summary: Merge for issue #25500 files: Doc/reference/import.rst | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -29,11 +29,10 @@ a name binding operation. When calling :func:`__import__` as part of an import statement, the -import system first checks the module global namespace for a function by -that name. If it is not found, then the standard builtin :func:`__import__` -is called. Other mechanisms for invoking the import system (such as -:func:`importlib.import_module`) do not perform this check and will always -use the standard import system. +standard builtin :func:`__import__` is called. Other mechanisms for +invoking the import system (such as :func:`importlib.import_module`) may +choose to subvert :func:`__import__` and use its own solution to +implement import semantics. When a module is first imported, Python searches for the module and if found, it creates a module object [#fnmo]_, initializing it. If the named module -- Repository URL: https://hg.python.org/cpython

1 0

Benchmark Results for Python 2.7 2015-12-04
by lp_benchmark_robot＠intel.com Dec. 4, 2015

Dec. 4, 2015

Results for project Python 2.7, build date 2015-12-04 04:49:17 +0000 commit: c4e950338e79f475278e4875e4a3a9d607ef7e10 revision date: 2015-12-03 01:23:10 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.34% 0.99% 2.59% 11.27% :-) pybench 0.21% 0.04% 6.12% 7.16% :-( regex_v8 1.39% -0.41% -3.76% 9.58% :-) nbody 0.79% -2.13% 7.58% 5.25% :-) json_dump_v2 0.29% -0.42% 2.33% 15.63% :-( normal_startup 2.01% -0.60% -2.07% 3.26% :-| ssbench 0.52% 0.07% 0.42% 1.96% ---------------------------------------------------------------------------------- Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration.

1 0

Benchmark Results for Python Default 2015-12-04
by lp_benchmark_robot＠intel.com Dec. 4, 2015

Dec. 4, 2015

Results for project Python default, build date 2015-12-04 04:02:09 +0000 commit: ad855c779bf33c59d176c23900ec0bc4fca3dfc6 revision date: 2015-12-04 01:32:05 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.35% -0.81% 9.28% 15.27% :-( pybench 0.19% -0.05% -2.11% 8.38% :-( regex_v8 2.72% 0.04% -3.64% 4.58% :-( nbody 0.10% -0.05% -3.14% 10.75% :-| json_dump_v2 0.20% -0.49% -1.58% 11.13% :-| normal_startup 1.01% -0.12% -0.14% 4.46% ---------------------------------------------------------------------------------- Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration.

1 0

Daily reference leaks (ad855c779bf3): sum=4
by solipsis＠pitrou.net Dec. 4, 2015

Dec. 4, 2015

results for ad855c779bf3 on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog8edhAb', '--timeout', '7200']

1 0

cpython (3.5): Add Awaitable, AsyncIterable, AsyncIterator to typing.py.
by guido.van.rossum Dec. 4, 2015

Dec. 4, 2015

https://hg.python.org/cpython/rev/e9aeae1b2ea9 changeset: 99429:e9aeae1b2ea9 branch: 3.5 parent: 99426:5fa855d20624 user: Guido van Rossum <guido(a)python.org> date: Thu Dec 03 17:31:24 2015 -0800 summary: Add Awaitable, AsyncIterable, AsyncIterator to typing.py. files: Lib/test/test_typing.py | 61 +++++++++++++++++++++++++++++ Lib/typing.py | 15 +++++++ 2 files changed, 76 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -1,3 +1,4 @@ +import asyncio import pickle import re import sys @@ -960,6 +961,36 @@ pass +T_a = TypeVar('T') + + +class AwaitableWrapper(typing.Awaitable[T_a]): + + def __init__(self, value): + self.value = value + + def __await__(self) -> typing.Iterator[T_a]: + yield + return self.value + + +class AsyncIteratorWrapper(typing.AsyncIterator[T_a]): + + def __init__(self, value: typing.Iterable[T_a]): + self.value = value + + def __aiter__(self) -> typing.AsyncIterator[T_a]: + return self + + @asyncio.coroutine + def __anext__(self) -> T_a: + data = yield from self.value + if data: + return data + else: + raise StopAsyncIteration + + class CollectionsAbcTests(TestCase): def test_hashable(self): @@ -984,6 +1015,36 @@ assert isinstance(it, typing.Iterator[int]) assert not isinstance(42, typing.Iterator) + def test_awaitable(self): + async def foo() -> typing.Awaitable[int]: + return await AwaitableWrapper(42) + g = foo() + assert issubclass(type(g), typing.Awaitable[int]) + assert isinstance(g, typing.Awaitable) + assert not isinstance(foo, typing.Awaitable) + assert issubclass(typing.Awaitable[Manager], + typing.Awaitable[Employee]) + assert not issubclass(typing.Awaitable[Employee], + typing.Awaitable[Manager]) + g.send(None) # Run foo() till completion, to avoid warning. + + def test_async_iterable(self): + base_it = range(10) # type: Iterator[int] + it = AsyncIteratorWrapper(base_it) + assert isinstance(it, typing.AsyncIterable) + assert isinstance(it, typing.AsyncIterable) + assert issubclass(typing.AsyncIterable[Manager], + typing.AsyncIterable[Employee]) + assert not isinstance(42, typing.AsyncIterable) + + def test_async_iterator(self): + base_it = range(10) # type: Iterator[int] + it = AsyncIteratorWrapper(base_it) + assert isinstance(it, typing.AsyncIterator) + assert issubclass(typing.AsyncIterator[Manager], + typing.AsyncIterator[Employee]) + assert not isinstance(42, typing.AsyncIterator) + def test_sized(self): assert isinstance([], typing.Sized) assert not isinstance(42, typing.Sized) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -28,6 +28,9 @@ # ABCs (from collections.abc). 'AbstractSet', # collections.abc.Set. + 'Awaitable', + 'AsyncIterator', + 'AsyncIterable', 'ByteString', 'Container', 'Hashable', @@ -1261,6 +1264,18 @@ Hashable = collections_abc.Hashable # Not generic. +class Awaitable(Generic[T_co], extra=collections_abc.Awaitable): + __slots__ = () + + +class AsyncIterable(Generic[T_co], extra=collections_abc.AsyncIterable): + __slots__ = () + + +class AsyncIterator(AsyncIterable[T_co], extra=collections_abc.AsyncIterator): + __slots__ = () + + class Iterable(Generic[T_co], extra=collections_abc.Iterable): __slots__ = () -- Repository URL: https://hg.python.org/cpython

1 0

cpython (merge 3.5 -> default): Add Awaitable, AsyncIterable, AsyncIterator to typing.py. (Merge 3.5->3.6)
by guido.van.rossum Dec. 4, 2015

Dec. 4, 2015

https://hg.python.org/cpython/rev/ad855c779bf3 changeset: 99430:ad855c779bf3 parent: 99427:ea576db13827 parent: 99429:e9aeae1b2ea9 user: Guido van Rossum <guido(a)dropbox.com> date: Thu Dec 03 17:32:05 2015 -0800 summary: Add Awaitable, AsyncIterable, AsyncIterator to typing.py. (Merge 3.5->3.6) files: Lib/test/test_typing.py | 61 +++++++++++++++++++++++++++++ Lib/typing.py | 15 +++++++ 2 files changed, 76 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -1,3 +1,4 @@ +import asyncio import pickle import re import sys @@ -960,6 +961,36 @@ pass +T_a = TypeVar('T') + + +class AwaitableWrapper(typing.Awaitable[T_a]): + + def __init__(self, value): + self.value = value + + def __await__(self) -> typing.Iterator[T_a]: + yield + return self.value + + +class AsyncIteratorWrapper(typing.AsyncIterator[T_a]): + + def __init__(self, value: typing.Iterable[T_a]): + self.value = value + + def __aiter__(self) -> typing.AsyncIterator[T_a]: + return self + + @asyncio.coroutine + def __anext__(self) -> T_a: + data = yield from self.value + if data: + return data + else: + raise StopAsyncIteration + + class CollectionsAbcTests(TestCase): def test_hashable(self): @@ -984,6 +1015,36 @@ assert isinstance(it, typing.Iterator[int]) assert not isinstance(42, typing.Iterator) + def test_awaitable(self): + async def foo() -> typing.Awaitable[int]: + return await AwaitableWrapper(42) + g = foo() + assert issubclass(type(g), typing.Awaitable[int]) + assert isinstance(g, typing.Awaitable) + assert not isinstance(foo, typing.Awaitable) + assert issubclass(typing.Awaitable[Manager], + typing.Awaitable[Employee]) + assert not issubclass(typing.Awaitable[Employee], + typing.Awaitable[Manager]) + g.send(None) # Run foo() till completion, to avoid warning. + + def test_async_iterable(self): + base_it = range(10) # type: Iterator[int] + it = AsyncIteratorWrapper(base_it) + assert isinstance(it, typing.AsyncIterable) + assert isinstance(it, typing.AsyncIterable) + assert issubclass(typing.AsyncIterable[Manager], + typing.AsyncIterable[Employee]) + assert not isinstance(42, typing.AsyncIterable) + + def test_async_iterator(self): + base_it = range(10) # type: Iterator[int] + it = AsyncIteratorWrapper(base_it) + assert isinstance(it, typing.AsyncIterator) + assert issubclass(typing.AsyncIterator[Manager], + typing.AsyncIterator[Employee]) + assert not isinstance(42, typing.AsyncIterator) + def test_sized(self): assert isinstance([], typing.Sized) assert not isinstance(42, typing.Sized) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -28,6 +28,9 @@ # ABCs (from collections.abc). 'AbstractSet', # collections.abc.Set. + 'Awaitable', + 'AsyncIterator', + 'AsyncIterable', 'ByteString', 'Container', 'Hashable', @@ -1261,6 +1264,18 @@ Hashable = collections_abc.Hashable # Not generic. +class Awaitable(Generic[T_co], extra=collections_abc.Awaitable): + __slots__ = () + + +class AsyncIterable(Generic[T_co], extra=collections_abc.AsyncIterable): + __slots__ = () + + +class AsyncIterator(AsyncIterable[T_co], extra=collections_abc.AsyncIterator): + __slots__ = () + + class Iterable(Generic[T_co], extra=collections_abc.Iterable): __slots__ = () -- Repository URL: https://hg.python.org/cpython

1 0

cpython (2.7): Issue #14285: Do not catch ImportError from __init__.py in runpy
by martin.panter Dec. 4, 2015

Dec. 4, 2015

https://hg.python.org/cpython/rev/c4e950338e79 changeset: 99428:c4e950338e79 branch: 2.7 parent: 99424:2ae5c51c5dea user: Martin Panter <vadmium+py(a)gmail.com> date: Thu Dec 03 01:23:10 2015 +0000 summary: Issue #14285: Do not catch ImportError from __init__.py in runpy Initialize package before calling get_loader() for __main__, so that we do not incorrectly handle ImportError from __init__.py. When runpy is used from the Python CLI, use an internal exception rather than ImportError, to avoid catching an unexpected ImportError. Also simplify message formatting: str() is redundant with %s. Also fix test_dash_m_error_code_is_one() in test_cmd_line_script, which was failing because the test package was not in the current directlry, rather the desired ValueError. files: Lib/runpy.py | 42 ++++++++---- Lib/test/script_helper.py | 4 +- Lib/test/test_cmd_line_script.py | 64 ++++++++++++++++++- Lib/test/test_runpy.py | 24 +++++++ Misc/NEWS | 4 + 5 files changed, 115 insertions(+), 23 deletions(-) diff --git a/Lib/runpy.py b/Lib/runpy.py --- a/Lib/runpy.py +++ b/Lib/runpy.py @@ -97,27 +97,35 @@ return None # Helper to get the loader, code and filename for a module -def _get_module_details(mod_name): - loader = get_loader(mod_name) - if loader is None: - raise ImportError("No module named %s" % mod_name) - if loader.is_package(mod_name): +def _get_module_details(mod_name, error=ImportError): + try: + loader = get_loader(mod_name) + if loader is None: + raise error("No module named %s" % mod_name) + ispkg = loader.is_package(mod_name) + except ImportError as e: + raise error(format(e)) + if ispkg: if mod_name == "__main__" or mod_name.endswith(".__main__"): - raise ImportError("Cannot use package as __main__ module") + raise error("Cannot use package as __main__ module") + __import__(mod_name) # Do not catch exceptions initializing package try: pkg_main_name = mod_name + ".__main__" return _get_module_details(pkg_main_name) except ImportError, e: - raise ImportError(("%s; %r is a package and cannot " + + raise error(("%s; %r is a package and cannot " + "be directly executed") %(e, mod_name)) - code = loader.get_code(mod_name) + try: + code = loader.get_code(mod_name) + except ImportError as e: + raise error(format(e)) if code is None: - raise ImportError("No code object available for %s" % mod_name) + raise error("No code object available for %s" % mod_name) filename = _get_filename(loader, mod_name) return mod_name, loader, code, filename -def _get_main_module_details(): +def _get_main_module_details(error=ImportError): # Helper that gives a nicer error message when attempting to # execute a zipfile or directory by invoking __main__.py main_name = "__main__" @@ -125,10 +133,13 @@ return _get_module_details(main_name) except ImportError as exc: if main_name in str(exc): - raise ImportError("can't find %r module in %r" % + raise error("can't find %r module in %r" % (main_name, sys.path[0])) raise +class _Error(Exception): + """Error that _run_module_as_main() should report without a traceback""" + # This function is the actual implementation of the -m switch and direct # execution of zipfiles and directories and is deliberately kept private. # This avoids a repeat of the situation where run_module() no longer met the @@ -148,11 +159,12 @@ """ try: if alter_argv or mod_name != "__main__": # i.e. -m switch - mod_name, loader, code, fname = _get_module_details(mod_name) + mod_name, loader, code, fname = _get_module_details( + mod_name, _Error) else: # i.e. directory or zipfile execution - mod_name, loader, code, fname = _get_main_module_details() - except ImportError as exc: - msg = "%s: %s" % (sys.executable, str(exc)) + mod_name, loader, code, fname = _get_main_module_details(_Error) + except _Error as exc: + msg = "%s: %s" % (sys.executable, exc) sys.exit(msg) pkg_name = mod_name.rpartition('.')[0] main_globals = sys.modules["__main__"].__dict__ diff --git a/Lib/test/script_helper.py b/Lib/test/script_helper.py --- a/Lib/test/script_helper.py +++ b/Lib/test/script_helper.py @@ -134,9 +134,9 @@ # zip_file.close() return zip_name, os.path.join(zip_name, name_in_zip) -def make_pkg(pkg_dir): +def make_pkg(pkg_dir, init_source=''): os.mkdir(pkg_dir) - make_script(pkg_dir, '__init__', '') + make_script(pkg_dir, '__init__', init_source) def make_zip_pkg(zip_dir, zip_basename, pkg_name, script_basename, source, depth=1, compiled=False): diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -1,5 +1,6 @@ # Tests command line execution of scripts +import contextlib import unittest import os import os.path @@ -207,18 +208,69 @@ launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg') self._check_import_error(launch_name, msg) + @contextlib.contextmanager + def setup_test_pkg(self, *args): + with temp_dir() as script_dir, \ + test.test_support.change_cwd(script_dir): + pkg_dir = os.path.join(script_dir, 'test_pkg') + make_pkg(pkg_dir, *args) + yield pkg_dir + + def check_dash_m_failure(self, *args): + rc, out, err = assert_python_failure('-m', *args) + if verbose > 1: + print(out) + self.assertEqual(rc, 1) + return err + def test_dash_m_error_code_is_one(self): # If a module is invoked with the -m command line flag # and results in an error that the return code to the # shell is '1' - with temp_dir() as script_dir: - pkg_dir = os.path.join(script_dir, 'test_pkg') - make_pkg(pkg_dir) + with self.setup_test_pkg() as pkg_dir: script_name = _make_test_script(pkg_dir, 'other', "if __name__ == '__main__': raise ValueError") - rc, out, err = assert_python_failure('-m', 'test_pkg.other', *example_args) - if verbose > 1: - print(out) + err = self.check_dash_m_failure('test_pkg.other', *example_args) + self.assertIn(b'ValueError', err) + + def test_dash_m_errors(self): + # Exercise error reporting for various invalid package executions + tests = ( + ('__builtin__', br'No code object available'), + ('__builtin__.x', br'No module named'), + ('__builtin__.x.y', br'No module named'), + ('os.path', br'Loader.*cannot handle'), + ('importlib', br'No module named.*' + br'is a package and cannot be directly executed'), + ('importlib.nonexistant', br'No module named'), + ) + for name, regex in tests: + rc, _, err = assert_python_failure('-m', name) self.assertEqual(rc, 1) + self.assertRegexpMatches(err, regex) + self.assertNotIn(b'Traceback', err) + + def test_dash_m_init_traceback(self): + # These were wrapped in an ImportError and tracebacks were + # suppressed; see Issue 14285 + exceptions = (ImportError, AttributeError, TypeError, ValueError) + for exception in exceptions: + exception = exception.__name__ + init = "raise {0}('Exception in __init__.py')".format(exception) + with self.setup_test_pkg(init) as pkg_dir: + err = self.check_dash_m_failure('test_pkg') + self.assertIn(exception.encode('ascii'), err) + self.assertIn(b'Exception in __init__.py', err) + self.assertIn(b'Traceback', err) + + def test_dash_m_main_traceback(self): + # Ensure that an ImportError's traceback is reported + with self.setup_test_pkg() as pkg_dir: + main = "raise ImportError('Exception in __main__ module')" + _make_test_script(pkg_dir, '__main__', main) + err = self.check_dash_m_failure('test_pkg') + self.assertIn(b'ImportError', err) + self.assertIn(b'Exception in __main__ module', err) + self.assertIn(b'Traceback', err) def test_main(): diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -270,6 +270,30 @@ if verbose: print "Testing package depth:", depth self._check_package(depth) + def test_run_package_init_exceptions(self): + # These were previously wrapped in an ImportError; see Issue 14285 + exceptions = (ImportError, AttributeError, TypeError, ValueError) + for exception in exceptions: + name = exception.__name__ + source = "raise {0}('{0} in __init__.py.')".format(name) + + result = self._make_pkg("", 1, "__main__") + pkg_dir, _, mod_name = result + mod_name = mod_name.replace(".__main__", "") + try: + init = os.path.join(pkg_dir, "__runpy_pkg__", "__init__.py") + with open(init, "wt") as mod_file: + mod_file.write(source) + try: + run_module(mod_name) + except exception as err: + msg = "cannot be directly executed" + self.assertNotIn(msg, format(err)) + else: + self.fail("Nothing raised; expected {}".format(name)) + finally: + self._del_pkg(pkg_dir, 1, mod_name) + def test_explicit_relative_import(self): for depth in range(2, 5): if verbose: print "Testing relative imports at depth:", depth diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,6 +22,10 @@ Library ------- +- Issue #14285: When executing a package with the "python -m package" option, + and package initialization raises ImportError, a proper traceback is now + reported. + - Issue #6478: _strptime's regexp cache now is reset after changing timezone with time.tzset(). -- Repository URL: https://hg.python.org/cpython

1 0

peps: Add Awaitable, AsyncIterable, AsyncIterator to PEP 484 (to be released 3.5.2).
by guido.van.rossum Dec. 3, 2015

Dec. 3, 2015

https://hg.python.org/peps/rev/73dde33e2a4a changeset: 6134:73dde33e2a4a user: Guido van Rossum <guido(a)python.org> date: Thu Dec 03 15:27:36 2015 -0800 summary: Add Awaitable, AsyncIterable, AsyncIterator to PEP 484 (to be released 3.5.2). files: pep-0484.txt | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/pep-0484.txt b/pep-0484.txt --- a/pep-0484.txt +++ b/pep-0484.txt @@ -1261,6 +1261,12 @@ Generic variants of container ABCs (and a few non-containers): +* Awaitable + +* AsyncIterable + +* AsyncIterator + * ByteString * Callable (see above, listed here for completeness) -- Repository URL: https://hg.python.org/peps

1 0

cpython (merge 3.4 -> 3.5): Issue #6478: _strptime's regexp cache now is reset after changing timezone
by serhiy.storchaka Dec. 3, 2015

Dec. 3, 2015

https://hg.python.org/cpython/rev/5fa855d20624 changeset: 99426:5fa855d20624 branch: 3.5 parent: 99421:784a64a21fd0 parent: 99425:4b0a4da1aa27 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 03 22:26:36 2015 +0200 summary: Issue #6478: _strptime's regexp cache now is reset after changing timezone with time.tzset(). files: Lib/_strptime.py | 22 +++++++++----- Lib/test/test_strptime.py | 40 ++++++++++++++++++++++---- Misc/NEWS | 3 ++ 3 files changed, 50 insertions(+), 15 deletions(-) diff --git a/Lib/_strptime.py b/Lib/_strptime.py --- a/Lib/_strptime.py +++ b/Lib/_strptime.py @@ -77,6 +77,8 @@ self.__calc_date_time() if _getlang() != self.lang: raise ValueError("locale changed during initialization") + if time.tzname != self.tzname or time.daylight != self.daylight: + raise ValueError("timezone changed during initialization") def __pad(self, seq, front): # Add '' to seq to either the front (is True), else the back. @@ -161,15 +163,17 @@ def __calc_timezone(self): # Set self.timezone by using time.tzname. - # Do not worry about possibility of time.tzname[0] == timetzname[1] - # and time.daylight; handle that in strptime . + # Do not worry about possibility of time.tzname[0] == time.tzname[1] + # and time.daylight; handle that in strptime. try: time.tzset() except AttributeError: pass - no_saving = frozenset({"utc", "gmt", time.tzname[0].lower()}) - if time.daylight: - has_saving = frozenset({time.tzname[1].lower()}) + self.tzname = time.tzname + self.daylight = time.daylight + no_saving = frozenset({"utc", "gmt", self.tzname[0].lower()}) + if self.daylight: + has_saving = frozenset({self.tzname[1].lower()}) else: has_saving = frozenset() self.timezone = (no_saving, has_saving) @@ -307,13 +311,15 @@ global _TimeRE_cache, _regex_cache with _cache_lock: - - if _getlang() != _TimeRE_cache.locale_time.lang: + locale_time = _TimeRE_cache.locale_time + if (_getlang() != locale_time.lang or + time.tzname != locale_time.tzname or + time.daylight != locale_time.daylight): _TimeRE_cache = TimeRE() _regex_cache.clear() + locale_time = _TimeRE_cache.locale_time if len(_regex_cache) > _CACHE_MAX_SIZE: _regex_cache.clear() - locale_time = _TimeRE_cache.locale_time format_regex = _regex_cache.get(format) if not format_regex: try: diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -4,6 +4,7 @@ import time import locale import re +import os import sys from test import support from datetime import date as datetime_date @@ -324,9 +325,10 @@ tz_name = time.tzname[0] if tz_name.upper() in ("UTC", "GMT"): self.skipTest('need non-UTC/GMT timezone') - try: - original_tzname = time.tzname - original_daylight = time.daylight + + with support.swap_attr(time, 'tzname', (tz_name, tz_name)), \ + support.swap_attr(time, 'daylight', 1), \ + support.swap_attr(time, 'tzset', lambda: None): time.tzname = (tz_name, tz_name) time.daylight = 1 tz_value = _strptime._strptime_time(tz_name, "%Z")[8] @@ -334,9 +336,6 @@ "%s lead to a timezone value of %s instead of -1 when " "time.daylight set to %s and passing in %s" % (time.tzname, tz_value, time.daylight, tz_name)) - finally: - time.tzname = original_tzname - time.daylight = original_daylight def test_date_time(self): # Test %c directive @@ -548,7 +547,7 @@ _strptime._strptime_time("10", "%d") self.assertIsNot(locale_time_id, _strptime._TimeRE_cache.locale_time) - def test_TimeRE_recreation(self): + def test_TimeRE_recreation_locale(self): # The TimeRE instance should be recreated upon changing the locale. locale_info = locale.getlocale(locale.LC_TIME) try: @@ -577,6 +576,33 @@ finally: locale.setlocale(locale.LC_TIME, locale_info) + @support.run_with_tz('STD-1DST') + def test_TimeRE_recreation_timezone(self): + # The TimeRE instance should be recreated upon changing the timezone. + oldtzname = time.tzname + tm = _strptime._strptime_time(time.tzname[0], '%Z') + self.assertEqual(tm.tm_isdst, 0) + tm = _strptime._strptime_time(time.tzname[1], '%Z') + self.assertEqual(tm.tm_isdst, 1) + # Get id of current cache object. + first_time_re = _strptime._TimeRE_cache + # Change the timezone and force a recreation of the cache. + os.environ['TZ'] = 'EST+05EDT,M3.2.0,M11.1.0' + time.tzset() + tm = _strptime._strptime_time(time.tzname[0], '%Z') + self.assertEqual(tm.tm_isdst, 0) + tm = _strptime._strptime_time(time.tzname[1], '%Z') + self.assertEqual(tm.tm_isdst, 1) + # Get the new cache object's id. + second_time_re = _strptime._TimeRE_cache + # They should not be equal. + self.assertIsNot(first_time_re, second_time_re) + # Make sure old names no longer accepted. + with self.assertRaises(ValueError): + _strptime._strptime_time(oldtzname[0], '%Z') + with self.assertRaises(ValueError): + _strptime._strptime_time(oldtzname[1], '%Z') + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,6 +22,9 @@ Library ------- +- Issue #6478: _strptime's regexp cache now is reset after changing timezone + with time.tzset(). + - Issue #14285: When executing a package with the "python -m package" option, and package initialization fails, a proper traceback is now reported. The "runpy" module now lets exceptions from package initialization pass back to -- Repository URL: https://hg.python.org/cpython

1 0

cpython (2.7): Issue #6478: _strptime's regexp cache now is reset after changing timezone
by serhiy.storchaka Dec. 3, 2015

Dec. 3, 2015

https://hg.python.org/cpython/rev/2ae5c51c5dea changeset: 99424:2ae5c51c5dea branch: 2.7 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 03 22:20:45 2015 +0200 summary: Issue #6478: _strptime's regexp cache now is reset after changing timezone with time.tzset(). files: Lib/_strptime.py | 21 ++++++++---- Lib/test/test_strptime.py | 44 +++++++++++++++++++++----- Lib/test/test_support.py | 35 ++++++++++++++++++++- Misc/NEWS | 3 + 4 files changed, 86 insertions(+), 17 deletions(-) diff --git a/Lib/_strptime.py b/Lib/_strptime.py --- a/Lib/_strptime.py +++ b/Lib/_strptime.py @@ -75,6 +75,8 @@ self.__calc_date_time() if _getlang() != self.lang: raise ValueError("locale changed during initialization") + if time.tzname != self.tzname or time.daylight != self.daylight: + raise ValueError("timezone changed during initialization") def __pad(self, seq, front): # Add '' to seq to either the front (is True), else the back. @@ -159,15 +161,17 @@ def __calc_timezone(self): # Set self.timezone by using time.tzname. - # Do not worry about possibility of time.tzname[0] == timetzname[1] - # and time.daylight; handle that in strptime . + # Do not worry about possibility of time.tzname[0] == time.tzname[1] + # and time.daylight; handle that in strptime. try: time.tzset() except AttributeError: pass - no_saving = frozenset(["utc", "gmt", time.tzname[0].lower()]) - if time.daylight: - has_saving = frozenset([time.tzname[1].lower()]) + self.tzname = time.tzname + self.daylight = time.daylight + no_saving = frozenset(["utc", "gmt", self.tzname[0].lower()]) + if self.daylight: + has_saving = frozenset([self.tzname[1].lower()]) else: has_saving = frozenset() self.timezone = (no_saving, has_saving) @@ -296,12 +300,15 @@ """Return a time struct based on the input string and the format string.""" global _TimeRE_cache, _regex_cache with _cache_lock: - if _getlang() != _TimeRE_cache.locale_time.lang: + locale_time = _TimeRE_cache.locale_time + if (_getlang() != locale_time.lang or + time.tzname != locale_time.tzname or + time.daylight != locale_time.daylight): _TimeRE_cache = TimeRE() _regex_cache.clear() + locale_time = _TimeRE_cache.locale_time if len(_regex_cache) > _CACHE_MAX_SIZE: _regex_cache.clear() - locale_time = _TimeRE_cache.locale_time format_regex = _regex_cache.get(format) if not format_regex: try: diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -4,8 +4,9 @@ import time import locale import re +import os import sys -from test import test_support +from test import test_support as support from datetime import date as datetime_date import _strptime @@ -314,9 +315,10 @@ tz_name = time.tzname[0] if tz_name.upper() in ("UTC", "GMT"): self.skipTest('need non-UTC/GMT timezone') - try: - original_tzname = time.tzname - original_daylight = time.daylight + + with support.swap_attr(time, 'tzname', (tz_name, tz_name)), \ + support.swap_attr(time, 'daylight', 1), \ + support.swap_attr(time, 'tzset', lambda: None): time.tzname = (tz_name, tz_name) time.daylight = 1 tz_value = _strptime._strptime_time(tz_name, "%Z")[8] @@ -324,9 +326,6 @@ "%s lead to a timezone value of %s instead of -1 when " "time.daylight set to %s and passing in %s" % (time.tzname, tz_value, time.daylight, tz_name)) - finally: - time.tzname = original_tzname - time.daylight = original_daylight def test_date_time(self): # Test %c directive @@ -538,7 +537,7 @@ _strptime._strptime_time("10", "%d") self.assertIsNot(locale_time_id, _strptime._TimeRE_cache.locale_time) - def test_TimeRE_recreation(self): + def test_TimeRE_recreation_locale(self): # The TimeRE instance should be recreated upon changing the locale. locale_info = locale.getlocale(locale.LC_TIME) try: @@ -567,9 +566,36 @@ finally: locale.setlocale(locale.LC_TIME, locale_info) + @support.run_with_tz('STD-1DST') + def test_TimeRE_recreation_timezone(self): + # The TimeRE instance should be recreated upon changing the timezone. + oldtzname = time.tzname + tm = _strptime._strptime_time(time.tzname[0], '%Z') + self.assertEqual(tm.tm_isdst, 0) + tm = _strptime._strptime_time(time.tzname[1], '%Z') + self.assertEqual(tm.tm_isdst, 1) + # Get id of current cache object. + first_time_re = _strptime._TimeRE_cache + # Change the timezone and force a recreation of the cache. + os.environ['TZ'] = 'EST+05EDT,M3.2.0,M11.1.0' + time.tzset() + tm = _strptime._strptime_time(time.tzname[0], '%Z') + self.assertEqual(tm.tm_isdst, 0) + tm = _strptime._strptime_time(time.tzname[1], '%Z') + self.assertEqual(tm.tm_isdst, 1) + # Get the new cache object's id. + second_time_re = _strptime._TimeRE_cache + # They should not be equal. + self.assertIsNot(first_time_re, second_time_re) + # Make sure old names no longer accepted. + with self.assertRaises(ValueError): + _strptime._strptime_time(oldtzname[0], '%Z') + with self.assertRaises(ValueError): + _strptime._strptime_time(oldtzname[1], '%Z') + def test_main(): - test_support.run_unittest( + support.run_unittest( getlang_Tests, LocaleTime_Tests, TimeRETests, diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -40,7 +40,7 @@ "threading_cleanup", "reap_threads", "start_threads", "cpython_only", "check_impl_detail", "get_attribute", "py3k_bytes", "import_fresh_module", "threading_cleanup", "reap_children", - "strip_python_stderr", "IPV6_ENABLED"] + "strip_python_stderr", "IPV6_ENABLED", "run_with_tz"] class Error(Exception): """Base class for regression test exceptions.""" @@ -1226,6 +1226,39 @@ return decorator #======================================================================= +# Decorator for running a function in a specific timezone, correctly +# resetting it afterwards. + +def run_with_tz(tz): + def decorator(func): + def inner(*args, **kwds): + try: + tzset = time.tzset + except AttributeError: + raise unittest.SkipTest("tzset required") + if 'TZ' in os.environ: + orig_tz = os.environ['TZ'] + else: + orig_tz = None + os.environ['TZ'] = tz + tzset() + + # now run the function, resetting the tz on exceptions + try: + return func(*args, **kwds) + finally: + if orig_tz is None: + del os.environ['TZ'] + else: + os.environ['TZ'] = orig_tz + time.tzset() + + inner.__name__ = func.__name__ + inner.__doc__ = func.__doc__ + return inner + return decorator + +#======================================================================= # Big-memory-test support. Separate from 'resources' because memory use should be configurable. # Some handy shorthands. Note that these are used for byte-limits as well diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,6 +22,9 @@ Library ------- +- Issue #6478: _strptime's regexp cache now is reset after changing timezone + with time.tzset(). + - Issue #25718: Fixed copying object with state with boolean value is false. - Issue #25742: :func:`locale.setlocale` now accepts a Unicode string for -- Repository URL: https://hg.python.org/cpython

1 0