Mailman 3 January 2014 - Python-checkins

cpython: Issue #20311, #20452: poll and epoll now round the timeout away from zero,
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/f9a09b40bc17 changeset: 88848:f9a09b40bc17 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 13:02:44 2014 +0100 summary: Issue #20311, #20452: poll and epoll now round the timeout away from zero, instead of rounding towards zero, in select and selectors modules: select.epoll.poll(), selectors.PollSelector.poll() and selectors.EpollSelector.poll(). For example, a timeout of one microsecond (1e-6) is now rounded to one millisecondi (1e-3), instead of being rounded to zero. Mention the change in Misc/NEWS. files: Misc/NEWS | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,13 @@ Library ------- +- Issue #20311, #20452: poll and epoll now round the timeout away from zero, + instead of rounding towards zero, in select and selectors modules: + select.epoll.poll(), selectors.PollSelector.poll() and + selectors.EpollSelector.poll(). For example, a timeout of one microsecond + (1e-6) is now rounded to one millisecondi (1e-3), instead of being rounded to + zero. + - asyncio: Some refactoring; add write flow control to unix pipes; support wait_for(f, None); don't log broken/disconnected pipes; use ValueError instead of assert for forbidden subprocess_{shell,exec} -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #20452: Remove debug code, no more needed
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/21aa1a224049 changeset: 88847:21aa1a224049 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 12:59:43 2014 +0100 summary: Issue #20452: Remove debug code, no more needed files: Lib/asyncio/base_events.py | 10 +--------- Lib/test/test_asyncio/test_events.py | 14 +------------- 2 files changed, 2 insertions(+), 22 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -620,18 +620,10 @@ timeout = min(timeout, deadline) # TODO: Instrumentation only in debug mode? - # FIXME: don't force log (issue #20452) - if True: #logger.isEnabledFor(logging.INFO): + if logger.isEnabledFor(logging.INFO): t0 = self.time() event_list = self._selector.select(timeout) t1 = self.time() - # FIXME: remove these debug info (issue #20452) - dt = t1-t0 - if timeout is not None and dt < timeout and not event_list: - print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" - % (timeout, dt, dt-timeout), file=sys.__stdout__) - print("WARNING: dt+%.20f > timeout? %s" - % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) if t1-t0 >= 1: level = logging.INFO else: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -28,15 +28,6 @@ from asyncio import selector_events from asyncio import test_utils -# FIXME: remove these info, used for debug purpose (issue #20452) -print("time.monotonic() info: %r" % (time.get_clock_info('monotonic'),)) -try: - SC_CLK_TCK = os.sysconf('SC_CLK_TCK') - print("os.sysconf('SC_CLK_TCK') = %s" % SC_CLK_TCK) -except Exception: - pass -# FIXME: remove these info, used for debug purpose (issue #20452) - def data_file(filename): if hasattr(support, 'TEST_HOME_DIR'): @@ -1186,10 +1177,7 @@ self.loop.run_until_complete(wait()) calls.append(self.loop._run_once_counter) - self.assertEqual(calls, [1, 3, 5, 6], - # FIXME: remove these info, used for debug purpose (issue #20452) - (self.loop._granularity, - self.loop._selector.resolution)) + self.assertEqual(calls, [1, 3, 5, 6]) class SubprocessTestsMixin: -- Repository URL: http://hg.python.org/cpython

1 0

cpython: asyncio: Fix error message in BaseEventLoop.subprocess_shell(). Patch written
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/1b09dcd39f76 changeset: 88846:1b09dcd39f76 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 12:28:30 2014 +0100 summary: asyncio: Fix error message in BaseEventLoop.subprocess_shell(). Patch written by Vajrasky Kok. files: Lib/asyncio/base_events.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -557,7 +557,7 @@ if universal_newlines: raise ValueError("universal_newlines must be False") if not shell: - raise ValueError("shell must be False") + raise ValueError("shell must be True") if bufsize != 0: raise ValueError("bufsize must be 0") protocol = protocol_factory() -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #20452: select and selectors round (again) timeout away from zero for
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/827d948ac6aa changeset: 88845:827d948ac6aa user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 12:12:53 2014 +0100 summary: Issue #20452: select and selectors round (again) timeout away from zero for poll and epoll Improve also debug info to analyze the issue files: Lib/asyncio/base_events.py | 15 +++------ Lib/selectors.py | 10 +++++- Lib/test/test_asyncio/test_events.py | 23 ++++++--------- Modules/selectmodule.c | 4 ++- 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -627,16 +627,11 @@ t1 = self.time() # FIXME: remove these debug info (issue #20452) dt = t1-t0 - if timeout is not None: - if dt < timeout and not event_list: - print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" - % (timeout, dt, dt-timeout), file=sys.__stdout__) - print("WARNING: dt+%.20f > timeout? %s" - % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) - else: - if not event_list: - print("WARNING: selector.select(timeout=%r) took dt=%.20f sec" - % (timeout, dt), file=sys.__stdout__) + if timeout is not None and dt < timeout and not event_list: + print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" + % (timeout, dt, dt-timeout), file=sys.__stdout__) + print("WARNING: dt+%.20f > timeout? %s" + % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) if t1-t0 >= 1: level = logging.INFO else: diff --git a/Lib/selectors.py b/Lib/selectors.py --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -8,6 +8,7 @@ from abc import ABCMeta, abstractmethod, abstractproperty from collections import namedtuple, Mapping import functools +import math import select import sys @@ -369,8 +370,9 @@ elif timeout <= 0: timeout = 0 else: - # Round towards zero - timeout = int(timeout * 1000) + # poll() has a resolution of 1 millisecond, round away from + # zero to wait *at least* timeout seconds. + timeout = int(math.ceil(timeout * 1e3)) ready = [] try: fd_event_list = self._poll.poll(timeout) @@ -430,6 +432,10 @@ timeout = -1 elif timeout <= 0: timeout = 0 + else: + # epoll_wait() has a resolution of 1 millisecond, round away + # from zero to wait *at least* timeout seconds. + timeout = math.ceil(timeout * 1e3) * 1e-3 max_ev = len(self._fd_to_key) ready = [] try: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -28,6 +28,15 @@ from asyncio import selector_events from asyncio import test_utils +# FIXME: remove these info, used for debug purpose (issue #20452) +print("time.monotonic() info: %r" % (time.get_clock_info('monotonic'),)) +try: + SC_CLK_TCK = os.sysconf('SC_CLK_TCK') + print("os.sysconf('SC_CLK_TCK') = %s" % SC_CLK_TCK) +except Exception: + pass +# FIXME: remove these info, used for debug purpose (issue #20452) + def data_file(filename): if hasattr(support, 'TEST_HOME_DIR'): @@ -1157,11 +1166,6 @@ w.close() def test_timeout_rounding(self): - # FIXME: remove this imports, used for debug purpose (issue #20452) - import time - import platform - import os - def _run_once(): self.loop._run_once_counter += 1 orig_run_once() @@ -1182,17 +1186,10 @@ self.loop.run_until_complete(wait()) calls.append(self.loop._run_once_counter) - try: - SC_CLK_TCK = os.sysconf('SC_CLK_TCK') - except Exception: - SC_CLK_TCK = None self.assertEqual(calls, [1, 3, 5, 6], # FIXME: remove these info, used for debug purpose (issue #20452) (self.loop._granularity, - self.loop._selector.resolution, - time.get_clock_info('monotonic'), - SC_CLK_TCK, - platform.platform())) + self.loop._selector.resolution)) class SubprocessTestsMixin: diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -1458,7 +1458,9 @@ return NULL; } else { - timeout = (int)(dtimeout * 1000.0); + /* epoll_wait() has a resolution of 1 millisecond, round away from zero + to wait *at least* dtimeout seconds. */ + timeout = (int)ceil(dtimeout * 1000.0); } if (maxevents == -1) { -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #20452: Oops, fix debug code :-/
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/83e110399f3c changeset: 88844:83e110399f3c user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 10:55:55 2014 +0100 summary: Issue #20452: Oops, fix debug code :-/ Add also event more debug info files: Lib/asyncio/base_events.py | 15 ++++++++++----- Lib/test/test_asyncio/test_events.py | 6 ++++++ 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -627,11 +627,16 @@ t1 = self.time() # FIXME: remove these debug info (issue #20452) dt = t1-t0 - if dt < timeout and not event_list: - print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" - % (timeout, dt, dt-timeout), file=sys.__stdout__) - print("WARNING: dt+%.20f > timeout? %s" - % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) + if timeout is not None: + if dt < timeout and not event_list: + print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" + % (timeout, dt, dt-timeout), file=sys.__stdout__) + print("WARNING: dt+%.20f > timeout? %s" + % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) + else: + if not event_list: + print("WARNING: selector.select(timeout=%r) took dt=%.20f sec" + % (timeout, dt), file=sys.__stdout__) if t1-t0 >= 1: level = logging.INFO else: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1160,6 +1160,7 @@ # FIXME: remove this imports, used for debug purpose (issue #20452) import time import platform + import os def _run_once(): self.loop._run_once_counter += 1 @@ -1181,11 +1182,16 @@ self.loop.run_until_complete(wait()) calls.append(self.loop._run_once_counter) + try: + SC_CLK_TCK = os.sysconf('SC_CLK_TCK') + except Exception: + SC_CLK_TCK = None self.assertEqual(calls, [1, 3, 5, 6], # FIXME: remove these info, used for debug purpose (issue #20452) (self.loop._granularity, self.loop._selector.resolution, time.get_clock_info('monotonic'), + SC_CLK_TCK, platform.platform())) -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (c30a3a6f3b39): sum=-4
by solipsis＠pitrou.net Jan. 31, 2014

Jan. 31, 2014

results for c30a3a6f3b39 on branch "default" -------------------------------------------- test_site leaked [-2, 2, -2] references, sum=-2 test_site leaked [-2, 2, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog2EW4tT', '-x']

1 0

cpython: Issue #20452: add more info in case of test_asyncio failure to try to debug the
by victor.stinner Jan. 31, 2014

Jan. 31, 2014

http://hg.python.org/cpython/rev/5545634d98a1 changeset: 88843:5545634d98a1 parent: 88841:c30a3a6f3b39 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Jan 31 09:29:35 2014 +0100 summary: Issue #20452: add more info in case of test_asyncio failure to try to debug the failure on buildbot "x86 Ubuntu Shared 3.x" files: Lib/asyncio/base_events.py | 10 +++++++++- Lib/test/test_asyncio/test_events.py | 11 ++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -620,10 +620,18 @@ timeout = min(timeout, deadline) # TODO: Instrumentation only in debug mode? - if logger.isEnabledFor(logging.INFO): + # FIXME: don't force log (issue #20452) + if True: #logger.isEnabledFor(logging.INFO): t0 = self.time() event_list = self._selector.select(timeout) t1 = self.time() + # FIXME: remove these debug info (issue #20452) + dt = t1-t0 + if dt < timeout and not event_list: + print("WARNING: selector.select(timeout=%.20f) took dt=%.20f sec (dt-timeout=%+.20f)" + % (timeout, dt, dt-timeout), file=sys.__stdout__) + print("WARNING: dt+%.20f > timeout? %s" + % (self._granularity, (dt + self._granularity) > timeout), file=sys.__stdout__) if t1-t0 >= 1: level = logging.INFO else: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1157,6 +1157,10 @@ w.close() def test_timeout_rounding(self): + # FIXME: remove this imports, used for debug purpose (issue #20452) + import time + import platform + def _run_once(): self.loop._run_once_counter += 1 orig_run_once() @@ -1177,7 +1181,12 @@ self.loop.run_until_complete(wait()) calls.append(self.loop._run_once_counter) - self.assertEqual(calls, [1, 3, 5, 6]) + self.assertEqual(calls, [1, 3, 5, 6], + # FIXME: remove these info, used for debug purpose (issue #20452) + (self.loop._granularity, + self.loop._selector.resolution, + time.get_clock_info('monotonic'), + platform.platform())) class SubprocessTestsMixin: -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Idle test: 2nd try at suppressing compile time warning (hint by Nick Coghlan).
by terry.reedy Jan. 30, 2014

Jan. 30, 2014

http://hg.python.org/cpython/rev/78a721fa9b67 changeset: 88842:78a721fa9b67 branch: 2.7 parent: 88838:03b399c807d6 user: Terry Jan Reedy <tjreedy(a)udel.edu> date: Thu Jan 30 21:37:24 2014 -0500 summary: Idle test: 2nd try at suppressing compile time warning (hint by Nick Coghlan). files: Lib/idlelib/idle_test/test_calltips.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idle_test/test_calltips.py b/Lib/idlelib/idle_test/test_calltips.py --- a/Lib/idlelib/idle_test/test_calltips.py +++ b/Lib/idlelib/idle_test/test_calltips.py @@ -167,7 +167,7 @@ with warnings.catch_warnings(): # Suppess message of py3 deprecation of parameter unpacking warnings.simplefilter("ignore") - def f((a,b), c=0.0): pass + exec "def f((a,b), c=0.0): pass" self.assertEqual(signature(f), '(<tuple>, c=0.0)') if __name__ == '__main__': -- Repository URL: http://hg.python.org/cpython

1 0

cpython: asyncio: Fix misc whitespace issues.
by guido.van.rossum Jan. 30, 2014

Jan. 30, 2014

http://hg.python.org/cpython/rev/c30a3a6f3b39 changeset: 88841:c30a3a6f3b39 user: Guido van Rossum <guido(a)python.org> date: Thu Jan 30 16:05:28 2014 -0800 summary: asyncio: Fix misc whitespace issues. files: Lib/asyncio/streams.py | 1 + Lib/test/test_asyncio/test_streams.py | 4 +- Lib/test/test_asyncio/test_tasks.py | 23 +++++++------- 3 files changed, 15 insertions(+), 13 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -14,6 +14,7 @@ _DEFAULT_LIMIT = 2**16 + class IncompleteReadError(EOFError): """ Incomplete read error. Attributes: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -415,7 +415,7 @@ server = MyServer(self.loop) server.start() msg = self.loop.run_until_complete(asyncio.Task(client(), - loop=self.loop)) + loop=self.loop)) server.stop() self.assertEqual(msg, b"hello world!\n") @@ -423,7 +423,7 @@ server = MyServer(self.loop) server.start_callback() msg = self.loop.run_until_complete(asyncio.Task(client(), - loop=self.loop)) + loop=self.loop)) server.stop() self.assertEqual(msg, b"hello world!\n") diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -379,7 +379,6 @@ self.assertAlmostEqual(0.1, loop.time()) self.assertEqual(foo_running, False) - def test_wait_for_blocking(self): loop = test_utils.TestLoop() self.addCleanup(loop.close) @@ -388,7 +387,9 @@ def coro(): return 'done' - res = loop.run_until_complete(asyncio.wait_for(coro(), timeout=None, loop=loop)) + res = loop.run_until_complete(asyncio.wait_for(coro(), + timeout=None, + loop=loop)) self.assertEqual(res, 'done') def test_wait_for_with_global_loop(self): @@ -490,7 +491,7 @@ self.assertRaises( ValueError, self.loop.run_until_complete, asyncio.wait([asyncio.sleep(10.0, loop=self.loop)], - return_when=-1, loop=self.loop)) + return_when=-1, loop=self.loop)) def test_wait_first_completed(self): @@ -508,7 +509,7 @@ b = asyncio.Task(asyncio.sleep(0.1, loop=loop), loop=loop) task = asyncio.Task( asyncio.wait([b, a], return_when=asyncio.FIRST_COMPLETED, - loop=loop), + loop=loop), loop=loop) done, pending = loop.run_until_complete(task) @@ -540,7 +541,7 @@ b = asyncio.Task(coro2(), loop=self.loop) task = asyncio.Task( asyncio.wait([b, a], return_when=asyncio.FIRST_COMPLETED, - loop=self.loop), + loop=self.loop), loop=self.loop) done, pending = self.loop.run_until_complete(task) @@ -570,7 +571,7 @@ b = asyncio.Task(exc(), loop=loop) task = asyncio.Task( asyncio.wait([b, a], return_when=asyncio.FIRST_EXCEPTION, - loop=loop), + loop=loop), loop=loop) done, pending = loop.run_until_complete(task) @@ -604,7 +605,7 @@ b = asyncio.Task(exc(), loop=loop) task = asyncio.wait([b, a], return_when=asyncio.FIRST_EXCEPTION, - loop=loop) + loop=loop) done, pending = loop.run_until_complete(task) self.assertEqual({b}, done) @@ -670,7 +671,7 @@ @asyncio.coroutine def foo(): done, pending = yield from asyncio.wait([b, a], timeout=0.11, - loop=loop) + loop=loop) self.assertEqual(done, set([a])) self.assertEqual(pending, set([b])) @@ -874,7 +875,7 @@ self.addCleanup(loop.close) t = asyncio.Task(asyncio.sleep(10.0, 'yeah', loop=loop), - loop=loop) + loop=loop) handle = None orig_call_later = loop.call_later @@ -1156,7 +1157,7 @@ task2 = asyncio.Task(coro2(self.loop), loop=self.loop) self.loop.run_until_complete(asyncio.wait((task1, task2), - loop=self.loop)) + loop=self.loop)) self.assertIsNone(asyncio.Task.current_task(loop=self.loop)) # Some thorough tests for cancellation propagation through @@ -1367,7 +1368,7 @@ def test_return_exceptions(self): a, b, c, d = [asyncio.Future(loop=self.one_loop) for i in range(4)] fut = asyncio.gather(*self.wrap_futures(a, b, c, d), - return_exceptions=True) + return_exceptions=True) cb = Mock() fut.add_done_callback(cb) exc = ZeroDivisionError() -- Repository URL: http://hg.python.org/cpython

1 0

cpython: asyncio: Fix granularity of test_utils.TestLoop.
by guido.van.rossum Jan. 30, 2014

Jan. 30, 2014

http://hg.python.org/cpython/rev/46c1f0e99ff0 changeset: 88840:46c1f0e99ff0 user: Victor Stinner <victor.stinner(a)gmail.com> date: Thu Jan 30 16:05:07 2014 -0800 summary: asyncio: Fix granularity of test_utils.TestLoop. files: Lib/asyncio/test_utils.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py --- a/Lib/asyncio/test_utils.py +++ b/Lib/asyncio/test_utils.py @@ -196,6 +196,7 @@ next(self._gen) self._time = 0 self._timers = [] + self._granularity = 1e-9 self._selector = TestSelector() self.readers = {} -- Repository URL: http://hg.python.org/cpython

1 0