Mailman 3 June 2016 - Python-checkins

cpython (merge 3.5 -> default): Merge 3.5 (copy doc)
by victor.stinner June 30, 2016

June 30, 2016

https://hg.python.org/cpython/rev/8fdf6804614e changeset: 102231:8fdf6804614e parent: 102228:c2e898813409 parent: 102230:54e9ca0a1639 user: Victor Stinner <victor.stinner(a)gmail.com> date: Thu Jun 30 11:50:59 2016 +0200 summary: Merge 3.5 (copy doc) files: Doc/library/copy.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/copy.rst b/Doc/library/copy.rst --- a/Doc/library/copy.rst +++ b/Doc/library/copy.rst @@ -48,7 +… [View More]

1 0

cpython (3.5): Issue #27418: Fixed Tools/importbench/importbench.py.
by serhiy.storchaka June 30, 2016

June 30, 2016

https://hg.python.org/cpython/rev/ba4548916f05 changeset: 102227:ba4548916f05 branch: 3.5 parent: 102224:0aec17c8f434 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jun 30 10:33:17 2016 +0300 summary: Issue #27418: Fixed Tools/importbench/importbench.py. files: Misc/NEWS | 5 +++++ Tools/importbench/importbench.py | 13 ++++++------- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --… [View More]- a/Misc/NEWS +++ b/Misc/NEWS @@ -72,6 +72,11 @@ - Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as an iterable of integers. Now only strings and bytes-like objects are accepted. +Tools/Demos +----------- + +- Issue #27418: Fixed Tools/importbench/importbench.py. + What's New in Python 3.5.2? =========================== diff --git a/Tools/importbench/importbench.py b/Tools/importbench/importbench.py --- a/Tools/importbench/importbench.py +++ b/Tools/importbench/importbench.py @@ -5,7 +5,6 @@ """ from test.test_importlib import util -from test.test_importlib.source import util as source_util import decimal import imp import importlib @@ -65,11 +64,11 @@ try: name = '__importlib_test_benchmark__' # Clears out sys.modules and puts an entry at the front of sys.path. - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: assert not os.path.exists(imp.cache_from_source(mapping[name])) sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) yield from bench(name, lambda: sys.modules.pop(name), repeat=repeat, seconds=seconds) @@ -102,10 +101,10 @@ """Source writing bytecode: small""" assert not sys.dont_write_bytecode name = '__importlib_test_benchmark__' - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) def cleanup(): sys.modules.pop(name) @@ -136,10 +135,10 @@ def source_using_bytecode(seconds, repeat): """Source w/ bytecode: small""" name = '__importlib_test_benchmark__' - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) py_compile.compile(mapping[name]) assert os.path.exists(imp.cache_from_source(mapping[name])) -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

cpython (merge 3.5 -> default): Issue #27418: Fixed Tools/importbench/importbench.py.
by serhiy.storchaka June 30, 2016

June 30, 2016

https://hg.python.org/cpython/rev/c2e898813409 changeset: 102228:c2e898813409 parent: 102226:4ea79767ff75 parent: 102227:ba4548916f05 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jun 30 10:34:52 2016 +0300 summary: Issue #27418: Fixed Tools/importbench/importbench.py. files: Misc/NEWS | 5 +++++ Tools/importbench/importbench.py | 13 ++++++------- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/Misc/… [View More]NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -125,6 +125,11 @@ - Don't use largefile support for GNU/Hurd. +Tools/Demos +----------- + +- Issue #27418: Fixed Tools/importbench/importbench.py. + What's New in Python 3.6.0 alpha 2 ================================== diff --git a/Tools/importbench/importbench.py b/Tools/importbench/importbench.py --- a/Tools/importbench/importbench.py +++ b/Tools/importbench/importbench.py @@ -5,7 +5,6 @@ """ from test.test_importlib import util -from test.test_importlib.source import util as source_util import decimal import imp import importlib @@ -65,11 +64,11 @@ try: name = '__importlib_test_benchmark__' # Clears out sys.modules and puts an entry at the front of sys.path. - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: assert not os.path.exists(imp.cache_from_source(mapping[name])) sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) yield from bench(name, lambda: sys.modules.pop(name), repeat=repeat, seconds=seconds) @@ -102,10 +101,10 @@ """Source writing bytecode: small""" assert not sys.dont_write_bytecode name = '__importlib_test_benchmark__' - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) def cleanup(): sys.modules.pop(name) @@ -136,10 +135,10 @@ def source_using_bytecode(seconds, repeat): """Source w/ bytecode: small""" name = '__importlib_test_benchmark__' - with source_util.create_modules(name) as mapping: + with util.create_modules(name) as mapping: sys.meta_path.append(importlib.machinery.PathFinder) loader = (importlib.machinery.SourceFileLoader, - importlib.machinery.SOURCE_SUFFIXES, True) + importlib.machinery.SOURCE_SUFFIXES) sys.path_hooks.append(importlib.machinery.FileFinder.path_hook(loader)) py_compile.compile(mapping[name]) assert os.path.exists(imp.cache_from_source(mapping[name])) -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

NEUTRAL Benchmark Results for Python 2.7 2016-06-29
by lp_benchmark_robot＠intel.com June 29, 2016

June 29, 2016

No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-06-29 02:46:25 +0000 commit: ccec979392d7 previous commit: e2d0d921d526 revision date: 2016-06-27 16:54:03 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:… [View More]00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% 0.11% 4.69% 3.86% :-) pybench 0.20% 0.10% 5.85% 4.18% :-( regex_v8 0.48% 0.01% -2.41% 11.07% :-) nbody 0.05% 0.03% 8.96% 3.00% :-) json_dump_v2 0.42% 0.81% 2.48% 11.78% :-( normal_startup 1.96% 0.23% -5.72% 1.43% :-| ssbench 0.17% 0.07% 2.00% 1.96% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-… Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. [View Less]

1 0

GOOD Benchmark Results for Python Default 2016-06-29
by lp_benchmark_robot＠intel.com June 29, 2016

June 29, 2016

Results for project Python default, build date 2016-06-29 02:02:48 +0000 commit: 57f3514564f6 previous commit: e5063a82f490 revision date: 2016-06-28 17:12:22 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------… [View More]------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% 2.81% 11.78% 14.88% :-| pybench 0.08% 0.03% 1.53% 8.85% :-( regex_v8 2.66% -0.13% -2.58% 3.61% :-| nbody 0.09% -0.02% -0.44% 12.00% :-| json_dump_v2 0.32% 0.55% -1.48% 11.23% :-| normal_startup 0.73% 0.15% 1.60% 5.66% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-def… Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. [View Less]

1 0

cpython: Issue #26721: Change StreamRequestHandler.wfile to BufferedIOBase
by martin.panter June 29, 2016

June 29, 2016

https://hg.python.org/cpython/rev/4ea79767ff75 changeset: 102226:4ea79767ff75 user: Martin Panter <vadmium+py(a)gmail.com> date: Wed Jun 29 10:12:22 2016 +0000 summary: Issue #26721: Change StreamRequestHandler.wfile to BufferedIOBase files: Doc/library/http.server.rst | 7 +- Doc/library/socketserver.rst | 9 ++ Doc/whatsnew/3.6.rst | 6 + Lib/socketserver.py | 24 ++++++- Lib/test/test_socketserver.py | 79 +++++++++++++++++++++++ … [View More]Lib/wsgiref/simple_server.py | 17 +--- Misc/NEWS | 4 + 7 files changed, 131 insertions(+), 15 deletions(-) diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst --- a/Doc/library/http.server.rst +++ b/Doc/library/http.server.rst @@ -98,8 +98,8 @@ .. attribute:: rfile - Contains an input stream, positioned at the start of the optional input - data. + An :class:`io.BufferedIOBase` input stream, ready to read from + the start of the optional input data. .. attribute:: wfile @@ -107,6 +107,9 @@ client. Proper adherence to the HTTP protocol must be used when writing to this stream. + .. versionchanged:: 3.6 + This is an :class:`io.BufferedIOBase` stream. + :class:`BaseHTTPRequestHandler` has the following attributes: .. attribute:: server_version diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -409,6 +409,15 @@ read or written, respectively, to get the request data or return data to the client. + The :attr:`rfile` attributes of both classes support the + :class:`io.BufferedIOBase` readable interface, and + :attr:`DatagramRequestHandler.wfile` supports the + :class:`io.BufferedIOBase` writable interface. + + .. versionchanged:: 3.6 + :attr:`StreamRequestHandler.wfile` also supports the + :class:`io.BufferedIOBase` writable interface. + Examples -------- diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -373,6 +373,12 @@ protocol. (Contributed by Aviv Palivoda in :issue:`26404`.) +The :attr:`~socketserver.StreamRequestHandler.wfile` attribute of +:class:`~socketserver.StreamRequestHandler` classes now implements +the :class:`io.BufferedIOBase` writable interface. In particular, +calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the +data in full. (Contributed by Martin Panter in :issue:`26721`.) + subprocess ---------- diff --git a/Lib/socketserver.py b/Lib/socketserver.py --- a/Lib/socketserver.py +++ b/Lib/socketserver.py @@ -132,6 +132,7 @@ import threading except ImportError: import dummy_threading as threading +from io import BufferedIOBase from time import monotonic as time __all__ = ["BaseServer", "TCPServer", "UDPServer", @@ -743,7 +744,10 @@ self.connection.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, True) self.rfile = self.connection.makefile('rb', self.rbufsize) - self.wfile = self.connection.makefile('wb', self.wbufsize) + if self.wbufsize == 0: + self.wfile = _SocketWriter(self.connection) + else: + self.wfile = self.connection.makefile('wb', self.wbufsize) def finish(self): if not self.wfile.closed: @@ -756,6 +760,24 @@ self.wfile.close() self.rfile.close() +class _SocketWriter(BufferedIOBase): + """Simple writable BufferedIOBase implementation for a socket + + Does not hold data in a buffer, avoiding any need to call flush().""" + + def __init__(self, sock): + self._sock = sock + + def writable(self): + return True + + def write(self, b): + self._sock.sendall(b) + with memoryview(b) as view: + return view.nbytes + + def fileno(self): + return self._sock.fileno() class DatagramRequestHandler(BaseRequestHandler): diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py --- a/Lib/test/test_socketserver.py +++ b/Lib/test/test_socketserver.py @@ -3,6 +3,7 @@ """ import contextlib +import io import os import select import signal @@ -376,6 +377,84 @@ self.active_children.clear() +class SocketWriterTest(unittest.TestCase): + def test_basics(self): + class Handler(socketserver.StreamRequestHandler): + def handle(self): + self.server.wfile = self.wfile + self.server.wfile_fileno = self.wfile.fileno() + self.server.request_fileno = self.request.fileno() + + server = socketserver.TCPServer((HOST, 0), Handler) + self.addCleanup(server.server_close) + s = socket.socket( + server.address_family, socket.SOCK_STREAM, socket.IPPROTO_TCP) + with s: + s.connect(server.server_address) + server.handle_request() + self.assertIsInstance(server.wfile, io.BufferedIOBase) + self.assertEqual(server.wfile_fileno, server.request_fileno) + + @unittest.skipUnless(threading, 'Threading required for this test.') + def test_write(self): + # Test that wfile.write() sends data immediately, and that it does + # not truncate sends when interrupted by a Unix signal + pthread_kill = test.support.get_attribute(signal, 'pthread_kill') + + class Handler(socketserver.StreamRequestHandler): + def handle(self): + self.server.sent1 = self.wfile.write(b'write data\n') + # Should be sent immediately, without requiring flush() + self.server.received = self.rfile.readline() + big_chunk = bytes(test.support.SOCK_MAX_SIZE) + self.server.sent2 = self.wfile.write(big_chunk) + + server = socketserver.TCPServer((HOST, 0), Handler) + self.addCleanup(server.server_close) + interrupted = threading.Event() + + def signal_handler(signum, frame): + interrupted.set() + + original = signal.signal(signal.SIGUSR1, signal_handler) + self.addCleanup(signal.signal, signal.SIGUSR1, original) + response1 = None + received2 = None + main_thread = threading.get_ident() + + def run_client(): + s = socket.socket(server.address_family, socket.SOCK_STREAM, + socket.IPPROTO_TCP) + with s, s.makefile('rb') as reader: + s.connect(server.server_address) + nonlocal response1 + response1 = reader.readline() + s.sendall(b'client response\n') + + reader.read(100) + # The main thread should now be blocking in a send() syscall. + # But in theory, it could get interrupted by other signals, + # and then retried. So keep sending the signal in a loop, in + # case an earlier signal happens to be delivered at an + # inconvenient moment. + while True: + pthread_kill(main_thread, signal.SIGUSR1) + if interrupted.wait(timeout=float(1)): + break + nonlocal received2 + received2 = len(reader.read()) + + background = threading.Thread(target=run_client) + background.start() + server.handle_request() + background.join() + self.assertEqual(server.sent1, len(response1)) + self.assertEqual(response1, b'write data\n') + self.assertEqual(server.received, b'client response\n') + self.assertEqual(server.sent2, test.support.SOCK_MAX_SIZE) + self.assertEqual(received2, test.support.SOCK_MAX_SIZE - 100) + + class MiscTestCase(unittest.TestCase): def test_all(self): diff --git a/Lib/wsgiref/simple_server.py b/Lib/wsgiref/simple_server.py --- a/Lib/wsgiref/simple_server.py +++ b/Lib/wsgiref/simple_server.py @@ -11,7 +11,6 @@ """ from http.server import BaseHTTPRequestHandler, HTTPServer -from io import BufferedWriter import sys import urllib.parse from wsgiref.handlers import SimpleHandler @@ -127,17 +126,11 @@ if not self.parse_request(): # An error code has been sent, just exit return - # Avoid passing the raw file object wfile, which can do partial - # writes (Issue 24291) - stdout = BufferedWriter(self.wfile) - try: - handler = ServerHandler( - self.rfile, stdout, self.get_stderr(), self.get_environ() - ) - handler.request_handler = self # backpointer for logging - handler.run(self.server.get_app()) - finally: - stdout.detach() + handler = ServerHandler( + self.rfile, self.wfile, self.get_stderr(), self.get_environ() + ) + handler.request_handler = self # backpointer for logging + handler.run(self.server.get_app()) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Library ------- +- Issue #26721: Change the socketserver.StreamRequestHandler.wfile attribute + to implement BufferedIOBase. In particular, the write() method no longer + does partial writes. + - Issue #22115: Added methods trace_add, trace_remove and trace_info in the tkinter.Variable class. They replace old methods trace_variable, trace, trace_vdelete and trace_vinfo that use obsolete Tcl commands and might -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

cpython (merge 3.5 -> default): Fix issue #27402: example for typing did not type-check. (Merge 3.5->3.6)
by guido.van.rossum June 28, 2016

June 28, 2016

https://hg.python.org/cpython/rev/57f3514564f6 changeset: 102225:57f3514564f6 parent: 102223:d96c8b1154da parent: 102224:0aec17c8f434 user: Guido van Rossum <guido(a)dropbox.com> date: Tue Jun 28 10:12:22 2016 -0700 summary: Fix issue #27402: example for typing did not type-check. (Merge 3.5->3.6) files: Doc/library/typing.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/… [View More]

1 0

cpython (3.5): Fix issue #27402: example for typing did not type-check.
by guido.van.rossum June 28, 2016

June 28, 2016

https://hg.python.org/cpython/rev/0aec17c8f434 changeset: 102224:0aec17c8f434 branch: 3.5 parent: 102222:e6525c6ad789 user: Guido van Rossum <guido(a)python.org> date: Tue Jun 28 10:08:17 2016 -0700 summary: Fix issue #27402: example for typing did not type-check. files: Doc/library/typing.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/… [View More]

1 0

cpython (3.5): asyncio: Use socket specs for getaddrinfo() in sock_connect()
by yury.selivanov June 28, 2016

June 28, 2016

https://hg.python.org/cpython/rev/e6525c6ad789 changeset: 102222:e6525c6ad789 branch: 3.5 parent: 102220:100cf3f268ca user: Yury Selivanov <yury(a)magic.io> date: Tue Jun 28 11:00:22 2016 -0400 summary: asyncio: Use socket specs for getaddrinfo() in sock_connect() Patch by Martin Richard. GH PR #365. files: Lib/asyncio/selector_events.py | 3 +- Lib/test/test_asyncio/test_selector_events.py | 11 ++++++++++ 2 files changed, 13 … [View More]

1 0

cpython (merge 3.5 -> default): Merge 3.5 (asyncio)
by yury.selivanov June 28, 2016

June 28, 2016

https://hg.python.org/cpython/rev/d96c8b1154da changeset: 102223:d96c8b1154da parent: 102221:a9b4da316283 parent: 102222:e6525c6ad789 user: Yury Selivanov <yury(a)magic.io> date: Tue Jun 28 11:00:39 2016 -0400 summary: Merge 3.5 (asyncio) files: Lib/asyncio/selector_events.py | 3 +- Lib/test/test_asyncio/test_selector_events.py | 11 ++++++++++ 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/selector_events.… [View More]

1 0