Mailman 3 August 2014 - Python-checkins

peps: add missing words
by benjamin.peterson Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/peps/rev/f7f926c75521 changeset: 5536:f7f926c75521 user: Benjamin Peterson <benjamin(a)python.org> date: Sun Aug 31 12:12:42 2014 -0400 summary: add missing words files: pep-0477.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0477.txt b/pep-0477.txt --- a/pep-0477.txt +++ b/pep-0477.txt @@ -38,9 +38,9 @@ reasons: 1. The Python 2.x to 3.x migration has a number of painpoints that are eased by - a number of third party modules such as six [#six]_, [#modernize]_, or - [#future]_. However relying on these tools requires that everyone who uses - the project have a tool to install these packages. + a number of third party modules such as six [#six]_, modernize [#modernize]_, + or future [#future]_. However relying on these tools requires that everyone + who uses the project have a tool to install these packages. 2. In addition to tooling to aid in migration from Python 2.x to 3.x, there are also a number of modules that are *new* in Python 3 for which there are backports available on PyPI. This can also aid in the ability for people -- Repository URL: http://hg.python.org/peps

1 0

peps: actually need two underscores...
by benjamin.peterson Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/peps/rev/c7c4104a8925 changeset: 5535:c7c4104a8925 user: Benjamin Peterson <benjamin(a)python.org> date: Sun Aug 31 11:25:32 2014 -0400 summary: actually need two underscores... files: pep-0477.txt | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/pep-0477.txt b/pep-0477.txt --- a/pep-0477.txt +++ b/pep-0477.txt @@ -113,12 +113,12 @@ References ========== -.. [#six] `six.py <https://pypi.python.org/pypi/six>`_ -.. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>`_ -.. [#future] `python-future <https://pypi.python.org/pypi/future>`_ -.. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>`_ -.. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>`_ -.. [#twine] `twine <https://pypi.python.org/pypi/twine>`_ +.. [#six] `six.py <https://pypi.python.org/pypi/six>`__ +.. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>`__ +.. [#future] `python-future <https://pypi.python.org/pypi/future>`__ +.. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>`__ +.. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>`__ +.. [#twine] `twine <https://pypi.python.org/pypi/twine>`__ Copyright -- Repository URL: http://hg.python.org/peps

1 0

peps: fix links
by benjamin.peterson Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/peps/rev/d3b7604e5ebf changeset: 5534:d3b7604e5ebf user: Benjamin Peterson <benjamin(a)python.org> date: Sun Aug 31 10:51:10 2014 -0400 summary: fix links files: pep-0477.txt | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/pep-0477.txt b/pep-0477.txt --- a/pep-0477.txt +++ b/pep-0477.txt @@ -113,12 +113,12 @@ References ========== -.. [#six] `six.py <https://pypi.python.org/pypi/six>` -.. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>` -.. [#future] `python-future <https://pypi.python.org/pypi/future>` -.. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>` -.. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>` -.. [#twine] `twine <https://pypi.python.org/pypi/twine>` +.. [#six] `six.py <https://pypi.python.org/pypi/six>`_ +.. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>`_ +.. [#future] `python-future <https://pypi.python.org/pypi/future>`_ +.. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>`_ +.. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>`_ +.. [#twine] `twine <https://pypi.python.org/pypi/twine>`_ Copyright -- Repository URL: http://hg.python.org/peps

1 0

cpython (2.7): don't index outside of the path (closes #22312)
by benjamin.peterson Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/cpython/rev/446d4dfcc220 changeset: 92281:446d4dfcc220 branch: 2.7 parent: 92274:59f2edeb8443 user: Benjamin Peterson <benjamin(a)python.org> date: Sun Aug 31 09:34:09 2014 -0400 summary: don't index outside of the path (closes #22312) files: Lib/ntpath.py | 2 +- Lib/test/test_ntpath.py | 1 + Misc/NEWS | 2 ++ 3 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/ntpath.py b/Lib/ntpath.py --- a/Lib/ntpath.py +++ b/Lib/ntpath.py @@ -113,7 +113,7 @@ """ if len(p) > 1: normp = p.replace(altsep, sep) - if (normp[0:2] == sep*2) and (normp[2] != sep): + if (normp[0:2] == sep*2) and (normp[2:3] != sep): # is a UNC path: # vvvvvvvvvvvvvvvvvvvv drive letter or UNC path # \\machine\mountpoint\directory\etc\... diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py --- a/Lib/test/test_ntpath.py +++ b/Lib/test/test_ntpath.py @@ -50,6 +50,7 @@ # Issue #19911: UNC part containing U+0130 self.assertEqual(ntpath.splitdrive(u'//conky/MOUNTPOİNT/foo/bar'), (u'//conky/MOUNTPOİNT', '/foo/bar')) + self.assertEqual(ntpath.splitdrive("//"), ("", "//")) def test_splitunc(self): tester('ntpath.splitunc("c:\\foo\\bar")', diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -19,6 +19,8 @@ Library ------- +- Issue #22312: Fix ntpath.splitdrive IndexError. + - Issue #22216: smtplib now resets its state more completely after a quit. The most obvious consequence of the previous behavior was a STARTTLS failure during a connect/starttls/quit/connect/starttls sequence. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.4 -> default): (Merge 3.4) asyncio, Tulip issue 205: Fix a race condition in
by victor.stinner Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/cpython/rev/28cbbe2ce104 changeset: 92280:28cbbe2ce104 parent: 92278:9138d60db0e4 parent: 92279:ad67f66a5f3c user: Victor Stinner <victor.stinner(a)gmail.com> date: Sun Aug 31 15:08:21 2014 +0200 summary: (Merge 3.4) asyncio, Tulip issue 205: Fix a race condition in BaseSelectorEventLoop.sock_connect() There is a race condition in create_connection() used with wait_for() to have a timeout. sock_connect() registers the file descriptor of the socket to be notified of write event (if connect() raises BlockingIOError). When create_connection() is cancelled with a TimeoutError, sock_connect() coroutine gets the exception, but it doesn't unregister the file descriptor for write event. create_connection() gets the TimeoutError and closes the socket. If you call again create_connection(), the new socket will likely gets the same file descriptor, which is still registered in the selector. When sock_connect() calls add_writer(), it tries to modify the entry instead of creating a new one. This issue was originally reported in the Trollius project, but the bug comes from Tulip in fact (Trollius is based on Tulip): https://bitbucket.org/enovance/trollius/issue/15/after-timeouterror-on-wait… This change fixes the race condition. It also makes sock_connect() more reliable (and portable) is sock.connect() raises an InterruptedError. files: Lib/asyncio/selector_events.py | 44 ++++- Lib/test/test_asyncio/test_selector_events.py | 74 +++++++-- 2 files changed, 83 insertions(+), 35 deletions(-) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -8,6 +8,7 @@ import collections import errno +import functools import socket try: import ssl @@ -345,26 +346,43 @@ except ValueError as err: fut.set_exception(err) else: - self._sock_connect(fut, False, sock, address) + self._sock_connect(fut, sock, address) return fut - def _sock_connect(self, fut, registered, sock, address): + def _sock_connect(self, fut, sock, address): fd = sock.fileno() - if registered: - self.remove_writer(fd) + try: + while True: + try: + sock.connect(address) + except InterruptedError: + continue + else: + break + except BlockingIOError: + fut.add_done_callback(functools.partial(self._sock_connect_done, + sock)) + self.add_writer(fd, self._sock_connect_cb, fut, sock, address) + except Exception as exc: + fut.set_exception(exc) + else: + fut.set_result(None) + + def _sock_connect_done(self, sock, fut): + self.remove_writer(sock.fileno()) + + def _sock_connect_cb(self, fut, sock, address): if fut.cancelled(): return + try: - if not registered: - # First time around. - sock.connect(address) - else: - err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) - if err != 0: - # Jump to the except clause below. - raise OSError(err, 'Connect call failed %s' % (address,)) + err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) + if err != 0: + # Jump to any except clause below. + raise OSError(err, 'Connect call failed %s' % (address,)) except (BlockingIOError, InterruptedError): - self.add_writer(fd, self._sock_connect, fut, True, sock, address) + # socket is still registered, the callback will be retried later + pass except Exception as exc: fut.set_exception(exc) else: diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -40,8 +40,9 @@ class BaseSelectorEventLoopTests(test_utils.TestCase): def setUp(self): - selector = mock.Mock() - self.loop = TestBaseSelectorEventLoop(selector) + self.selector = mock.Mock() + self.selector.select.return_value = [] + self.loop = TestBaseSelectorEventLoop(self.selector) self.set_event_loop(self.loop, cleanup=False) def test_make_socket_transport(self): @@ -303,63 +304,92 @@ f = self.loop.sock_connect(sock, ('127.0.0.1', 8080)) self.assertIsInstance(f, asyncio.Future) self.assertEqual( - (f, False, sock, ('127.0.0.1', 8080)), + (f, sock, ('127.0.0.1', 8080)), self.loop._sock_connect.call_args[0]) + def test_sock_connect_timeout(self): + # Tulip issue #205: sock_connect() must unregister the socket on + # timeout error + + # prepare mocks + self.loop.add_writer = mock.Mock() + self.loop.remove_writer = mock.Mock() + sock = test_utils.mock_nonblocking_socket() + sock.connect.side_effect = BlockingIOError + + # first call to sock_connect() registers the socket + fut = self.loop.sock_connect(sock, ('127.0.0.1', 80)) + self.assertTrue(sock.connect.called) + self.assertTrue(self.loop.add_writer.called) + self.assertEqual(len(fut._callbacks), 1) + + # on timeout, the socket must be unregistered + sock.connect.reset_mock() + fut.set_exception(asyncio.TimeoutError) + with self.assertRaises(asyncio.TimeoutError): + self.loop.run_until_complete(fut) + self.assertTrue(self.loop.remove_writer.called) + def test__sock_connect(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 - self.loop._sock_connect(f, False, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect(f, sock, ('127.0.0.1', 8080)) self.assertTrue(f.done()) self.assertIsNone(f.result()) self.assertTrue(sock.connect.called) - def test__sock_connect_canceled_fut(self): + def test__sock_connect_cb_cancelled_fut(self): sock = mock.Mock() + self.loop.remove_writer = mock.Mock() f = asyncio.Future(loop=self.loop) f.cancel() - self.loop._sock_connect(f, False, sock, ('127.0.0.1', 8080)) - self.assertFalse(sock.connect.called) + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) + self.assertFalse(sock.getsockopt.called) - def test__sock_connect_unregister(self): + def test__sock_connect_writer(self): + # check that the fd is registered and then unregistered + self.loop._process_events = mock.Mock() + self.loop.add_writer = mock.Mock() + self.loop.remove_writer = mock.Mock() + sock = mock.Mock() sock.fileno.return_value = 10 + sock.connect.side_effect = BlockingIOError + sock.getsockopt.return_value = 0 + address = ('127.0.0.1', 8080) f = asyncio.Future(loop=self.loop) - f.cancel() + self.loop._sock_connect(f, sock, address) + self.assertTrue(self.loop.add_writer.called) + self.assertEqual(10, self.loop.add_writer.call_args[0][0]) - self.loop.remove_writer = mock.Mock() - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect_cb(f, sock, address) + # need to run the event loop to execute _sock_connect_done() callback + self.loop.run_until_complete(f) self.assertEqual((10,), self.loop.remove_writer.call_args[0]) - def test__sock_connect_tryagain(self): + def test__sock_connect_cb_tryagain(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 sock.getsockopt.return_value = errno.EAGAIN - self.loop.add_writer = mock.Mock() - self.loop.remove_writer = mock.Mock() + # check that the exception is handled + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) - self.assertEqual( - (10, self.loop._sock_connect, f, - True, sock, ('127.0.0.1', 8080)), - self.loop.add_writer.call_args[0]) - - def test__sock_connect_exception(self): + def test__sock_connect_cb_exception(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 sock.getsockopt.return_value = errno.ENOTCONN self.loop.remove_writer = mock.Mock() - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) self.assertIsInstance(f.exception(), OSError) def test_sock_accept(self): -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.4): asyncio, Tulip issue 205: Fix a race condition in
by victor.stinner Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/cpython/rev/ad67f66a5f3c changeset: 92279:ad67f66a5f3c branch: 3.4 parent: 92277:9480287f85a0 user: Victor Stinner <victor.stinner(a)gmail.com> date: Sun Aug 31 15:07:57 2014 +0200 summary: asyncio, Tulip issue 205: Fix a race condition in BaseSelectorEventLoop.sock_connect() There is a race condition in create_connection() used with wait_for() to have a timeout. sock_connect() registers the file descriptor of the socket to be notified of write event (if connect() raises BlockingIOError). When create_connection() is cancelled with a TimeoutError, sock_connect() coroutine gets the exception, but it doesn't unregister the file descriptor for write event. create_connection() gets the TimeoutError and closes the socket. If you call again create_connection(), the new socket will likely gets the same file descriptor, which is still registered in the selector. When sock_connect() calls add_writer(), it tries to modify the entry instead of creating a new one. This issue was originally reported in the Trollius project, but the bug comes from Tulip in fact (Trollius is based on Tulip): https://bitbucket.org/enovance/trollius/issue/15/after-timeouterror-on-wait… This change fixes the race condition. It also makes sock_connect() more reliable (and portable) is sock.connect() raises an InterruptedError. files: Lib/asyncio/selector_events.py | 44 ++++- Lib/test/test_asyncio/test_selector_events.py | 74 +++++++-- 2 files changed, 83 insertions(+), 35 deletions(-) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -8,6 +8,7 @@ import collections import errno +import functools import socket try: import ssl @@ -345,26 +346,43 @@ except ValueError as err: fut.set_exception(err) else: - self._sock_connect(fut, False, sock, address) + self._sock_connect(fut, sock, address) return fut - def _sock_connect(self, fut, registered, sock, address): + def _sock_connect(self, fut, sock, address): fd = sock.fileno() - if registered: - self.remove_writer(fd) + try: + while True: + try: + sock.connect(address) + except InterruptedError: + continue + else: + break + except BlockingIOError: + fut.add_done_callback(functools.partial(self._sock_connect_done, + sock)) + self.add_writer(fd, self._sock_connect_cb, fut, sock, address) + except Exception as exc: + fut.set_exception(exc) + else: + fut.set_result(None) + + def _sock_connect_done(self, sock, fut): + self.remove_writer(sock.fileno()) + + def _sock_connect_cb(self, fut, sock, address): if fut.cancelled(): return + try: - if not registered: - # First time around. - sock.connect(address) - else: - err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) - if err != 0: - # Jump to the except clause below. - raise OSError(err, 'Connect call failed %s' % (address,)) + err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) + if err != 0: + # Jump to any except clause below. + raise OSError(err, 'Connect call failed %s' % (address,)) except (BlockingIOError, InterruptedError): - self.add_writer(fd, self._sock_connect, fut, True, sock, address) + # socket is still registered, the callback will be retried later + pass except Exception as exc: fut.set_exception(exc) else: diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -40,8 +40,9 @@ class BaseSelectorEventLoopTests(test_utils.TestCase): def setUp(self): - selector = mock.Mock() - self.loop = TestBaseSelectorEventLoop(selector) + self.selector = mock.Mock() + self.selector.select.return_value = [] + self.loop = TestBaseSelectorEventLoop(self.selector) self.set_event_loop(self.loop, cleanup=False) def test_make_socket_transport(self): @@ -303,63 +304,92 @@ f = self.loop.sock_connect(sock, ('127.0.0.1', 8080)) self.assertIsInstance(f, asyncio.Future) self.assertEqual( - (f, False, sock, ('127.0.0.1', 8080)), + (f, sock, ('127.0.0.1', 8080)), self.loop._sock_connect.call_args[0]) + def test_sock_connect_timeout(self): + # Tulip issue #205: sock_connect() must unregister the socket on + # timeout error + + # prepare mocks + self.loop.add_writer = mock.Mock() + self.loop.remove_writer = mock.Mock() + sock = test_utils.mock_nonblocking_socket() + sock.connect.side_effect = BlockingIOError + + # first call to sock_connect() registers the socket + fut = self.loop.sock_connect(sock, ('127.0.0.1', 80)) + self.assertTrue(sock.connect.called) + self.assertTrue(self.loop.add_writer.called) + self.assertEqual(len(fut._callbacks), 1) + + # on timeout, the socket must be unregistered + sock.connect.reset_mock() + fut.set_exception(asyncio.TimeoutError) + with self.assertRaises(asyncio.TimeoutError): + self.loop.run_until_complete(fut) + self.assertTrue(self.loop.remove_writer.called) + def test__sock_connect(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 - self.loop._sock_connect(f, False, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect(f, sock, ('127.0.0.1', 8080)) self.assertTrue(f.done()) self.assertIsNone(f.result()) self.assertTrue(sock.connect.called) - def test__sock_connect_canceled_fut(self): + def test__sock_connect_cb_cancelled_fut(self): sock = mock.Mock() + self.loop.remove_writer = mock.Mock() f = asyncio.Future(loop=self.loop) f.cancel() - self.loop._sock_connect(f, False, sock, ('127.0.0.1', 8080)) - self.assertFalse(sock.connect.called) + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) + self.assertFalse(sock.getsockopt.called) - def test__sock_connect_unregister(self): + def test__sock_connect_writer(self): + # check that the fd is registered and then unregistered + self.loop._process_events = mock.Mock() + self.loop.add_writer = mock.Mock() + self.loop.remove_writer = mock.Mock() + sock = mock.Mock() sock.fileno.return_value = 10 + sock.connect.side_effect = BlockingIOError + sock.getsockopt.return_value = 0 + address = ('127.0.0.1', 8080) f = asyncio.Future(loop=self.loop) - f.cancel() + self.loop._sock_connect(f, sock, address) + self.assertTrue(self.loop.add_writer.called) + self.assertEqual(10, self.loop.add_writer.call_args[0][0]) - self.loop.remove_writer = mock.Mock() - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect_cb(f, sock, address) + # need to run the event loop to execute _sock_connect_done() callback + self.loop.run_until_complete(f) self.assertEqual((10,), self.loop.remove_writer.call_args[0]) - def test__sock_connect_tryagain(self): + def test__sock_connect_cb_tryagain(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 sock.getsockopt.return_value = errno.EAGAIN - self.loop.add_writer = mock.Mock() - self.loop.remove_writer = mock.Mock() + # check that the exception is handled + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) - self.assertEqual( - (10, self.loop._sock_connect, f, - True, sock, ('127.0.0.1', 8080)), - self.loop.add_writer.call_args[0]) - - def test__sock_connect_exception(self): + def test__sock_connect_cb_exception(self): f = asyncio.Future(loop=self.loop) sock = mock.Mock() sock.fileno.return_value = 10 sock.getsockopt.return_value = errno.ENOTCONN self.loop.remove_writer = mock.Mock() - self.loop._sock_connect(f, True, sock, ('127.0.0.1', 8080)) + self.loop._sock_connect_cb(f, sock, ('127.0.0.1', 8080)) self.assertIsInstance(f.exception(), OSError) def test_sock_accept(self): -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.4 -> default): (Merge 3.4) Closes #22275: asyncio: enhance documentation of OS support
by victor.stinner Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/cpython/rev/9138d60db0e4 changeset: 92278:9138d60db0e4 parent: 92276:164a17eca081 parent: 92277:9480287f85a0 user: Victor Stinner <victor.stinner(a)gmail.com> date: Sun Aug 31 14:47:56 2014 +0200 summary: (Merge 3.4) Closes #22275: asyncio: enhance documentation of OS support files: Doc/library/asyncio-eventloop.rst | 26 ++++++++++++++--- Doc/library/asyncio-eventloops.rst | 8 +++-- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -221,6 +221,8 @@ to bind the socket to locally. The *local_host* and *local_port* are looked up using getaddrinfo(), similarly to *host* and *port*. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. seealso:: The :func:`open_connection` function can be used to get a pair of @@ -239,6 +241,8 @@ See the :meth:`BaseEventLoop.create_connection` method for parameters. + On Windows with :class:`ProactorEventLoop`, this method is not supported. + .. method:: BaseEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) @@ -251,6 +255,8 @@ establish the connection in the background. When successful, the coroutine returns a ``(transport, protocol)`` pair. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + See the :meth:`BaseEventLoop.create_connection` method for parameters. Availability: UNIX. @@ -261,19 +267,19 @@ .. method:: BaseEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None) - Create a TCP server bound to host and port. Return a :class:`Server` object, + Create a TCP server bound to *host* and *port*. Return a :class:`Server` object, its :attr:`~Server.sockets` attribute contains created sockets. Use the :meth:`Server.close` method to stop the server: close listening sockets. This method is a :ref:`coroutine <coroutine>`. - If *host* is an empty string or None all interfaces are assumed + If *host* is an empty string or ``None``, all interfaces are assumed and a list of multiple sockets will be returned (most likely one for IPv4 and another one for IPv6). - *family* can be set to either :data:`~socket.AF_INET` or + *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set - it will be determined from host (defaults to :data:`~socket.AF_UNSPEC`). + it will be determined from host (defaults to :data:`socket.AF_UNSPEC`). *flags* is a bitmask for :meth:`getaddrinfo`. @@ -283,7 +289,7 @@ *backlog* is the maximum number of queued connections passed to :meth:`~socket.socket.listen` (defaults to 100). - ssl can be set to an :class:`~ssl.SSLContext` to enable SSL over the + *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the accepted connections. *reuse_address* tells the kernel to reuse a local socket in @@ -291,6 +297,8 @@ expire. If not specified will automatically be set to True on UNIX. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. seealso:: The function :func:`start_server` creates a (:class:`StreamReader`, @@ -308,6 +316,11 @@ Watch file descriptors ---------------------- +On Windows with :class:`SelectorEventLoop`, only socket handles are supported +(ex: pipe file descriptors are not supported). + +On Windows with :class:`ProactorEventLoop`, these methods are not supported. + .. method:: BaseEventLoop.add_reader(fd, callback, \*args) Start watching the file descriptor for read availability and then call the @@ -419,6 +432,9 @@ Connect pipes ------------- +On Windows with :class:`SelectorEventLoop`, these methods are not supported. +Use :class:`ProactorEventLoop` to support pipes on Windows. + .. method:: BaseEventLoop.connect_read_pipe(protocol_factory, pipe) Register read pipe in eventloop. diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst --- a/Doc/library/asyncio-eventloops.rst +++ b/Doc/library/asyncio-eventloops.rst @@ -39,6 +39,10 @@ Use the most efficient selector available on the platform. + On Windows, only sockets are supported (ex: pipes are not supported): + see the `MSDN documentation of select + <http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%…>`_. + .. class:: ProactorEventLoop Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. @@ -83,9 +87,7 @@ :class:`SelectorEventLoop` specific limits: -- :class:`~selectors.SelectSelector` is used but it only supports sockets, - see the `MSDN documentation of select - <http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%…>`_ +- :class:`~selectors.SelectSelector` is used but it only supports sockets - :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` only accept file descriptors of sockets - Pipes are not supported -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.4): Closes #22275: asyncio: enhance documentation of OS support
by victor.stinner Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/cpython/rev/9480287f85a0 changeset: 92277:9480287f85a0 branch: 3.4 parent: 92275:c30163548f64 user: Victor Stinner <victor.stinner(a)gmail.com> date: Sun Aug 31 14:47:37 2014 +0200 summary: Closes #22275: asyncio: enhance documentation of OS support files: Doc/library/asyncio-eventloop.rst | 26 ++++++++++++++--- Doc/library/asyncio-eventloops.rst | 8 +++-- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -221,6 +221,8 @@ to bind the socket to locally. The *local_host* and *local_port* are looked up using getaddrinfo(), similarly to *host* and *port*. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. seealso:: The :func:`open_connection` function can be used to get a pair of @@ -239,6 +241,8 @@ See the :meth:`BaseEventLoop.create_connection` method for parameters. + On Windows with :class:`ProactorEventLoop`, this method is not supported. + .. method:: BaseEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) @@ -251,6 +255,8 @@ establish the connection in the background. When successful, the coroutine returns a ``(transport, protocol)`` pair. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + See the :meth:`BaseEventLoop.create_connection` method for parameters. Availability: UNIX. @@ -261,19 +267,19 @@ .. method:: BaseEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None) - Create a TCP server bound to host and port. Return a :class:`Server` object, + Create a TCP server bound to *host* and *port*. Return a :class:`Server` object, its :attr:`~Server.sockets` attribute contains created sockets. Use the :meth:`Server.close` method to stop the server: close listening sockets. This method is a :ref:`coroutine <coroutine>`. - If *host* is an empty string or None all interfaces are assumed + If *host* is an empty string or ``None``, all interfaces are assumed and a list of multiple sockets will be returned (most likely one for IPv4 and another one for IPv6). - *family* can be set to either :data:`~socket.AF_INET` or + *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set - it will be determined from host (defaults to :data:`~socket.AF_UNSPEC`). + it will be determined from host (defaults to :data:`socket.AF_UNSPEC`). *flags* is a bitmask for :meth:`getaddrinfo`. @@ -283,7 +289,7 @@ *backlog* is the maximum number of queued connections passed to :meth:`~socket.socket.listen` (defaults to 100). - ssl can be set to an :class:`~ssl.SSLContext` to enable SSL over the + *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the accepted connections. *reuse_address* tells the kernel to reuse a local socket in @@ -291,6 +297,8 @@ expire. If not specified will automatically be set to True on UNIX. + On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. seealso:: The function :func:`start_server` creates a (:class:`StreamReader`, @@ -308,6 +316,11 @@ Watch file descriptors ---------------------- +On Windows with :class:`SelectorEventLoop`, only socket handles are supported +(ex: pipe file descriptors are not supported). + +On Windows with :class:`ProactorEventLoop`, these methods are not supported. + .. method:: BaseEventLoop.add_reader(fd, callback, \*args) Start watching the file descriptor for read availability and then call the @@ -419,6 +432,9 @@ Connect pipes ------------- +On Windows with :class:`SelectorEventLoop`, these methods are not supported. +Use :class:`ProactorEventLoop` to support pipes on Windows. + .. method:: BaseEventLoop.connect_read_pipe(protocol_factory, pipe) Register read pipe in eventloop. diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst --- a/Doc/library/asyncio-eventloops.rst +++ b/Doc/library/asyncio-eventloops.rst @@ -39,6 +39,10 @@ Use the most efficient selector available on the platform. + On Windows, only sockets are supported (ex: pipes are not supported): + see the `MSDN documentation of select + <http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%…>`_. + .. class:: ProactorEventLoop Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. @@ -83,9 +87,7 @@ :class:`SelectorEventLoop` specific limits: -- :class:`~selectors.SelectSelector` is used but it only supports sockets, - see the `MSDN documentation of select - <http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%…>`_ +- :class:`~selectors.SelectSelector` is used but it only supports sockets - :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` only accept file descriptors of sockets - Pipes are not supported -- Repository URL: http://hg.python.org/cpython

1 0

peps: PEP 477: selected 2.7 backports for ensurepip
by nick.coghlan Aug. 31, 2014

Aug. 31, 2014

http://hg.python.org/peps/rev/d57a77ac9f8a changeset: 5533:d57a77ac9f8a user: Nick Coghlan <ncoghlan(a)gmail.com> date: Sun Aug 31 21:58:00 2014 +1000 summary: PEP 477: selected 2.7 backports for ensurepip files: pep-0477.txt | 138 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 138 insertions(+), 0 deletions(-) diff --git a/pep-0477.txt b/pep-0477.txt new file mode 100644 --- /dev/null +++ b/pep-0477.txt @@ -0,0 +1,138 @@ +PEP: 477 +Title: Backport ensurepip (PEP 453) to Python 2.7 +Version: $Revision$ +Last-Modified: $Date$ +Author: Donald Stufft <donald(a)stufft.io> + Nick Coghlan <ncoghlan(a)gmail.com> +Status: Active +Type: Process +Content-Type: text/x-rst +Created: 26-Aug-2014 +Post-History: 1-Sep-2014 + + +Abstract +======== + +This PEP proposes that the ``ensurepip`` module, added to Python 3.4 by PEP +453, be backported to Python 2.7. It also proposes that automatic invocation +of ``ensurepip`` be added to the Python 2.7 Windows and OSX installers. However +it does **not** propose that automatic invocation be added to the ``Makefile``. + +It also proposes that the documentation changes for the package distribution +and installation guides be updated to match that in 3.4, which references using +the ``ensurepip`` module to bootstrap the installer. + + +Rationale +========= + +Python 2.7 is effectively a LTS release of Python which represents the end of +the 2.x series and there is still a very large contingent of users whom are +still using Python 2.7 as their primary version. These users, in order to +participate in the wider Python ecosystem, must manually attempt to go out and +find the correct way to bootstrap the packaging tools. + +It is the opinion of this PEP that making it as easy as possible for end users +to participate in the wider Python ecosystem is important for 3 primary +reasons: + +1. The Python 2.x to 3.x migration has a number of painpoints that are eased by + a number of third party modules such as six [#six]_, [#modernize]_, or + [#future]_. However relying on these tools requires that everyone who uses + the project have a tool to install these packages. +2. In addition to tooling to aid in migration from Python 2.x to 3.x, there are + also a number of modules that are *new* in Python 3 for which there are + backports available on PyPI. This can also aid in the ability for people + to write 2.x and 3.x compatible software as well as enable them to use some + of the newer features of Python 3 on Python 2. +3. Users also will need a number of tools in order to create python packages + that conform to the newer standards that are being proposed. Things like + setuptools [#setuptools]_, Wheel [#wheel]_, and twine [#twine]_ are enabling + a safer, faster, and more reliable packaging tool chain. These tools can be + difficult for people to use if first they must be told how to go out and + install the package manager. +4. One of Pythons biggest strengths is in the huge ecosystem of libraries and + projects that have been built on top of it, most of which are distributed + through PyPI. However in order to benefit from this wide ecosystem + meaningfully requires end users, some of which are going to be new, to make + a decision on which package manager they should get, how to get it, and then + finally actually installing it first. + +Furthermore, alternative implementations of Python are recognizing the benefits +of PEP 453 and both PyPy and Jython have plans to backport ensurepip to their +2.7 runtimes. + + +Automatic Invocation +==================== + +PEP 453 has ``ensurepip`` automatically invoked by default in the ``Makefile`` +and the Windows and OSX Installers. This allowed it to ensure that, by default, +all users would get Python with pip already installed. This PEP however +believes that while this is fine for the Python 2.7 Windows and Mac OS X +installers it is *not* ok for the Python 2.7 ``Makefile`` in general. + +The primary consumers of the ``Makefile`` are downstream package managers which +distribute Python themselves. These downstream distributors typically do not +want pip to be installed via ``ensurepip`` and would prefer that end users +install it with their own package manager. Not invoking ``ensurepip`` +automatically from the ``Makefile`` would allow these distributors to simply +ignore the fact that ``ensurepip`` has been backported and still not end up +with pip installed via it. + +The primary consumers of the OSX and Windows installers are end users who are +attempting to install Python on their own machine. There is not a package +manager available where these users can install pip into their Python through +a more supported mechanism. For this reason it is the belief of this PEP that +installing by default on OSX and Windows is the best course of action. + + +Documentation +============= + +As part of this PEP, the updated packaging distribution and installation +guides for Python 3.4 would be backported to Python 2.7. + + +Disabling ensurepip by Downstream Distributors +============================================== + +Due to its use in the ``venv`` module, downstream distributors cannot disable +the ``ensurepip`` module in Python 3.4. However since Python 2.7 has no such +module it is explicitly allowed for downstream distributors to patch the +``ensurepip`` module to prevent it from installing anything. + +If a downstream distributor wishes to disable ``ensurepip`` completely in +Python 2.7, they should still at least provide the module and allow +`python -m ensurepip` style invocation. However it should raise errors or +otherwise exit with a non-zero exit code and print out an error on stderr +directing users to what they can/should use instead of ``ensurepip``. + + +References +========== + +.. [#six] `six.py <https://pypi.python.org/pypi/six>` +.. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>` +.. [#future] `python-future <https://pypi.python.org/pypi/future>` +.. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>` +.. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>` +.. [#twine] `twine <https://pypi.python.org/pypi/twine>` + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: http://hg.python.org/peps

1 0

Daily reference leaks (164a17eca081): sum=149135
by solipsis＠pitrou.net Aug. 31, 2014

Aug. 31, 2014

results for 164a17eca081 on branch "default" -------------------------------------------- test_codecs leaked [5825, 5825, 5825] references, sum=17475 test_codecs leaked [1172, 1174, 1174] memory blocks, sum=3520 test_distutils leaked [37000, 37000, 37000] references, sum=111000 test_distutils leaked [5711, 5713, 5713] memory blocks, sum=17137 test_functools leaked [0, 0, 3] memory blocks, sum=3 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogcF8ulw', '-x']

1 0