From python-checkins at python.org Sat Mar 1 08:04:20 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 08:04:20 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4xKTogSXNzdWUgIzIwMjQ2?= =?utf-8?q?=3A_Fix_test_failures_on_FreeBSD=2E_Patch_by_Ryan_Smith-Roberts?= =?utf-8?q?=2E?= Message-ID: <3fbds03nWMz7LlJ@mail.python.org> http://hg.python.org/cpython/rev/c25e1442529f changeset: 89430:c25e1442529f branch: 3.1 parent: 89294:fbb4d48046e5 user: Stefan Krah date: Tue Jan 21 22:58:40 2014 +0100 summary: Issue #20246: Fix test failures on FreeBSD. Patch by Ryan Smith-Roberts. files: Lib/test/test_socket.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1430,7 +1430,7 @@ self.assertRaises(ValueError, self.cli_conn.recvfrom_into, buf, 1024) def _testRecvFromIntoSmallBuffer(self): - self.serv_conn.send(MSG*2048) + self.serv_conn.send(MSG) TIPC_STYPE = 2000 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 08:04:21 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 08:04:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzIwMjQ2?= =?utf-8?q?=3A_Fix_test_failures_on_FreeBSD=2E_Patch_by_Ryan_Smith-Roberts?= =?utf-8?q?=2E?= Message-ID: <3fbds15GN6z7LlJ@mail.python.org> http://hg.python.org/cpython/rev/e82dcd700e8c changeset: 89431:e82dcd700e8c branch: 3.2 parent: 89296:29b1eebecb8e user: Stefan Krah date: Tue Jan 21 22:58:40 2014 +0100 summary: Issue #20246: Fix test failures on FreeBSD. Patch by Ryan Smith-Roberts. files: Lib/test/test_socket.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1974,7 +1974,7 @@ self.assertRaises(ValueError, self.cli_conn.recvfrom_into, buf, 1024) def _testRecvFromIntoSmallBuffer(self): - self.serv_conn.send(MSG*2048) + self.serv_conn.send(MSG) TIPC_STYPE = 2000 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 08:04:22 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 08:04:22 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2?= Message-ID: <3fbds26mwjz7LtG@mail.python.org> http://hg.python.org/cpython/rev/afb76ca87022 changeset: 89432:afb76ca87022 branch: 3.3 parent: 89428:432cb56db05d parent: 89431:e82dcd700e8c user: Benjamin Peterson date: Sat Mar 01 02:03:57 2014 -0500 summary: merge 3.2 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 08:04:24 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 08:04:24 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fbds41lc1z7Ltb@mail.python.org> http://hg.python.org/cpython/rev/345391bb8ee1 changeset: 89433:345391bb8ee1 parent: 89429:b6e999c8907c parent: 89432:afb76ca87022 user: Benjamin Peterson date: Sat Mar 01 02:04:10 2014 -0500 summary: merge 3.3 files: -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Mar 1 09:11:35 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 01 Mar 2014 09:11:35 +0100 Subject: [Python-checkins] Daily reference leaks (b6e999c8907c): sum=-4 Message-ID: results for b6e999c8907c on branch "default" -------------------------------------------- test_site leaked [-2, 2, -2] references, sum=-2 test_site leaked [-2, 2, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogN3_Je4', '-x'] From python-checkins at python.org Sat Mar 1 13:53:52 2014 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 1 Mar 2014 13:53:52 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2316135=3A_remove_mention?= =?utf-8?q?s_of_OS/2_from_the_documentation?= Message-ID: <3fbncJ53Rkz7LmX@mail.python.org> http://hg.python.org/cpython/rev/5e05d7d3db9c changeset: 89434:5e05d7d3db9c user: Andrew Kuchling date: Sat Mar 01 07:53:28 2014 -0500 summary: #16135: remove mentions of OS/2 from the documentation files: Doc/distutils/apiref.rst | 11 ----------- Doc/library/curses.rst | 2 +- Doc/library/os.rst | 3 +-- Doc/library/socket.rst | 3 +-- Doc/library/undoc.rst | 2 +- 5 files changed, 4 insertions(+), 17 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -853,17 +853,6 @@ port of GCC (same as cygwin in no-cygwin mode). -:mod:`distutils.emxccompiler` --- OS/2 EMX Compiler -=================================================== - -.. module:: distutils.emxccompiler - :synopsis: OS/2 EMX Compiler support - - -This module provides the EMXCCompiler class, a subclass of -:class:`UnixCCompiler` that handles the EMX port of the GNU C compiler to OS/2. - - :mod:`distutils.archive_util` --- Archiving utilities ====================================================== diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -12,7 +12,7 @@ de-facto standard for portable advanced terminal handling. While curses is most widely used in the Unix environment, versions are available -for DOS, OS/2, and possibly other systems as well. This extension module is +for Windows, DOS, and possibly other systems as well. This extension module is designed to match the API of ncurses, an open-source curses library hosted on Linux and the BSD variants of Unix. diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2653,7 +2653,7 @@ Fork a child process. Return ``0`` in the child and the child's process id in the parent. If an error occurs :exc:`OSError` is raised. - Note that some platforms including FreeBSD <= 6.3, Cygwin and OS/2 EMX have + Note that some platforms including FreeBSD <= 6.3 and Cygwin have known issues when using fork() from a thread. .. warning:: @@ -2899,7 +2899,6 @@ :manpage:`times(2)` or the corresponding Windows Platform API documentation. On Windows, only :attr:`user` and :attr:`system` are known; the other attributes are zero. - On OS/2, only :attr:`elapsed` is known; the other attributes are zero. Availability: Unix, Windows. diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -6,8 +6,7 @@ This module provides access to the BSD *socket* interface. It is available on -all modern Unix systems, Windows, MacOS, OS/2, and probably additional -platforms. +all modern Unix systems, Windows, MacOS, and probably additional platforms. .. note:: diff --git a/Doc/library/undoc.rst b/Doc/library/undoc.rst --- a/Doc/library/undoc.rst +++ b/Doc/library/undoc.rst @@ -20,7 +20,7 @@ documented beyond this mention. There's little need to document these. :mod:`ntpath` - --- Implementation of :mod:`os.path` on Win32, Win64, WinCE, and OS/2 platforms. + --- Implementation of :mod:`os.path` on Win32, Win64, and WinCE platforms. :mod:`posixpath` --- Implementation of :mod:`os.path` on POSIX. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 16:34:34 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 16:34:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fix_test_on_de?= =?utf-8?q?bug_builds_=28closes_=2320731=29?= Message-ID: <3fbs9k6ZRgz7Lp8@mail.python.org> http://hg.python.org/cpython/rev/dcf4fbf446ca changeset: 89435:dcf4fbf446ca branch: 3.3 parent: 89432:afb76ca87022 user: Benjamin Peterson date: Sat Mar 01 10:31:36 2014 -0500 summary: fix test on debug builds (closes #20731) files: Lib/test/test_coding.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_coding.py b/Lib/test/test_coding.py --- a/Lib/test/test_coding.py +++ b/Lib/test/test_coding.py @@ -59,12 +59,13 @@ msg=c.exception.args[0]) def test_20731(self): - sub = subprocess.Popen([sys.executable, + sub = subprocess.Popen([sys.executable, os.path.join(os.path.dirname(__file__), 'coding20731.py')], stderr=subprocess.PIPE) err = sub.communicate()[1] - self.assertEquals(err, b'') + self.assertEqual(sub.returncode, 0) + self.assertNotIn(b'SyntaxError', err) if __name__ == "__main__": unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 16:34:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 1 Mar 2014 16:34:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fbs9m14ywz7LmX@mail.python.org> http://hg.python.org/cpython/rev/a4726b5d0365 changeset: 89436:a4726b5d0365 parent: 89434:5e05d7d3db9c parent: 89435:dcf4fbf446ca user: Benjamin Peterson date: Sat Mar 01 10:34:22 2014 -0500 summary: merge 3.3 files: Lib/test/test_source_encoding.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_source_encoding.py b/Lib/test/test_source_encoding.py --- a/Lib/test/test_source_encoding.py +++ b/Lib/test/test_source_encoding.py @@ -65,7 +65,8 @@ 'coding20731.py')], stderr=subprocess.PIPE) err = sub.communicate()[1] - self.assertEquals(err, b'') + self.assertEqual(sub.returncode, 0) + self.assertNotIn(b'SyntaxError', err) def test_error_message(self): compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 16:57:56 2014 From: python-checkins at python.org (brett.cannon) Date: Sat, 1 Mar 2014 16:57:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Bring_PEP_8_in_line_with_PEP_?= =?utf-8?q?257?= Message-ID: <3fbshh5YZTz7Lt3@mail.python.org> http://hg.python.org/peps/rev/5efe00002b3e changeset: 5390:5efe00002b3e user: Brett Cannon date: Sat Mar 01 10:57:51 2014 -0500 summary: Bring PEP 8 in line with PEP 257 files: pep-0008.txt | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -550,7 +550,6 @@ """Return a foobang Optional plotz says to frobnicate the bizbaz first. - """ - For one liner docstrings, it's okay to keep the closing ``"""`` on -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sat Mar 1 18:11:54 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:11:54 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_return_types_o?= =?utf-8?q?f_re_functions=2E?= Message-ID: <3fbvL21Sxqz7Lt3@mail.python.org> http://hg.python.org/cpython/rev/962f340f2487 changeset: 89437:962f340f2487 user: R David Murray date: Thu Feb 27 18:32:32 2014 -0500 summary: whatsnew: return types of re functions. files: Doc/whatsnew/3.4.rst | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1987,6 +1987,13 @@ in theory this should not cause backward compatibility issues since the disallowed command forms didn't make any sense and are unlikely to be in use. +* The :func:`re.split`, :func:`re.findall`, and :func:`re.sub` functions, and + the :meth:`~re.match.group` and :meth:`~re.match.groups` methods of + :class:``match`` objects now always return a *bytes* object when the string + to be matched is a :term:`bytes-like object`. Previously the return type + matched the input type, so if your code was depending on the return value + being, say, a ``bytearray``, you will need to change your code. + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 18:11:55 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:11:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_tarfile_stat_c?= =?utf-8?q?onstants_removal=2E?= Message-ID: <3fbvL344H3z7LtV@mail.python.org> http://hg.python.org/cpython/rev/8b50f1ba2a2c changeset: 89438:8b50f1ba2a2c user: R David Murray date: Thu Feb 27 18:55:34 2014 -0500 summary: whatsnew: tarfile stat constants removal. files: Doc/whatsnew/3.4.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1811,6 +1811,9 @@ ``_mac_ver_gstalt``, and ``_bcd2str``, which would only have ever been called on badly broken OSX systems (see :issue:`18393`). +* The hardcoded copies of certain :mod:`stat` constants that were included in + the :mod:`tarfile` module namespace have been removed. + Porting to Python 3.4 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 18:11:56 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:11:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_compression_li?= =?utf-8?q?braries_support_=27x=27_mode=2E?= Message-ID: <3fbvL45RbBz7Lp8@mail.python.org> http://hg.python.org/cpython/rev/6bf20f05a7e2 changeset: 89439:6bf20f05a7e2 user: R David Murray date: Thu Feb 27 20:05:42 2014 -0500 summary: whatsnew: compression libraries support 'x' mode. files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1609,6 +1609,10 @@ set just the error handler, without changing the default encoding. (Contributed by Serhiy Storchaka in :issue:`18818`.) +* The :mod:`bz2`, :mod:`lzma`, and :mod:`gzip` module ``open`` functions now + support ``x`` (exclusive creation) mode. (Contributed by Tim Heaney and + Vajrasky Kok in :issue:`19201`, :issue:`19222`, and :issue:`19223`.) + Significant Optimizations ------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 18:11:57 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:11:57 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_give_Serhiy_cr?= =?utf-8?q?edit_for_audioop_24-bit_sample_support=2E?= Message-ID: <3fbvL56zMtz7LmX@mail.python.org> http://hg.python.org/cpython/rev/ebd3037cde40 changeset: 89440:ebd3037cde40 user: R David Murray date: Fri Feb 28 13:30:27 2014 -0500 summary: whatsnew: give Serhiy credit for audioop 24-bit sample support. files: Doc/whatsnew/3.4.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -547,7 +547,8 @@ audioop ------- -Added support for 24-bit samples (:issue:`12866`). +The module now supports 24-bit samples. (Contributed by Serhiy Storchaka +in :issue:`12866`.) Added the :func:`~audioop.byteswap` function to convert big-endian samples to little-endian and vice versa (:issue:`19641`). -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 18:11:59 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:11:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_unittest_disco?= =?utf-8?q?ver_works_on_namespace_packages_=28=2317457=29=2E?= Message-ID: <3fbvL71J3Nz7Lt3@mail.python.org> http://hg.python.org/cpython/rev/57cb8a6e8f10 changeset: 89441:57cb8a6e8f10 user: R David Murray date: Sat Mar 01 11:57:58 2014 -0500 summary: whatsnew: unittest discover works on namespace packages (#17457). I also added the docs that never got committed, editing the patch provided by Claudiu. I collapsed the now three versionchanged directives for discovery into one. I tried several different styles for this. The most obvious is a bulleted list, but currently the only way I could find to produce that is to have *something* appear on the versionchanged line after the colon, which combined with the blank space before the bulleted list just looks wrong. I also tried the current single-line-three-sentences version with bullet characters before each sentence, and I almost went with that as it made it clear that the three sentences are independent. But I decided to just go with the sentences for simplicity reasons. files: Doc/library/unittest.rst | 21 ++++++++++++--------- Doc/whatsnew/3.4.rst | 3 +++ 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -239,9 +239,10 @@ Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be :ref:`modules ` or -:ref:`packages ` importable from the top-level directory of -the project (this means that their filenames must be valid -:ref:`identifiers `). +:ref:`packages ` (including :term:`namespace packages +`) importable from the top-level directory of +the project (this means that their filenames must be valid :ref:`identifiers +`). Test discovery is implemented in :meth:`TestLoader.discover`, but can also be used from the command line. The basic command-line usage is:: @@ -306,6 +307,9 @@ Test modules and packages can customize test loading and discovery by through the `load_tests protocol`_. +.. versionchanged:: 3.4 + Test discovery supports :term:`namespace packages `. + .. _organizing-tests: @@ -1654,12 +1658,11 @@ .. versionchanged:: 3.4 Modules that raise :exc:`SkipTest` on import are recorded as skips, - not errors. - - .. versionchanged:: 3.4 - Paths are sorted before being imported to ensure execution order for a - given test suite is the same even if the underlying file system's ordering - is not dependent on file name like in ext3/4. + not errors. + Discovery works for :term:`namespace packages `. + Paths are sorted before being imported so that execution order is + the same even if the underlying file system's ordering is not + dependent on file name. The following attributes of a :class:`TestLoader` can be configured either by diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1355,6 +1355,9 @@ formatted messages that were logged. (Contributed by Antoine Pitrou in :issue:`18937`.) +Test discovery now works with namespace packages (Contributed by Claudiu Popa +in :issue:`17457`.) + venv ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 18:12:00 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 1 Mar 2014 18:12:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_memoryview_is_?= =?utf-8?q?Sequence_and_supports_Reversed=2E?= Message-ID: <3fbvL82mF3z7LtS@mail.python.org> http://hg.python.org/cpython/rev/c440c5893d09 changeset: 89442:c440c5893d09 user: R David Murray date: Sat Mar 01 12:09:55 2014 -0500 summary: whatsnew: memoryview is Sequence and supports Reversed. files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -377,6 +377,10 @@ references to local variables from the frame. (Contributed by Antoine Pitrou in :issue:`17934`.) +* :class:`memoryview` is now registered as a :class:`Sequence `, + and supports the :func:`reversed` builtin. (Contributed by Nick Coghlan + and Claudiu Popa in :issue:`18690` and :issue:`19078`.) + New Modules =========== -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 23:01:25 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 1 Mar 2014 23:01:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320465=3A_Update_O?= =?utf-8?q?S_X_installer_build_to_use_SQLite_3=2E8=2E0=2E1=2E?= Message-ID: <3fc1m50CGqz7LqS@mail.python.org> http://hg.python.org/cpython/rev/5fa3f6d82d61 changeset: 89443:5fa3f6d82d61 user: Ned Deily date: Sat Mar 01 14:00:46 2014 -0800 summary: Issue #20465: Update OS X installer build to use SQLite 3.8.0.1. files: Mac/BuildScript/build-installer.py | 6 +++--- Misc/NEWS | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -283,9 +283,9 @@ ), ), dict( - name="SQLite 3.8.3", - url="http://www.sqlite.org/2014/sqlite-autoconf-3080300.tar.gz", - checksum='11572878dc0ac74ae370367a464ab5cf', + name="SQLite 3.8.3.1", + url="http://www.sqlite.org/2014/sqlite-autoconf-3080301.tar.gz", + checksum='509ff98d8dc9729b618b7e96612079c6', extra_cflags=('-Os ' '-DSQLITE_ENABLE_FTS4 ' '-DSQLITE_ENABLE_FTS3_PARENTHESIS ' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,8 @@ uninstalling pip (rather than failing) if the user has updated pip to a different version from the one bundled with ensurepip. +- Issue #20465: Update OS X installer build to use SQLite 3.8.0.1. + Tools/Demos ----------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 1 23:05:21 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 1 Mar 2014 23:05:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320465=3A_fix_NEWS?= =?utf-8?b?IHR5cG8gKGl0J3MgMy44LjMuMSk=?= Message-ID: <3fc1rd6cCDz7LvR@mail.python.org> http://hg.python.org/cpython/rev/a8470f88e7b4 changeset: 89444:a8470f88e7b4 user: Ned Deily date: Sat Mar 01 14:04:48 2014 -0800 summary: Issue #20465: fix NEWS typo (it's 3.8.3.1) files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,7 +46,7 @@ uninstalling pip (rather than failing) if the user has updated pip to a different version from the one bundled with ensurepip. -- Issue #20465: Update OS X installer build to use SQLite 3.8.0.1. +- Issue #20465: Update OS X installer build to use SQLite 3.8.3.1. Tools/Demos ----------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:14:56 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:14:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogZml4IHRlc3RfcG9z?= =?utf-8?q?ix=2Etest=5Finitgroups_to_work_without_supplemental_groups_=28c?= =?utf-8?q?loses?= Message-ID: <3fc4k862JMz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/82bcc626ffd4 changeset: 89445:82bcc626ffd4 branch: 3.3 parent: 89435:dcf4fbf446ca user: Benjamin Peterson date: Sat Mar 01 19:14:12 2014 -0500 summary: fix test_posix.test_initgroups to work without supplemental groups (closes #20249) files: Lib/test/test_posix.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -1144,7 +1144,7 @@ def test_initgroups(self): # find missing group - g = max(self.saved_groups) + 1 + g = max(self.saved_groups or [0]) + 1 name = pwd.getpwuid(posix.getuid()).pw_name posix.initgroups(name, g) self.assertIn(g, posix.getgroups()) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:14:58 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:14:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogZml4IHRlc3RfcG9z?= =?utf-8?q?ix=2Etest=5Finitgroups_to_work_without_supplemental_groups_=28c?= =?utf-8?q?loses?= Message-ID: <3fc4kB0bhlz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/1fbec1c44911 changeset: 89446:1fbec1c44911 branch: 2.7 parent: 89418:43c4073cb2e2 user: Benjamin Peterson date: Sat Mar 01 19:14:12 2014 -0500 summary: fix test_posix.test_initgroups to work without supplemental groups (closes #20249) files: Lib/test/test_posix.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -546,7 +546,7 @@ def test_initgroups(self): # find missing group - g = max(self.saved_groups) + 1 + g = max(self.saved_groups or [0]) + 1 name = pwd.getpwuid(posix.getuid()).pw_name posix.initgroups(name, g) self.assertIn(g, posix.getgroups()) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:14:59 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:14:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMjAyNDkp?= Message-ID: <3fc4kC2Nn1z7LlK@mail.python.org> http://hg.python.org/cpython/rev/a0abc7555d0e changeset: 89447:a0abc7555d0e parent: 89444:a8470f88e7b4 parent: 89445:82bcc626ffd4 user: Benjamin Peterson date: Sat Mar 01 19:14:48 2014 -0500 summary: merge 3.3 (#20249) files: Lib/test/test_posix.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -1161,7 +1161,7 @@ def test_initgroups(self): # find missing group - g = max(self.saved_groups) + 1 + g = max(self.saved_groups or [0]) + 1 name = pwd.getpwuid(posix.getuid()).pw_name posix.initgroups(name, g) self.assertIn(g, posix.getgroups()) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:16:58 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:16:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_add_Chris_Ange?= =?utf-8?q?lico?= Message-ID: <3fc4mV56Dsz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/406851b1814c changeset: 89448:406851b1814c branch: 3.3 parent: 89445:82bcc626ffd4 user: Benjamin Peterson date: Sat Mar 01 19:16:12 2014 -0500 summary: add Chris Angelico files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -40,6 +40,7 @@ Oliver Andrich Ross Andrus Juancarlo A?ez +Chris Angelico J?r?my Anger Ankur Ankan Jon Anglin -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:16:59 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:16:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_add_Chris_Ange?= =?utf-8?q?lico?= Message-ID: <3fc4mW6xPrz7LkL@mail.python.org> http://hg.python.org/cpython/rev/893c9e441cd7 changeset: 89449:893c9e441cd7 branch: 2.7 parent: 89446:1fbec1c44911 user: Benjamin Peterson date: Sat Mar 01 19:16:12 2014 -0500 summary: add Chris Angelico files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -33,6 +33,7 @@ Erik Anders?n Oliver Andrich Ross Andrus +Chris Angelico Ankur Ankan Heidi Annexstad ?ric Araujo -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 01:17:01 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 2 Mar 2014 01:17:01 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fc4mY1fMCz7Lks@mail.python.org> http://hg.python.org/cpython/rev/c07354bbb3c8 changeset: 89450:c07354bbb3c8 parent: 89447:a0abc7555d0e parent: 89448:406851b1814c user: Benjamin Peterson date: Sat Mar 01 19:16:51 2014 -0500 summary: merge 3.3 files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -40,6 +40,7 @@ Oliver Andrich Ross Andrus Juancarlo A?ez +Chris Angelico J?r?my Anger Ankur Ankan Jon Anglin -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 07:08:54 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 2 Mar 2014 07:08:54 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_pydist=2Ejson_should_be_read_?= =?utf-8?q?from_a_valid_container?= Message-ID: <3fcDZZ1SGgz7LjP@mail.python.org> http://hg.python.org/peps/rev/2d41b932cccc changeset: 5391:2d41b932cccc user: Nick Coghlan date: Sun Mar 02 16:08:11 2014 +1000 summary: pydist.json should be read from a valid container files: pep-0426.txt | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/pep-0426.txt b/pep-0426.txt --- a/pep-0426.txt +++ b/pep-0426.txt @@ -459,7 +459,13 @@ also be a wheel 1.1 format update after this PEP is approved that mandates provision of 2.0+ metadata. -Other tools involved in Python distribution may also use this format. +Note that these metadata files SHOULD NOT be processed if the version of the +containing location is too low to indicate that they are valid. Specifically, +unversioned ``sdist`` archives, unversioned installation database directories +and version 1.0 of the ``wheel`` specification do not cover ``pydist.json`` +files. + +Other tools involved in Python distribution MAY also use this format. As JSON files are generally awkward to edit by hand, it is RECOMMENDED that these metadata files be generated by build tools based on other -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 2 07:08:55 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 2 Mar 2014 07:08:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Postpone_python=2Emetadata=5F?= =?utf-8?q?hooks_for_the_time_being?= Message-ID: <3fcDZb3n1Lz7LjP@mail.python.org> http://hg.python.org/peps/rev/4a34df0faaec changeset: 5392:4a34df0faaec user: Nick Coghlan date: Sun Mar 02 16:08:45 2014 +1000 summary: Postpone python.metadata_hooks for the time being files: pep-0459.txt | 176 --------------------------------------- 1 files changed, 0 insertions(+), 176 deletions(-) diff --git a/pep-0459.txt b/pep-0459.txt --- a/pep-0459.txt +++ b/pep-0459.txt @@ -47,7 +47,6 @@ * ``python.integrator`` * ``python.commands`` * ``python.exports`` -* ``python.metadata_hooks`` All standard extensions are currently at version ``1.0``, and thus the ``extension_metadata`` field may be omitted without losing access to any @@ -512,181 +511,6 @@ } - -The ``metadata_hooks`` extension -================================ - -The ``metadata_hooks`` extension is used to define operations to be -invoked on the distribution in the following situations: - -* a relevant distribution has been installed or changed on the system -* a relevant distribution has been completely uninstalled from the system - -The following subfields determine when hooks are triggered: - -* ``export_groups``: trigger based on named export groups -* ``extensions``: trigger based on named extensions - -Note that distributions *do* trigger their own install hooks, but do -*not* trigger their own uninstall hooks. - - -Hook signatures ---------------- - -The currently defined metadata hooks are: - -* ``postinstall``: run after a relevant distribution has been installed, - upgraded or downgraded on the current system. May also be run as part - of a system state resync operation. If the hook is not defined, it - indicates no distribution specific actions are needed following - installation. -* ``postuninstall``: run after a relevant distribution has been completely - removed from the current system. If the hook is not defined, it indicates - no distribution specific actions are needed following uninstallation. - -The required signatures of these hooks are as follows:: - - def postinstall(current_meta, previous_meta=None): - """Run following installation or upgrade of a relevant distribution - - *current_meta* is the distribution metadata for the version now - installed on the current system - *previous_meta* is either omitted or ``None`` (indicating a fresh - install) or else the distribution metadata for the version that - was previously installed (indicating an upgrade, downgrade or - resynchronisation of the system state). - """ - - def postuninstall(previous_meta): - """Run after complete uninstallation of a relevant distribution - - *previous_meta* is the distribution metadata for the version that - was previously installed on the current system - """ - -Export group hooks ------------------- - -Export group hooks are named after the export group of interest:: - - "export_groups": { - "ComfyChair.plugins": { - "postinstall": "ComfyChair.plugins:install_hook", - "postuininstall": "ComfyChair.plugins:uninstall_hook" - } - } - -The nominated hooks will then be invoked appropriately for any distribution -that publishes that export group as part of their ``python.exports`` -extension metadata. - -A trailing ".*" may be used to request prefix matching rather than -requiring an exact match on the export group name. - - -Extension hooks ---------------- - -Extension hooks are named after the metadata extension of interest:: - - "extensions": { - "python.exports": { - "postinstall": "pip.export_group_hooks:run_install_hooks", - "postuininstall": "pip.export_group_hooks:run_uninstall_hooks" - } - "python.commands": { - "postinstall": "pip.command_hook:install_wrapper_scripts", - } - } - -(Note: this is just an example, but the intent is that pip *could* implement -that functionality that way if it wanted to). - -A trailing ".*" may be used to request prefix matching rather than -requiring an exact match on the extension name. - - -Guidelines for metadata hook invocation ---------------------------------------- - -.. note:: - - Metadata hooks are likely to run with elevated privileges, this needs - to be considered carefully (e.g. by *requiring* that metadata hook - installation be opt in when using the standard tools and running with - elevated privileges). - -The given parameter names are considered part of the hook signature. -Installation tools MUST call metadata hooks solely with keyword arguments. - -When metadata hooks are defined, it is assumed that they MUST be executed -to obtain a properly working installation of the distribution, and to -properly remove the distribution from a system. - -Installation tools MUST ensure the new or updated distribution is fully -installed, and available through the import system and installation database -when invoking install hooks. - -Installation tools MUST ensure the removed distribution is fully uninstalled, -and no longer available through the import system and installation database -when invoking uninstall hooks. - -Installation tools MUST call metadata hooks with full metadata (including -all extensions), rather than only the core metadata. - -Installation tools SHOULD invoke metadata hooks automatically after -installing a distribution from a binary archive. - -When installing from an sdist, source archive or VCS checkout, installation -tools SHOULD create a binary archive using ``setup.py bdist_wheel`` and -then install the binary archive normally (including invocation of any -metadata hooks). Installation tools SHOULD NOT invoke ``setup.py install`` -directly. - -Installation tools SHOULD treat an exception thrown by a metadata install -hook as a failure of the installation and revert any other changes made -to the system. The installed distribution responsible for the hook that -failed should be clearly indicated to the user. - -Installation tools SHOULD provide a warning to the user for any exception -thrown by a metadata uninstall hook, again clearly indicating to the user -the installed distribution that triggered the warning. - -Installation tools MUST NOT silently ignore metadata hooks, as failing -to call these hooks may result in a misconfigured installation that fails -unexpectedly at runtime. Installation tools MAY refuse to install -distributions that define metadata hooks, or require that users -explicitly opt in to permitting the installation of packages that -define such hooks. - - -Guidelines for metadata hook implementations --------------------------------------------- - -The given parameter names are considered part of the hook signature. -Metadata hook implementations MUST use the given parameter names. - -Metadata hooks SHOULD NOT be used to provide functionality that is -expected to be provided by installation tools (such as rewriting of -shebang lines and generation of executable wrappers for Windows). - -Metadata hook implementations MUST NOT make any assumptions regarding the -current working directory when they are invoked, and MUST NOT make -persistent alterations to the working directory or any other process global -state (other than potentially importing additional modules, or other -expected side effects of running the distribution). - -Metadata install hooks have access to the full metadata for the release being -installed, that of the previous/next release (as appropriate), as well as -to all the normal runtime information (such as available imports). Hook -implementations can use this information to perform additional platform -specific installation steps. To check for the presence or absence of -"extras", hook implementations should use the same runtime checks that -would be used during normal operation (such as checking for the availability -of the relevant dependencies). - - Copyright ========= -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 2 07:46:39 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 2 Mar 2014 07:46:39 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_formatting?= Message-ID: <3fcFQ70LPkz7Ljd@mail.python.org> http://hg.python.org/peps/rev/c728cf303050 changeset: 5393:c728cf303050 user: Nick Coghlan date: Sun Mar 02 16:41:09 2014 +1000 summary: Fix formatting files: pep-0461.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0461.txt b/pep-0461.txt --- a/pep-0461.txt +++ b/pep-0461.txt @@ -50,7 +50,7 @@ Proposed semantics for ``bytes`` and ``bytearray`` formatting -======================================= +============================================================= %-interpolation --------------- -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 2 07:46:40 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 2 Mar 2014 07:46:40 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Account_for_metadata_hook_def?= =?utf-8?q?erral_in_PEP_426?= Message-ID: <3fcFQ83rRvz7Ljr@mail.python.org> http://hg.python.org/peps/rev/272debef6b77 changeset: 5394:272debef6b77 user: Nick Coghlan date: Sun Mar 02 16:46:28 2014 +1000 summary: Account for metadata hook deferral in PEP 426 files: pep-0426.txt | 428 ++++++++++++++++++++------------------ 1 files changed, 225 insertions(+), 203 deletions(-) diff --git a/pep-0426.txt b/pep-0426.txt --- a/pep-0426.txt +++ b/pep-0426.txt @@ -1200,6 +1200,12 @@ ``extensions`` key. The keys MUST be valid prefixed names, while the values MUST themselves be nested mappings. +Two key names are reserved and MUST NOT be used by extensions, except as +described below: + +* ``extension_version`` +* ``required_extension`` + The following example shows the ``python.details`` and ``python.commands`` standard extensions from :pep:`459`:: @@ -1235,6 +1241,14 @@ the meaning of the extension. This practice will also make it easier to find authoritative documentation for metadata extensions. +Metadata extensions allow development tools to record information in the +metadata that may be useful during later phases of distribution, but is +not essential for dependency resolution or building the software. + + +Extension versioning +-------------------- + Extensions MUST be versioned, using the ``extension_version`` key. However, if this key is omitted, then the implied version is ``1.0``. @@ -1248,9 +1262,24 @@ extension metadata using the lowest metadata version that includes all of the needed fields. -Metadata extensions allow development tools to record information in the -metadata that may be useful during later phases of distribution, but is -not essential for dependency resolution or building the software. + +Required extensions +------------------- + +A project may consider correct handling of some extensions to be essential +to correct installation of the software. This is indicated by setting the +``required_extension`` field to ``true``. Setting it to ``false`` or +omitting it altogether indicates that processing the extension when +installing the distribution is not considered mandatory by the developers. + +Installation tools MUST fail if ``required_extension`` is set to ``true`` +for an extension and the tool does not have any ability to process that +particular extension (whether directly or through a tool-specific plugin +system). + +If an installation tool encounters a required extension it doesn't +understand when attempting to install from a wheel archive, it MAY fall +back on attempting to install from source rather than failing entirely. Extras (optional dependencies) @@ -1569,8 +1598,6 @@ * Updated obsolescence mechanism -* Metadata hook system - * More flexible system for defining contact points and contributors * Defined a recommended set of project URLs @@ -1709,20 +1736,199 @@ particular extensions to be provided as optional features. Possible future uses for extensions include declaration of plugins for -other distributions, hints for automatic conversion to Linux system -packages, and inclusion of CVE references to mark security releases. +other distributions and hints for automatic conversion to Linux system +packages. + +The ability to declare an extension as required is included primarily to +allow the definition of the metadata hooks extension to be deferred until +some time after the initial adoption of the metadata 2.0 specification. If +a distribution needs a ``postinstall`` hook to run in order to complete +the installation successfully, then earlier versions of tools should fall +back to installing from source rather than installing from a wheel file and +then failing to run the expected postinstall hook. + + +Changes to environment markers +------------------------------ + +There are three substantive changes to environment markers in this version: + +* ``platform_release`` was added, as it provides more useful information + than ``platform_version`` on at least Linux and Mac OS X (specifically, + it provides details of the running kernel version) +* ordered comparison of strings is allowed, as this is more useful for + setting minimum and maximum versions where conditional dependencies + are needed or where a platform is supported +* comparison chaining is explicitly allowed, as this becomes useful in the + presence of ordered comparisons + +The other changes to environment markers are just clarifications and +simplifications to make them easier to use. + +The arbitrariness of the choice of ``.`` and ``_`` in the different +variables was addressed by standardising on ``_`` (as these are all +predefined variables rather than live references into the Python module +namespace) + +The use of parentheses for grouping was explicitly noted to address some +underspecified behaviour in the previous version of the specification. + + +Updated contact information +--------------------------- + +This feature is provided by the ``python.project`` and +``python.integrator`` extensions in :pep:`459`. + +The switch to JSON made it possible to provide a more flexible +system for defining multiple contact points for a project, as well as +listing other contributors. + +The ``type`` concept allows for preservation of the distinction between +the original author of a project, and a lead maintainer that takes over +at a later date. + + +Changes to project URLs +----------------------- + +This feature is provided by the ``python.project`` and +``python.integrator`` extensions in :pep:`459`. + +In addition to allow arbitrary strings as project URL labels, the new +metadata standard also defines a recommend set of four URL labels for +a distribution's home page, documentation, source control and issue tracker. + + +Changes to platform support +--------------------------- + +The new environment marker system makes it possible to define supported +platforms in a way that is actually amenable to automated processing. This +has been used to replace several older fields with poorly defined semantics. + +For the moment, the old ``Requires-External`` field has been removed +entirely. The metadata extension mechanism will hopefully prove to be a more +useful replacement. + + +Updated obsolescence mechanism +------------------------------ + +The marker to indicate when a project is obsolete and should be replaced +has been moved to the obsolete project (the new ``obsoleted_by`` field), +replacing the previous marker on the replacement project (the removed +``Obsoletes-Dist`` field). + +This should allow distribution tools to more easily warn users of +obsolete projects and their suggested replacements. + +The ``Obsoletes-Dist`` header is removed rather than deprecated as it +is not widely supported, and so removing it does not present any significant +barrier to tools and projects adopting the new metadata format. + + +Included text documents +----------------------- + +This feature is provided by the ``python.details`` extension in :pep:`459`. + +Currently, PyPI attempts to determine the description's markup format by +rendering it as reStructuredText, and if that fails, treating it as plain +text. + +Furthermore, many projects simply read their long description in from an +existing README file in ``setup.py``. The popularity of this practice is +only expected to increase, as many online version control systems +(including both GitHub and BitBucket) automatically display such files +on the landing page for the project. + +Standardising on the inclusion of the long description as a separate +file in the ``dist-info`` directory allows this to be simplified: + +* An existing file can just be copied into the ``dist-info`` directory as + part of creating the sdist +* The expected markup format can be determined by inspecting the file + extension of the specified path + +Allowing the intended format to be stated explicitly in the path allows +the format guessing to be removed and more informative error reports to be +provided to users when a rendering error occurs. + +This is especially helpful since PyPI applies additional restrictions to +the rendering process for security reasons, thus a description that renders +correctly on a developer's system may still fail to render on the server. + +The document naming system used to achieve this then makes it relatively +straightforward to allow declaration of alternative markup formats like +HTML, Markdown and AsciiDoc through the use of appropriate file +extensions, as well as to define similar included documents for the +project's license and changelog. + +Grouping the included document names into a single top level field gives +automated tools the option of treating them as arbitrary documents without +worrying about their contents. + +Requiring that the included documents be added to the ``dist-info`` metadata +directory means that the complete metadata for the distribution can be +extracted from an sdist or binary archive simply by extracting that +directory, without needing to check for references to other files in the +sdist. + + +Appendix D: Deferred features +============================= + +Several potentially useful features have been deliberately deferred in +order to better prioritise our efforts in migrating to the new metadata +standard. These all reflect information that may be nice to have in the +new metadata, but which can be readily added in metadata 2.1 without +breaking any use cases already supported by metadata 2.0. + +Once the ``pypi``, ``setuptools``, ``pip``, ``wheel`` and ``distlib`` +projects support creation and consumption of metadata 2.0, then we may +revisit the creation of metadata 2.1 with some or all of these additional +features. + + +MIME type registration +---------------------- + +At some point after acceptance of the PEP, I will likely submit the +following MIME type registration requests to IANA: + +* Full metadata: ``application/vnd.python.pydist+json`` +* Essential dependency resolution metadata: + ``application/vnd.python.pydist-dependencies+json`` + +It's even possible we may be able to just register the ``vnd.python`` +namespace under the banner of the PSF rather than having to register +the individual subformats. + + +String methods in environment markers +------------------------------------- + +Supporting at least ".startswith" and ".endswith" string methods in +environment markers would allow some conditions to be written more +naturally. For example, ``"sys.platform.startswith('win')"`` is a +somewhat more intuitive way to mark Windows specific dependencies, +since ``"'win' in sys.platform"`` is incorrect thanks to ``cygwin`` +and the fact that 64-bit Windows still shows up as ``win32`` is more +than a little strange. Support for metadata hooks --------------------------- -This feature is provided by the ``python.metadata_hooks`` extension in -:pep:`459`. - -The new metadata hook system is designed to allow the wheel format to fully -replace direct installation on deployment targets, by allowing projects to -explicitly define code that should be executed following installation from -a wheel file. +While a draft proposal for a `metadata hook system +`__ +has been created, that proposal is not part of the initial set of standard +metadata extensions in PEP 459. + +A metadata hook system would allow the wheel format to fully replace direct +installation on deployment targets, by allowing projects to explicitly +define code that should be executed following installation from a wheel file. This may be something relatively simple, like the `two line refresh `__ @@ -1741,195 +1947,11 @@ meaning they cannot be deferred to implicit execution on first use of the distribution. -The metadata hook and metadata extension systems allow support for such -activities to be pursued independently by the individual platform -communities, while still interoperating with the cross-platform Python -tools. - -Legacy packages that expect to able to run code on target systems using -``setup.py install`` will no longer work correctly. Such packages will -already break when pip 1.4+ is configured to use a wheel cache directory. - - -Changes to environment markers ------------------------------- - -There are three substantive changes to environment markers in this version: - -* ``platform_release`` was added, as it provides more useful information - than ``platform_version`` on at least Linux and Mac OS X (specifically, - it provides details of the running kernel version) -* ordered comparison of strings is allowed, as this is more useful for - setting minimum and maximum versions where conditional dependencies - are needed or where a platform is supported -* comparison chaining is explicitly allowed, as this becomes useful in the - presence of ordered comparisons - -The other changes to environment markers are just clarifications and -simplifications to make them easier to use. - -The arbitrariness of the choice of ``.`` and ``_`` in the different -variables was addressed by standardising on ``_`` (as these are all -predefined variables rather than live references into the Python module -namespace) - -The use of parentheses for grouping was explicitly noted to address some -underspecified behaviour in the previous version of the specification. - - -Updated contact information ---------------------------- - -This feature is provided by the ``python.project`` and -``python.integrator`` extensions in :pep:`459`. - -The switch to JSON made it possible to provide a more flexible -system for defining multiple contact points for a project, as well as -listing other contributors. - -The ``type`` concept allows for preservation of the distinction between -the original author of a project, and a lead maintainer that takes over -at a later date. - - -Changes to project URLs ------------------------ - -This feature is provided by the ``python.project`` and -``python.integrator`` extensions in :pep:`459`. - -In addition to allow arbitrary strings as project URL labels, the new -metadata standard also defines a recommend set of four URL labels for -a distribution's home page, documentation, source control and issue tracker. - - -Changes to platform support ---------------------------- - -The new environment marker system makes it possible to define supported -platforms in a way that is actually amenable to automated processing. This -has been used to replace several older fields with poorly defined semantics. - -For the moment, the old ``Requires-External`` field has been removed -entirely. The combination of explicit support for post install hooks and the -metadata extension mechanism will hopefully prove to be a more useful -replacement. - - -Updated obsolescence mechanism ------------------------------- - -The marker to indicate when a project is obsolete and should be replaced -has been moved to the obsolete project (the new ``obsoleted_by`` field), -replacing the previous marker on the replacement project (the removed -``Obsoletes-Dist`` field). - -This should allow distribution tools to more easily warn users of -obsolete projects and their suggested replacements. - -The ``Obsoletes-Dist`` header is removed rather than deprecated as it -is not widely supported, and so removing it does not present any significant -barrier to tools and projects adopting the new metadata format. - - -Included text documents ------------------------ - -This feature is provided by the ``python.details`` extension in :pep:`459`. - -Currently, PyPI attempts to determine the description's markup format by -rendering it as reStructuredText, and if that fails, treating it as plain -text. - -Furthermore, many projects simply read their long description in from an -existing README file in ``setup.py``. The popularity of this practice is -only expected to increase, as many online version control systems -(including both GitHub and BitBucket) automatically display such files -on the landing page for the project. - -Standardising on the inclusion of the long description as a separate -file in the ``dist-info`` directory allows this to be simplified: - -* An existing file can just be copied into the ``dist-info`` directory as - part of creating the sdist -* The expected markup format can be determined by inspecting the file - extension of the specified path - -Allowing the intended format to be stated explicitly in the path allows -the format guessing to be removed and more informative error reports to be -provided to users when a rendering error occurs. - -This is especially helpful since PyPI applies additional restrictions to -the rendering process for security reasons, thus a description that renders -correctly on a developer's system may still fail to render on the server. - -The document naming system used to achieve this then makes it relatively -straightforward to allow declaration of alternative markup formats like -HTML, Markdown and AsciiDoc through the use of appropriate file -extensions, as well as to define similar included documents for the -project's license and changelog. - -Grouping the included document names into a single top level field gives -automated tools the option of treating them as arbitrary documents without -worrying about their contents. - -Requiring that the included documents be added to the ``dist-info`` metadata -directory means that the complete metadata for the distribution can be -extracted from an sdist or binary archive simply by extracting that -directory, without needing to check for references to other files in the -sdist. - - -Appendix D: Deferred features -============================= - -Several potentially useful features have been deliberately deferred in -order to better prioritise our efforts in migrating to the new metadata -standard. These all reflect information that may be nice to have in the -new metadata, but which can be readily added in metadata 2.1 without -breaking any use cases already supported by metadata 2.0. - -Once the ``pypi``, ``setuptools``, ``pip``, ``wheel`` and ``distlib`` -projects support creation and consumption of metadata 2.0, then we may -revisit the creation of metadata 2.1 with some or all of these additional -features. - - -MIME type registration ----------------------- - -At some point after acceptance of the PEP, I will likely submit the -following MIME type registration requests to IANA: - -* Full metadata: ``application/vnd.python.pydist+json`` -* Essential dependency resolution metadata: - ``application/vnd.python.pydist-dependencies+json`` - -It's even possible we may be able to just register the ``vnd.python`` -namespace under the banner of the PSF rather than having to register -the individual subformats. - - -String methods in environment markers -------------------------------------- - -Supporting at least ".startswith" and ".endswith" string methods in -environment markers would allow some conditions to be written more -naturally. For example, ``"sys.platform.startswith('win')"`` is a -somewhat more intuitive way to mark Windows specific dependencies, -since ``"'win' in sys.platform"`` is incorrect thanks to ``cygwin`` -and the fact that 64-bit Windows still shows up as ``win32`` is more -than a little strange. - - -Additional install hooks ------------------------- - -In addition to the postinstall and postuninstall hooks currently described -in :pep:`459`, other distribution systems (like RPM) include the notion of -preinstall and postuninstall hooks. These hooks would run with the runtime -dependencies installed, but without the distribution itself. These have -been deliberately omitted for the time being. +For the time being, any such system is being left to the realm of tool +specific metadata extensions. This does mean that affected projects may +choose not to publish wheel files, instead continuing to rely on source +distributions until the relevant extension is well defined and widely +supported. Metabuild system -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 2 09:22:52 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:52 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQnVtcCB0byAzLjMu?= =?utf-8?q?5rc1=2E?= Message-ID: <3fcHY82Mfnz7Ljr@mail.python.org> http://hg.python.org/cpython/rev/64e58df40786 changeset: 89451:64e58df40786 branch: 3.3 parent: 89332:a69945e43ff3 user: Georg Brandl date: Sun Feb 23 08:30:06 2014 +0100 summary: Bump to 3.3.5rc1. files: Include/patchlevel.h | 8 ++++---- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Misc/NEWS | 2 +- Misc/RPM/python-3.3.spec | 2 +- README | 4 ++-- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 3 -#define PY_MICRO_VERSION 4 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL -#define PY_RELEASE_SERIAL 0 +#define PY_MICRO_VERSION 5 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA +#define PY_RELEASE_SERIAL 1 /* Version as a string */ -#define PY_VERSION "3.3.4+" +#define PY_VERSION "3.3.5rc1" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.4" +__version__ = "3.3.5rc1" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.4" +IDLE_VERSION = "3.3.5rc1" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -5,7 +5,7 @@ What's New in Python 3.3.5 release candidate 1? =============================================== -*Release date: XXXX-XX-XX* +*Release date: 23-Feb-2014* Core and Builtins ----------------- diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.4 +%define version 3.3.5rc1 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,5 +1,5 @@ -This is Python version 3.3.4 -============================ +This is Python version 3.3.5 release candidate 1 +================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Python Software Foundation. All rights reserved. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:53 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzIwNzE5OiBEaXNh?= =?utf-8?q?ble_the_robotparser_python=2Eorg_test_until_the_gzip_encoding_i?= =?utf-8?q?ssue?= Message-ID: <3fcHY93mS2z7LkB@mail.python.org> http://hg.python.org/cpython/rev/540ce9bb19e8 changeset: 89452:540ce9bb19e8 branch: 3.3 parent: 89333:d8f48717b74e user: Georg Brandl date: Sun Feb 23 08:45:15 2014 +0100 summary: #20719: Disable the robotparser python.org test until the gzip encoding issue can be sorted. files: Lib/test/test_robotparser.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -275,6 +275,7 @@ self.skipTest('%s is unavailable' % url) self.assertEqual(parser.can_fetch("*", robots_url), False) + @unittest.skip('does not handle the gzip encoding delivered by pydotorg') def testPythonOrg(self): support.requires('network') with support.transient_internet('www.python.org'): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:54 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:54 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?= =?utf-8?q?_merge?= Message-ID: <3fcHYB56wyz7Ll3@mail.python.org> http://hg.python.org/cpython/rev/9ec811df548e changeset: 89453:9ec811df548e branch: 3.3 tag: v3.3.5rc1 parent: 89452:540ce9bb19e8 parent: 89451:64e58df40786 user: Georg Brandl date: Sun Feb 23 08:46:00 2014 +0100 summary: merge files: Include/patchlevel.h | 8 ++++---- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Misc/NEWS | 2 +- Misc/RPM/python-3.3.spec | 2 +- README | 4 ++-- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 3 -#define PY_MICRO_VERSION 4 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL -#define PY_RELEASE_SERIAL 0 +#define PY_MICRO_VERSION 5 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA +#define PY_RELEASE_SERIAL 1 /* Version as a string */ -#define PY_VERSION "3.3.4+" +#define PY_VERSION "3.3.5rc1" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.4" +__version__ = "3.3.5rc1" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.4" +IDLE_VERSION = "3.3.5rc1" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -5,7 +5,7 @@ What's New in Python 3.3.5 release candidate 1? =============================================== -*Release date: XXXX-XX-XX* +*Release date: 23-Feb-2014* Core and Builtins ----------------- diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.4 +%define version 3.3.5rc1 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,5 +1,5 @@ -This is Python version 3.3.4 -============================ +This is Python version 3.3.5 release candidate 1 +================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Python Software Foundation. All rights reserved. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:55 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Added_tag_v3?= =?utf-8?q?=2E3=2E5rc1_for_changeset_9ec811df548e?= Message-ID: <3fcHYC6RTKz7Lkp@mail.python.org> http://hg.python.org/cpython/rev/7118f103c593 changeset: 89454:7118f103c593 branch: 3.3 user: Georg Brandl date: Sun Feb 23 08:46:11 2014 +0100 summary: Added tag v3.3.5rc1 for changeset 9ec811df548e files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -120,3 +120,4 @@ c3896275c0f61b2510a6c7e6c458a750359a91b8 v3.3.3 fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 +9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:57 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:57 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E3=2E5rc1_release_commits_with_3=2E3_branch?= Message-ID: <3fcHYF1BCnz7LkB@mail.python.org> http://hg.python.org/cpython/rev/e56522b1c42d changeset: 89455:e56522b1c42d branch: 3.3 parent: 89448:406851b1814c parent: 89454:7118f103c593 user: Georg Brandl date: Sun Mar 02 08:54:15 2014 +0100 summary: merge 3.3.5rc1 release commits with 3.3 branch files: .hgtags | 1 + Include/patchlevel.h | 8 ++++---- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Lib/test/test_robotparser.py | 1 + Misc/NEWS | 2 +- Misc/RPM/python-3.3.spec | 2 +- README | 4 ++-- 8 files changed, 12 insertions(+), 10 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -120,3 +120,4 @@ c3896275c0f61b2510a6c7e6c458a750359a91b8 v3.3.3 fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 +9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 3 -#define PY_MICRO_VERSION 4 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL -#define PY_RELEASE_SERIAL 0 +#define PY_MICRO_VERSION 5 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA +#define PY_RELEASE_SERIAL 1 /* Version as a string */ -#define PY_VERSION "3.3.4+" +#define PY_VERSION "3.3.5rc1" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.4" +__version__ = "3.3.5rc1" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.4" +IDLE_VERSION = "3.3.5rc1" diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -275,6 +275,7 @@ self.skipTest('%s is unavailable' % url) self.assertEqual(parser.can_fetch("*", robots_url), False) + @unittest.skip('does not handle the gzip encoding delivered by pydotorg') def testPythonOrg(self): support.requires('network') with support.transient_internet('www.python.org'): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -5,7 +5,7 @@ What's New in Python 3.3.5 release candidate 1? =============================================== -*Release date: XXXX-XX-XX* +*Release date: 23-Feb-2014* Core and Builtins ----------------- diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.4 +%define version 3.3.5rc1 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,5 +1,5 @@ -This is Python version 3.3.4 -============================ +This is Python version 3.3.5 release candidate 1 +================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Python Software Foundation. All rights reserved. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:58 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwNDA0?= =?utf-8?q?=3A_reject_non-text_encodings_early_in_TextIOWrapper=2E?= Message-ID: <3fcHYG44ncz7LlS@mail.python.org> http://hg.python.org/cpython/rev/140a69d950eb changeset: 89456:140a69d950eb branch: 3.3 user: Georg Brandl date: Sun Mar 02 09:18:31 2014 +0100 summary: Issue #20404: reject non-text encodings early in TextIOWrapper. files: Include/codecs.h | 20 +++++++ Lib/_pyio.py | 5 + Lib/test/test_io.py | 30 ++++++++-- Modules/_io/textio.c | 34 ++++++++---- Python/codecs.c | 88 +++++++++++++++++++++++-------- 5 files changed, 136 insertions(+), 41 deletions(-) diff --git a/Include/codecs.h b/Include/codecs.h --- a/Include/codecs.h +++ b/Include/codecs.h @@ -104,7 +104,14 @@ Please note that these APIs are internal and should not be used in Python C extensions. + XXX (ncoghlan): should we make these, or something like them, public + in Python 3.5+? + */ +PyAPI_FUNC(PyObject *) _PyCodec_LookupTextEncoding( + const char *encoding, + const char *alternate_command + ); PyAPI_FUNC(PyObject *) _PyCodec_EncodeText( PyObject *object, @@ -117,6 +124,19 @@ const char *encoding, const char *errors ); + +/* These two aren't actually text encoding specific, but _io.TextIOWrapper + * is the only current API consumer. + */ +PyAPI_FUNC(PyObject *) _PyCodecInfo_GetIncrementalDecoder( + PyObject *codec_info, + const char *errors + ); + +PyAPI_FUNC(PyObject *) _PyCodecInfo_GetIncrementalEncoder( + PyObject *codec_info, + const char *errors + ); #endif diff --git a/Lib/_pyio.py b/Lib/_pyio.py --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -1495,6 +1495,11 @@ if not isinstance(encoding, str): raise ValueError("invalid encoding: %r" % encoding) + if not codecs.lookup(encoding)._is_text_encoding: + msg = ("%r is not a text encoding; " + "use codecs.open() to handle arbitrary codecs") + raise LookupError(msg % encoding) + if errors is None: errors = "strict" else: diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -1955,6 +1955,15 @@ self.assertRaises(TypeError, t.__init__, b, newline=42) self.assertRaises(ValueError, t.__init__, b, newline='xyzzy') + def test_non_text_encoding_codecs_are_rejected(self): + # Ensure the constructor complains if passed a codec that isn't + # marked as a text encoding + # http://bugs.python.org/issue20404 + r = self.BytesIO() + b = self.BufferedWriter(r) + with self.assertRaisesRegex(LookupError, "is not a text encoding"): + self.TextIOWrapper(b, encoding="hex_codec") + def test_detach(self): r = self.BytesIO() b = self.BufferedWriter(r) @@ -2607,15 +2616,22 @@ def test_illegal_decoder(self): # Issue #17106 + # Bypass the early encoding check added in issue 20404 + def _make_illegal_wrapper(): + quopri = codecs.lookup("quopri_codec") + quopri._is_text_encoding = True + try: + t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), + newline='\n', encoding="quopri_codec") + finally: + quopri._is_text_encoding = False + return t # Crash when decoder returns non-string - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + t = _make_illegal_wrapper() self.assertRaises(TypeError, t.read, 1) - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + t = _make_illegal_wrapper() self.assertRaises(TypeError, t.readline) - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + t = _make_illegal_wrapper() self.assertRaises(TypeError, t.read) @@ -3053,6 +3069,7 @@ class CMiscIOTest(MiscIOTest): io = io + shutdown_error = "RuntimeError: could not find io module state" def test_readinto_buffer_overflow(self): # Issue #18025 @@ -3065,6 +3082,7 @@ class PyMiscIOTest(MiscIOTest): io = pyio + shutdown_error = "LookupError: unknown encoding: ascii" @unittest.skipIf(os.name == 'nt', 'POSIX signals required for this test.') diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -836,7 +836,7 @@ char *kwlist[] = {"buffer", "encoding", "errors", "newline", "line_buffering", "write_through", NULL}; - PyObject *buffer, *raw; + PyObject *buffer, *raw, *codec_info = NULL; char *encoding = NULL; char *errors = NULL; char *newline = NULL; @@ -951,6 +951,17 @@ "could not determine default encoding"); } + /* Check we have been asked for a real text encoding */ + codec_info = _PyCodec_LookupTextEncoding(encoding, "codecs.open()"); + if (codec_info == NULL) { + Py_CLEAR(self->encoding); + goto error; + } + + /* XXX: Failures beyond this point have the potential to leak elements + * of the partially constructed object (like self->encoding) + */ + if (errors == NULL) errors = "strict"; self->errors = PyBytes_FromString(errors); @@ -965,7 +976,7 @@ if (newline) { self->readnl = PyUnicode_FromString(newline); if (self->readnl == NULL) - return -1; + goto error; } self->writetranslate = (newline == NULL || newline[0] != '\0'); if (!self->readuniversal && self->readnl) { @@ -989,8 +1000,8 @@ if (r == -1) goto error; if (r == 1) { - self->decoder = PyCodec_IncrementalDecoder( - encoding, errors); + self->decoder = _PyCodecInfo_GetIncrementalDecoder(codec_info, + errors); if (self->decoder == NULL) goto error; @@ -1014,17 +1025,12 @@ if (r == -1) goto error; if (r == 1) { - PyObject *ci; - self->encoder = PyCodec_IncrementalEncoder( - encoding, errors); + self->encoder = _PyCodecInfo_GetIncrementalEncoder(codec_info, + errors); if (self->encoder == NULL) goto error; /* Get the normalized named of the codec */ - ci = _PyCodec_Lookup(encoding); - if (ci == NULL) - goto error; - res = _PyObject_GetAttrId(ci, &PyId_name); - Py_DECREF(ci); + res = _PyObject_GetAttrId(codec_info, &PyId_name); if (res == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError)) PyErr_Clear(); @@ -1044,6 +1050,9 @@ Py_XDECREF(res); } + /* Finished sorting out the codec details */ + Py_DECREF(codec_info); + self->buffer = buffer; Py_INCREF(buffer); @@ -1106,6 +1115,7 @@ return 0; error: + Py_XDECREF(codec_info); return -1; } diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -243,20 +243,15 @@ return v; } -/* Helper function to create an incremental codec. */ +/* Helper functions to create an incremental codec. */ +static +PyObject *codec_makeincrementalcodec(PyObject *codec_info, + const char *errors, + const char *attrname) +{ + PyObject *ret, *inccodec; -static -PyObject *codec_getincrementalcodec(const char *encoding, - const char *errors, - const char *attrname) -{ - PyObject *codecs, *ret, *inccodec; - - codecs = _PyCodec_Lookup(encoding); - if (codecs == NULL) - return NULL; - inccodec = PyObject_GetAttrString(codecs, attrname); - Py_DECREF(codecs); + inccodec = PyObject_GetAttrString(codec_info, attrname); if (inccodec == NULL) return NULL; if (errors) @@ -267,6 +262,21 @@ return ret; } +static +PyObject *codec_getincrementalcodec(const char *encoding, + const char *errors, + const char *attrname) +{ + PyObject *codec_info, *ret; + + codec_info = _PyCodec_Lookup(encoding); + if (codec_info == NULL) + return NULL; + ret = codec_makeincrementalcodec(codec_info, errors, attrname); + Py_DECREF(codec_info); + return ret; +} + /* Helper function to create a stream codec. */ static @@ -290,6 +300,24 @@ return streamcodec; } +/* Helpers to work with the result of _PyCodec_Lookup + + */ +PyObject *_PyCodecInfo_GetIncrementalDecoder(PyObject *codec_info, + const char *errors) +{ + return codec_makeincrementalcodec(codec_info, errors, + "incrementaldecoder"); +} + +PyObject *_PyCodecInfo_GetIncrementalEncoder(PyObject *codec_info, + const char *errors) +{ + return codec_makeincrementalcodec(codec_info, errors, + "incrementalencoder"); +} + + /* Convenience APIs to query the Codec registry. All APIs return a codec object with incremented refcount. @@ -447,15 +475,12 @@ } /* Text encoding/decoding API */ -static -PyObject *codec_getitem_checked(const char *encoding, - const char *operation_name, - int index) +PyObject * _PyCodec_LookupTextEncoding(const char *encoding, + const char *alternate_command) { _Py_IDENTIFIER(_is_text_encoding); PyObject *codec; PyObject *attr; - PyObject *v; int is_text_codec; codec = _PyCodec_Lookup(encoding); @@ -482,27 +507,44 @@ Py_DECREF(codec); PyErr_Format(PyExc_LookupError, "'%.400s' is not a text encoding; " - "use codecs.%s() to handle arbitrary codecs", - encoding, operation_name); + "use %s to handle arbitrary codecs", + encoding, alternate_command); return NULL; } } } + /* This appears to be a valid text encoding */ + return codec; +} + + +static +PyObject *codec_getitem_checked(const char *encoding, + const char *alternate_command, + int index) +{ + PyObject *codec; + PyObject *v; + + codec = _PyCodec_LookupTextEncoding(encoding, alternate_command); + if (codec == NULL) + return NULL; + v = PyTuple_GET_ITEM(codec, index); + Py_INCREF(v); Py_DECREF(codec); - Py_INCREF(v); return v; } static PyObject * _PyCodec_TextEncoder(const char *encoding) { - return codec_getitem_checked(encoding, "encode", 0); + return codec_getitem_checked(encoding, "codecs.encode()", 0); } static PyObject * _PyCodec_TextDecoder(const char *encoding) { - return codec_getitem_checked(encoding, "decode", 1); + return codec_getitem_checked(encoding, "codecs.decode()", 1); } PyObject *_PyCodec_EncodeText(PyObject *object, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:22:59 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:22:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Update_NEWS_fo?= =?utf-8?b?ciAzLjMuNXJjMi4=?= Message-ID: <3fcHYH5v5hz7LlF@mail.python.org> http://hg.python.org/cpython/rev/69305c61cd5b changeset: 89457:69305c61cd5b branch: 3.3 user: Georg Brandl date: Sun Mar 02 09:18:41 2014 +0100 summary: Update NEWS for 3.3.5rc2. files: Misc/NEWS | 55 +++++++++++++++++++++++++++++------------- 1 files changed, 38 insertions(+), 17 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,10 +2,10 @@ Python News +++++++++++ -What's New in Python 3.3.5 release candidate 1? +What's New in Python 3.3.5 release candidate 2? =============================================== -*Release date: 23-Feb-2014* +*Release date: 02-Mar-2014* Core and Builtins ----------------- @@ -19,13 +19,6 @@ TypeError for an unexpected output type. (The latter mechanism remains in place for third party non-text encodings) -- Issue #20588: Make Python-ast.c C89 compliant. - -- Issue #20437: Fixed 21 potential bugs when deleting objects references. - -- Issue #20538: UTF-7 incremental decoder produced inconsistant string when - input was truncated in BASE64 section. - Library ------- @@ -38,6 +31,42 @@ spurious crashes or SystemErrors when importing modules or packages from a zip file. The change causing the problem was reverted. +- Issue #20404: io.TextIOWrapper (and hence the open() builtin) now uses the + internal codec marking system added for issue #19619 to throw LookupError + for known non-text encodings at stream construction time. The existing + output type checks remain in place to deal with unmarked third party + codecs. + +Tests +----- + +- Issue #20743: Fix a reference leak in test_tcl. + +Tools/Demos +----------- + +- Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. + Patch by Arfrever Frehtes Taifersar Arahesis. + + +What's New in Python 3.3.5 release candidate 1? +=============================================== + +*Release date: 23-Feb-2014* + +Core and Builtins +----------------- + +- Issue #20588: Make Python-ast.c C89 compliant. + +- Issue #20437: Fixed 21 potential bugs when deleting objects references. + +- Issue #20538: UTF-7 incremental decoder produced inconsistant string when + input was truncated in BASE64 section. + +Library +------- + - Issue #20635: Fixed grid_columnconfigure() and grid_rowconfigure() methods of Tkinter widgets to work in wantobjects=True mode. @@ -135,8 +164,6 @@ Tests ----- -- Issue #20743: Fix a reference leak in test_tcl. - - Issue #20510: Rewrote test_exit in test_sys to match existing comments, use modern unittest features, and use helpers from test.script_helper instead of using subprocess directly. Patch by Gareth Rees. @@ -167,12 +194,6 @@ - Issue #20609: Restored the ability to build 64-bit Windows binaries on 32-bit Windows, which was broken by the change in issue #19788. -Tools/Demos ------------ - -- Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. - Patch by Arfrever Frehtes Taifersar Arahesis. - What's New in Python 3.3.4? =========================== -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 09:23:01 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:23:01 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQnVtcCB0byAzLjMu?= =?utf-8?q?5rc2=2E?= Message-ID: <3fcHYK0Q9kz7LlF@mail.python.org> http://hg.python.org/cpython/rev/ca5635efe090 changeset: 89458:ca5635efe090 branch: 3.3 user: Georg Brandl date: Sun Mar 02 09:19:03 2014 +0100 summary: Bump to 3.3.5rc2. files: Include/patchlevel.h | 4 ++-- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Misc/RPM/python-3.3.spec | 2 +- README | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 3 #define PY_MICRO_VERSION 5 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 1 +#define PY_RELEASE_SERIAL 2 /* Version as a string */ -#define PY_VERSION "3.3.5rc1" +#define PY_VERSION "3.3.5rc2" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.5rc1" +__version__ = "3.3.5rc2" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.5rc1" +IDLE_VERSION = "3.3.5rc2" diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.5rc1 +%define version 3.3.5rc2 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.3.5 release candidate 1 +This is Python version 3.3.5 release candidate 2 ================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Mar 2 09:24:01 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 02 Mar 2014 09:24:01 +0100 Subject: [Python-checkins] Daily reference leaks (c07354bbb3c8): sum=4 Message-ID: results for c07354bbb3c8 on branch "default" -------------------------------------------- test_asyncio leaked [0, 4, 0] memory blocks, sum=4 test_site leaked [0, 2, -2] references, sum=0 test_site leaked [0, 2, -2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogyWCgdI', '-x'] From python-checkins at python.org Sun Mar 2 09:27:31 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 09:27:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_merge_with_3=2E3?= Message-ID: <3fcHfW2h2xz7LjX@mail.python.org> http://hg.python.org/cpython/rev/ea03b0fa5bd3 changeset: 89459:ea03b0fa5bd3 parent: 89450:c07354bbb3c8 parent: 89458:ca5635efe090 user: Georg Brandl date: Sun Mar 02 09:28:24 2014 +0100 summary: merge with 3.3 files: .hgtags | 1 + Lib/test/test_robotparser.py | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -120,6 +120,7 @@ c3896275c0f61b2510a6c7e6c458a750359a91b8 v3.3.3 fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 +9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 46535f65e7f3bcdcf176f36d34bc1fed719ffd2b v3.4.0a1 9265a2168e2cb2a84785d8717792acc661e6b692 v3.4.0a2 dd9cdf90a5073510877e9dd5112f8e6cf20d5e89 v3.4.0a3 diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -275,6 +275,7 @@ self.skipTest('%s is unavailable' % url) self.assertEqual(parser.can_fetch("*", robots_url), False) + @unittest.skip('does not handle the gzip encoding delivered by pydotorg') def testPythonOrg(self): support.requires('network') with support.transient_internet('www.python.org'): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 10:17:28 2014 From: python-checkins at python.org (terry.reedy) Date: Sun, 2 Mar 2014 10:17:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_issue_=2320728=3A_R?= =?utf-8?q?emove_unused_import_added_in_rev_42366e293b7b_and?= Message-ID: <3fcJm833KGz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/d3732760e3d6 changeset: 89460:d3732760e3d6 user: Terry Jan Reedy date: Sun Mar 02 04:17:01 2014 -0500 summary: Closes issue #20728: Remove unused import added in rev 42366e293b7b and noticed by Claudiu Popa. files: Lib/base64.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/base64.py b/Lib/base64.py --- a/Lib/base64.py +++ b/Lib/base64.py @@ -9,7 +9,6 @@ import re import struct import binascii -import itertools __all__ = [ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 18:03:58 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 18:03:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Add_first_version_of_my_sc?= =?utf-8?q?ript_to_automatically_publish_release_files_on?= Message-ID: <3fcW6Q54fcz7Ljd@mail.python.org> http://hg.python.org/release/rev/e2e377d936fd changeset: 77:e2e377d936fd user: Georg Brandl date: Sun Mar 02 18:04:53 2014 +0100 summary: Add first version of my script to automatically publish release files on python.org. files: add-to-pydotorg.py | 194 +++++++++++++++++++++++++++++++++ 1 files changed, 194 insertions(+), 0 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py new file mode 100644 --- /dev/null +++ b/add-to-pydotorg.py @@ -0,0 +1,194 @@ +""" +Script to add ReleaseFile objects for Python releases on the new pydotorg. + +To use: + +* Copy this script to dinsdale (it needs access to all the release files). + You could also download all files, then you need to adapt the "ftp_root" + string below. + +* Create a new Release object via the Django admin (API is currently broken), + the name MUST be "Python RELEASE". + +* Make sure all download files are in place in the /data/ftp.python.org + subdirectory. + +* Put an AUTH_INFO variable containing "username:api_key" in your environment. + +* Call this script as "python add-to-pydotorg.py RELEASE". + + Each call will remove all previous file objects, so you can call the script + multiple times. +""" + +import os +import re +import sys +import json +import hashlib +from os import path + +import requests + +try: + auth_info = os.environ['AUTH_INFO'] +except KeyError: + print 'Please set an environment variable named AUTH_INFO ' \ + 'containing "username:api_key".' + sys.exit() + +base_url = 'http://127.0.0.1:8124/api/v1/' +ftp_root = '/data/ftp.python.org/pub/python/' +download_root = 'http://www.python.org/ftp/python/' + +headers = {'Authorization': 'ApiKey %s' % auth_info, 'Content-Type': 'application/json'} + +rx = re.compile +# value is (file "name", OS id, file "description") +file_descriptions = [ + (rx(r'\.tgz$'), ('Gzipped source tarball', 3, '')), + (rx(r'\.tar\.xz$'), ('XZ compressed source tarball', 3, '')), + (rx(r'\.amd64\.msi$'), ('Windows x86-64 MSI installer', 1, + 'for AMD64/EM64T/x64, not Itanium processors')), + (rx(r'\.msi$'), ('Windows x86 MSI installer', 1, '')), + (rx(r'\.chm$'), ('Windows help file', 1, '')), + (rx(r'amd64-pdb\.zip$'), ('Windows debug information files for 64-bit binaries', 1, '')), + (rx(r'-pdb\.zip$'), ('Windows debug information files', 1, '')), + (rx(r'-macosx10\.5(_rev\d)?\.dmg$'), ('Mac OS X 32-bit i386/PPC installer', 2, + 'for Mac OS X 10.5 and later')), + (rx(r'-macosx10\.6(_rev\d)?\.dmg$'), ('Mac OS X 64-bit/32-bit installer', 2, + 'for Mac OS X 10.6 and later')), +] + +def changelog_for(release): + new_url = 'http://docs.python.org/release/%s/whatsnew/changelog.html' % release + if requests.head(new_url).status_code != 200: + return 'http://hg.python.org/cpython/file/v%s/Misc/NEWS' % release + +def slug_for(release): + return release[0] + '-' + release[2] + '-' + release[4] + \ + ('-' + release[5:] if release[5:] else '') + +def sigfile_for(release, rfile): + return download_root + '%s/%s.asc' % (release, rfile) + +def md5sum_for(release, rfile): + return hashlib.md5(open(ftp_root + release[:5] + '/' + rfile, 'rb').read()).hexdigest() + +def filesize_for(release, rfile): + return path.getsize(ftp_root + release[:5] + '/' + rfile) + +def make_slug(text): + return re.sub('[^a-zA-Z0-9_-]', '', text.replace(' ', '-')) + +def build_release_dict(release, reldate, is_latest, page_pk): + """Return a dictionary with all needed fields for a Release object.""" + return dict( + name = 'Python ' + release, + slug = slug_for(release), + version = int(release[0]), + is_published = True, + release_date = reldate, # in "YYYY-MM-ddTHH:MM:SS" + pre_release = bool(release[5:]), + release_page = '/api/v1/pages/page/%s/' % page_pk, # XXX doesn't work yet + release_notes_url = changelog_for(release), + show_on_download_page = True, + ) + +def build_file_dict(release, rfile, rel_pk, file_desc, os_pk, add_desc): + """Return a dictionary with all needed fields for a ReleaseFile object.""" + return dict( + name = file_desc, + slug = slug_for(release) + '-' + make_slug(file_desc), + os = '/api/v1/downloads/os/%s/' % os_pk, + release = '/api/v1/downloads/release/%s/' % rel_pk, + description = add_desc, + is_source = os_pk == 3, + url = download_root + '%s/%s' % (release, rfile), + gpg_signature_file = sigfile_for(release, rfile), + md5_sum = md5sum_for(release, rfile), + filesize = filesize_for(release, rfile), + download_button = 'tar.xz' in rfile or + 'macosx10.6.dmg' in rfile or + ('.msi' in rfile and not 'amd64' in rfile), + ) + +def list_files(release): + """List all of the release's download files.""" + reldir = release[:5] + for rfile in os.listdir(path.join(ftp_root, reldir)): + if not path.isfile(path.join(ftp_root, reldir, rfile)): + continue + if rfile.endswith('.asc'): + continue + for prefix in ('python', 'Python'): + if rfile.startswith(prefix): + break + else: + print ' File %s/%s has wrong prefix' % (reldir, rfile) + continue + if rfile.endswith('.chm'): + if rfile[:-4] != 'python' + release.replace('.', ''): + print ' File %s/%s has a different version' % (reldir, rfile) + continue + else: + try: + prefix, rest = rfile.split('-', 1) + except: + prefix, rest = rfile, '' + if not rest.startswith((release + '-', release + '.')): + print ' File %s/%s has a different version' % (reldir, rfile) + continue + for rx, info in file_descriptions: + if rx.search(rfile): + file_desc, os_pk, add_desc = info + yield rfile, file_desc, os_pk, add_desc + break + else: + print ' File %s/%s not recognized' % (reldir, rfile) + continue + +def query_object(objtype, **params): + """Find an API object by query parameters.""" + uri = base_url + 'downloads/%s/' % objtype + uri += '?' + '&'.join('%s=%s' % v for v in params.items()) + resp = requests.get(uri, headers=headers) + if resp.status_code != 200 or not json.loads(resp.text)['objects']: + raise RuntimeError('no object for %s params=%r' % (objtype, params)) + obj = json.loads(resp.text)['objects'][0] + return int(obj['resource_uri'].strip('/').split('/')[-1]) + +def post_object(objtype, datadict): + """Create a new API object.""" + resp = requests.post(base_url + 'downloads/' + objtype + '/', + data=json.dumps(datadict), headers=headers) + if resp.status_code != 201: + try: + print resp.json['traceback'] + except: + pass + raise RuntimeError('creating %s failed: %s' % (objtype, resp.status_code)) + newloc = resp.headers['Location'] + pk = int(newloc.strip('/').split('/')[-1]) + return pk + +def main(): + rel = sys.argv[1] + print 'Querying python.org for release', rel + rel_pk = query_object('release', name='Python+' + rel) + print 'Found Release object: id =', rel_pk + n = 0 + print 'Deleting previous release files' + resp = requests.delete(base_url + 'downloads/release_file/?release=%s' % rel_pk, + headers=headers) + if resp.status_code != 204: + raise RuntimeError('deleting previous releases failed: %s' % resp.status_code) + for rfile, file_desc, os_pk, add_desc in list_files(rel): + print 'Creating ReleaseFile object for', rfile + file_dict = build_file_dict(rel, rfile, rel_pk, file_desc, os_pk, add_desc) + file_pk = post_object('release_file', file_dict) + print 'Created as id =', file_pk + n += 1 + print 'Done - %d files added' % n + +main() -- Repository URL: http://hg.python.org/release From python-checkins at python.org Sun Mar 2 18:25:38 2014 From: python-checkins at python.org (yury.selivanov) Date: Sun, 2 Mar 2014 18:25:38 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320786=3A_Fix_sign?= =?utf-8?q?atures_for_dict=2E=5F=5Fdelitem=5F=5F_and_property=2E=5F=5Fdele?= =?utf-8?q?te=5F=5F?= Message-ID: <3fcWbQ0T48z7LjX@mail.python.org> http://hg.python.org/cpython/rev/c9861ec8754c changeset: 89461:c9861ec8754c user: Yury Selivanov date: Sun Mar 02 12:25:27 2014 -0500 summary: Issue #20786: Fix signatures for dict.__delitem__ and property.__delete__ files: Lib/test/test_inspect.py | 5 +++++ Misc/NEWS | 3 +++ Objects/typeobject.c | 4 ++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -1764,6 +1764,11 @@ __call__ = type test_callable(ThisWorksNow()) + # Regression test for issue #20786 + test_unbound_method(dict.__delitem__) + test_unbound_method(property.__delete__) + + @cpython_only @unittest.skipIf(MISSING_C_DOCSTRINGS, "Signature information for builtins requires docstrings") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ - Issue #20637: Key-sharing now also works for instance dictionaries of subclasses. Patch by Peter Ingebretson. +- Issue #20786: Fix signatures for dict.__delitem__ and + property.__delete__ builtins. + Library ------- diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6192,7 +6192,7 @@ "__set__($self, instance, value, /)\n--\n\nSet an attribute of instance to value."), TPSLOT("__delete__", tp_descr_set, slot_tp_descr_set, wrap_descr_delete, - "__delete__(instance, /)\n--\n\nDelete an attribute of instance."), + "__delete__($self, instance, /)\n--\n\nDelete an attribute of instance."), FLSLOT("__init__", tp_init, slot_tp_init, (wrapperfunc)wrap_init, "__init__($self, /, *args, **kwargs)\n--\n\n" "Initialize self. See help(type(self)) for accurate signature.", @@ -6289,7 +6289,7 @@ "__setitem__($self, key, value, /)\n--\n\nSet self[key] to value."), MPSLOT("__delitem__", mp_ass_subscript, slot_mp_ass_subscript, wrap_delitem, - "__delitem__(key)\n--\n\nDelete self[key]."), + "__delitem__($self, key, /)\n--\n\nDelete self[key]."), SQSLOT("__len__", sq_length, slot_sq_length, wrap_lenfunc, "__len__($self, /)\n--\n\nReturn len(self)."), -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 18:29:37 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 2 Mar 2014 18:29:37 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Make_PEP_8_follow_PEP_257_on_?= =?utf-8?q?docstring_endings=2E?= Message-ID: <3fcWh10z3Kz7Lk6@mail.python.org> http://hg.python.org/peps/rev/380301e300a6 changeset: 5395:380301e300a6 user: Guido van Rossum date: Sun Mar 02 09:29:33 2014 -0800 summary: Make PEP 8 follow PEP 257 on docstring endings. files: pep-0008.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -545,14 +545,14 @@ - PEP 257 describes good docstring conventions. Note that most importantly, the ``"""`` that ends a multiline docstring should be - on a line by itself, and preferably preceded by a blank line, e.g.:: + on a line by itself, e.g.:: """Return a foobang Optional plotz says to frobnicate the bizbaz first. """ -- For one liner docstrings, it's okay to keep the closing ``"""`` on +- For one liner docstrings, please keep the closing ``"""`` on the same line. -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 2 19:12:30 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 2 Mar 2014 19:12:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_a_few_fixes?= Message-ID: <3fcXdV0G66z7Ljj@mail.python.org> http://hg.python.org/release/rev/14aa9229450b changeset: 78:14aa9229450b user: Georg Brandl date: Sun Mar 02 19:13:23 2014 +0100 summary: a few fixes files: add-to-pydotorg.py | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py --- a/add-to-pydotorg.py +++ b/add-to-pydotorg.py @@ -1,17 +1,16 @@ """ Script to add ReleaseFile objects for Python releases on the new pydotorg. - -To use: +To use (RELEASE is something like 3.3.5rc2): * Copy this script to dinsdale (it needs access to all the release files). You could also download all files, then you need to adapt the "ftp_root" string below. -* Create a new Release object via the Django admin (API is currently broken), - the name MUST be "Python RELEASE". +* Make sure all download files are in place in the correct /data/ftp.python.org + subdirectory. -* Make sure all download files are in place in the /data/ftp.python.org - subdirectory. +* Create a new Release object via the Django admin (adding via API is + currently broken), the name MUST be "Python RELEASE". * Put an AUTH_INFO variable containing "username:api_key" in your environment. @@ -19,6 +18,8 @@ Each call will remove all previous file objects, so you can call the script multiple times. + +Georg Brandl, March 2014. """ import os @@ -37,7 +38,7 @@ 'containing "username:api_key".' sys.exit() -base_url = 'http://127.0.0.1:8124/api/v1/' +base_url = 'http://www.python.org/api/v1/' ftp_root = '/data/ftp.python.org/pub/python/' download_root = 'http://www.python.org/ftp/python/' -- Repository URL: http://hg.python.org/release From python-checkins at python.org Sun Mar 2 19:16:35 2014 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 2 Mar 2014 19:16:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320568=3A_Fix_typo?= =?utf-8?q?_in_pip_option=2E?= Message-ID: <3fcXkC3XgMz7LjX@mail.python.org> http://hg.python.org/cpython/rev/16f91d87ff39 changeset: 89462:16f91d87ff39 user: Martin v. L?wis date: Sun Mar 02 19:15:47 2014 +0100 summary: Issue #20568: Fix typo in pip option. files: Tools/msi/msi.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -420,7 +420,7 @@ compileargs = r'-Wi "[TARGETDIR]Lib\compileall.py" -f -x "bad_coding|badsyntax|site-packages|py2_|lib2to3\\tests|venv\\scripts" "[TARGETDIR]Lib"' lib2to3args = r'-c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"' - updatepipargs = r'-m ensurepip -U --default-install' + updatepipargs = r'-m ensurepip -U --default-pip' removepipargs = r'-m ensurepip._uninstall' # See "CustomAction Table" add_data(db, "CustomAction", [ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 19:29:59 2014 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 2 Mar 2014 19:29:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A__Issue_=2320748=3A_Uninsta?= =?utf-8?q?lling_pip_does_not_leave_behind_the_pyc_of?= Message-ID: <3fcY1g1h4Vz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/7b3c40510a08 changeset: 89463:7b3c40510a08 user: Martin v. L?wis date: Sun Mar 02 19:29:19 2014 +0100 summary: Issue #20748: Uninstalling pip does not leave behind the pyc of the uninstaller anymore. files: Misc/NEWS | 3 +++ Tools/msi/msi.py | 2 +- 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,9 @@ Build ----- +- Issue #20748: Uninstalling pip does not leave behind the pyc of + the uninstaller anymore. + - Issue #20568: The Windows installer now installs the unversioned ``pip`` command in addition to the versioned ``pip3`` and ``pip3.4`` commands. diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -421,7 +421,7 @@ compileargs = r'-Wi "[TARGETDIR]Lib\compileall.py" -f -x "bad_coding|badsyntax|site-packages|py2_|lib2to3\\tests|venv\\scripts" "[TARGETDIR]Lib"' lib2to3args = r'-c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"' updatepipargs = r'-m ensurepip -U --default-pip' - removepipargs = r'-m ensurepip._uninstall' + removepipargs = r'-B -m ensurepip._uninstall' # See "CustomAction Table" add_data(db, "CustomAction", [ # msidbCustomActionTypeFirstSequence + msidbCustomActionTypeTextData + msidbCustomActionTypeProperty -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 19:43:21 2014 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 2 Mar 2014 19:43:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320465=3A_Update_W?= =?utf-8?q?indows_installer_to_SQLite_3=2E8=2E3=2E1=2E?= Message-ID: <3fcYK552Rfz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/9459f517d854 changeset: 89464:9459f517d854 user: Martin v. L?wis date: Sun Mar 02 19:42:50 2014 +0100 summary: Issue #20465: Update Windows installer to SQLite 3.8.3.1. files: Misc/NEWS | 3 ++- PC/VS9.0/pyproject.vsprops | 2 +- PCbuild/pyproject.props | 2 +- PCbuild/readme.txt | 2 +- Tools/buildbot/external-common.bat | 6 +++--- 5 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -52,7 +52,8 @@ uninstalling pip (rather than failing) if the user has updated pip to a different version from the one bundled with ensurepip. -- Issue #20465: Update OS X installer build to use SQLite 3.8.3.1. +- Issue #20465: Update OS X and Windows installer builds to use + SQLite 3.8.3.1. Tools/Demos ----------- diff --git a/PC/VS9.0/pyproject.vsprops b/PC/VS9.0/pyproject.vsprops --- a/PC/VS9.0/pyproject.vsprops +++ b/PC/VS9.0/pyproject.vsprops @@ -50,7 +50,7 @@ /> $(OutDir)python$(PyDebugExt).exe $(OutDir)kill_python$(PyDebugExt).exe ..\.. - $(externalsDir)\sqlite-3.8.1 + $(externalsDir)\sqlite-3.8.3.1 $(externalsDir)\bzip2-1.0.6 $(externalsDir)\xz-5.0.5 $(externalsDir)\openssl-1.0.1e diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -206,7 +206,7 @@ The ssl sub-project does not have the ability to clean the OpenSSL build; if you need to rebuild, you'll have to clean it by hand. _sqlite3 - Wraps SQLite 3.8.1, which is itself built by sqlite3.vcxproj + Wraps SQLite 3.8.3.1, which is itself built by sqlite3.vcxproj Homepage: http://www.sqlite.org/ _tkinter diff --git a/Tools/buildbot/external-common.bat b/Tools/buildbot/external-common.bat --- a/Tools/buildbot/external-common.bat +++ b/Tools/buildbot/external-common.bat @@ -37,9 +37,9 @@ if not exist tk-8.6.1.0 svn export http://svn.python.org/projects/external/tk-8.6.1.0 @rem sqlite3 -if not exist sqlite-3.8.1 ( - rd /s/q sqlite-source-3.7.12 - svn export http://svn.python.org/projects/external/sqlite-3.8.1 +if not exist sqlite-3.8.3.1 ( + rd /s/q sqlite-source-3.8.1 + svn export http://svn.python.org/projects/external/sqlite-3.8.3.1 ) @rem lzma -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 20:29:50 2014 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 2 Mar 2014 20:29:50 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2314512=3A_Launch_p?= =?utf-8?q?ydoc_-b_instead_of_pydocgui=2Epyw_on_Windows=2E?= Message-ID: <3fcZLk4wmnz7LkL@mail.python.org> http://hg.python.org/cpython/rev/de81e0fe4905 changeset: 89465:de81e0fe4905 user: Martin v. L?wis date: Sun Mar 02 20:29:18 2014 +0100 summary: Issue #14512: Launch pydoc -b instead of pydocgui.pyw on Windows. files: Misc/NEWS | 2 ++ Tools/msi/msi.py | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,8 @@ Build ----- +- Issue #14512: Launch pydoc -b instead of pydocgui.pyw on Windows. + - Issue #20748: Uninstalling pip does not leave behind the pyc of the uninstaller anymore. diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -1324,8 +1324,6 @@ tcltkshortcuts = [ ("IDLE", "MenuDir", "IDLE|IDLE (Python GUI)", "pythonw.exe", tcltk.id, r'"[TARGETDIR]Lib\idlelib\idle.pyw"', None, None, "python_icon.exe", 0, None, "TARGETDIR"), - ("PyDoc", "MenuDir", "MODDOCS|Module Docs", "pythonw.exe", - tcltk.id, r'"[TARGETDIR]Tools\scripts\pydocgui.pyw"', None, None, "python_icon.exe", 0, None, "TARGETDIR"), ] add_data(db, "Shortcut", tcltkshortcuts + @@ -1340,6 +1338,8 @@ ("Manual", "MenuDir", "MANUAL|Python Manuals", "REGISTRY.doc", "[#%s]" % docfile, None, None, None, None, None, None, None), + ("PyDoc", "MenuDir", "MODDOCS|Module Docs", "python.exe", + default_feature.id, r'-m pydoc -b', None, None, "python_icon.exe", 0, None, "TARGETDIR"), ("Uninstall", "MenuDir", "UNINST|Uninstall Python", "REGISTRY", SystemFolderName+"msiexec", "/x%s" % product_code, None, None, None, None, None, None), -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 2 23:42:07 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 2 Mar 2014 23:42:07 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_add_some_issue?= =?utf-8?q?_number_annotations=2E?= Message-ID: <3fcfcb5bH9z7LjZ@mail.python.org> http://hg.python.org/cpython/rev/fceec0368cfa changeset: 89466:fceec0368cfa user: R David Murray date: Sun Mar 02 17:14:33 2014 -0500 summary: whatsnew: add some issue number annotations. Add issue number to tick_counter, PyOS_ReadlineFunctionPointer, marshal optimizations (add Antoine's contribution to the entry under the marshal module), and absolute paths in __file__. files: Doc/whatsnew/3.4.rst | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -926,7 +926,8 @@ this "one copy" ability to any object type (including handling recursive references). This reduces both the size of ``.pyc`` files and the amount of memory a module occupies in memory when it is loaded from a ``.pyc`` (or -``.pyo``) file. (Contributed by Kristj?n Valur J?nsson in :issue:`16475`.) +``.pyo``) file. (Contributed by Kristj?n Valur J?nsson in :issue:`16475`, +with additional speedups by Antoine Pitrou in :issue:`19219`.) mmap @@ -1782,7 +1783,8 @@ ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`). * The ``PyThreadState.tick_counter`` field has been removed; its value has - been meaningless since Python 3.2, when the "new GIL" was introduced. + been meaningless since Python 3.2, when the "new GIL" was introduced + (:issue:`19199`). * ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`. (Contributed by Taras Lyapun in :issue:`15641`.) @@ -1912,7 +1914,7 @@ will not find all keys. A module's ``__file__`` when imported in the current working directory will also now have an absolute path, including when using ``-m`` with the interpreter (this does not influence when the path to a file - is specified on the command-line). + is specified on the command-line) (:issue:`18416`). * The removal of the *strict* argument to :class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` changes the meaning of the @@ -2032,7 +2034,8 @@ * The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must now be a string allocated by :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a - string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. + string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc` + (:issue:`16742`) * :c:func:`PyThread_set_key_value` now always set the value. In Python 3.3, the function did nothing if the key already exists (if the current -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Mon Mar 3 10:01:23 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 03 Mar 2014 10:01:23 +0100 Subject: [Python-checkins] Daily reference leaks (fceec0368cfa): sum=0 Message-ID: results for fceec0368cfa on branch "default" -------------------------------------------- test_site leaked [-2, 0, 2] references, sum=0 test_site leaked [-2, 0, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog7IoOXy', '-x'] From python-checkins at python.org Mon Mar 3 10:51:20 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 3 Mar 2014 10:51:20 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_3=2E3=2E5_schedule_update=2E?= Message-ID: <3fcxSm1GHXz7Ll4@mail.python.org> http://hg.python.org/peps/rev/a7ccffe9cd9e changeset: 5396:a7ccffe9cd9e user: Georg Brandl date: Mon Mar 03 10:52:18 2014 +0100 summary: 3.3.5 schedule update. files: pep-0398.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0398.txt b/pep-0398.txt --- a/pep-0398.txt +++ b/pep-0398.txt @@ -95,7 +95,8 @@ -------------- - 3.3.5 candidate 1: February 22, 2014 -- 3.3.5 final: March 1, 2014 +- 3.3.5 candidate 2: March 1, 2014 +- 3.3.5 final: March 8, 2014 3.3.6 schedule -------------- -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Mar 3 11:22:08 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 3 Mar 2014 11:22:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Fix_errors_in_the_download?= =?utf-8?q?_file_URLs=2E?= Message-ID: <3fcy8J6sWTz7LjM@mail.python.org> http://hg.python.org/release/rev/3688da9b5212 changeset: 79:3688da9b5212 user: Georg Brandl date: Mon Mar 03 11:23:06 2014 +0100 summary: Fix errors in the download file URLs. files: add-to-pydotorg.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py --- a/add-to-pydotorg.py +++ b/add-to-pydotorg.py @@ -71,7 +71,7 @@ ('-' + release[5:] if release[5:] else '') def sigfile_for(release, rfile): - return download_root + '%s/%s.asc' % (release, rfile) + return download_root + '%s/%s.asc' % (release[:5], rfile) def md5sum_for(release, rfile): return hashlib.md5(open(ftp_root + release[:5] + '/' + rfile, 'rb').read()).hexdigest() @@ -105,7 +105,7 @@ release = '/api/v1/downloads/release/%s/' % rel_pk, description = add_desc, is_source = os_pk == 3, - url = download_root + '%s/%s' % (release, rfile), + url = download_root + '%s/%s' % (release[:5], rfile), gpg_signature_file = sigfile_for(release, rfile), md5_sum = md5sum_for(release, rfile), filesize = filesize_for(release, rfile), -- Repository URL: http://hg.python.org/release From python-checkins at python.org Mon Mar 3 11:58:52 2014 From: python-checkins at python.org (victor.stinner) Date: Mon, 3 Mar 2014 11:58:52 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2320814=3A_doc=3A_F?= =?utf-8?q?ix_=22Pretty_top=22_example_of_tracemalloc?= Message-ID: <3fcyyh6v46z7LjQ@mail.python.org> http://hg.python.org/cpython/rev/a9058b772807 changeset: 89467:a9058b772807 user: Victor Stinner date: Mon Mar 03 11:57:57 2014 +0100 summary: Close #20814: doc: Fix "Pretty top" example of tracemalloc files: Doc/library/tracemalloc.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/library/tracemalloc.rst b/Doc/library/tracemalloc.rst --- a/Doc/library/tracemalloc.rst +++ b/Doc/library/tracemalloc.rst @@ -200,8 +200,7 @@ # replace "/path/to/module/file.py" with "module/file.py" filename = os.sep.join(frame.filename.split(os.sep)[-2:]) print("#%s: %s:%s: %.1f KiB" - % (index, filename, frame.lineno, - stat.size / 1024)) + % (index, filename, frame.lineno, stat.size / 1024)) other = top_stats[limit:] if other: @@ -215,7 +214,7 @@ # ... run your application ... snapshot = tracemalloc.take_snapshot() - display_top(snapshot, 10) + display_top(snapshot) Example of output of the Python test suite:: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 3 20:21:11 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 3 Mar 2014 20:21:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Correct_and_im?= =?utf-8?q?prove_comments_in_test=5Ffileinput_=28closes_=2320501=29=2E?= Message-ID: <3fdB6H68C6z7Ljy@mail.python.org> http://hg.python.org/cpython/rev/d37f963394aa changeset: 89468:d37f963394aa branch: 2.7 parent: 89449:893c9e441cd7 user: Serhiy Storchaka date: Mon Mar 03 21:16:27 2014 +0200 summary: Correct and improve comments in test_fileinput (closes #20501). Thanks Vajrasky Kok and Zachary Ware. files: Lib/test/test_fileinput.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -228,6 +228,8 @@ self.addCleanup(safe_unlink, TESTFN) fi = FileInput(files=TESTFN, openhook=hook_encoded('ascii'), bufsize=8) + # The most likely failure is a UnicodeDecodeError due to the entire + # file being read when it shouldn't have been. self.assertEqual(fi.readline(), u'A\n') self.assertEqual(fi.readline(), u'B\r\n') self.assertEqual(fi.readline(), u'C\r') @@ -240,11 +242,9 @@ """Unit tests for fileinput.hook_encoded()""" def test_modes(self): - # Unlikely UTF-7 is locale encoding with open(TESTFN, 'wb') as f: + # UTF-7 is a convenient, seldom used encoding f.write('A\nB\r\nC\rD+IKw-') - t1 = TESTFN - #t1 = writeTmp(1, ['A\nB\r\nC\rD+IKw-'], mode='wb') self.addCleanup(safe_unlink, TESTFN) def check(mode, expected_lines): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 3 20:21:13 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 3 Mar 2014 20:21:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Correct_commen?= =?utf-8?q?ts_and_improve_failure_reports_in_test=5Ffileinput_=28closes_?= =?utf-8?b?IzIwNTAxKS4=?= Message-ID: <3fdB6K0v0Nz7Lln@mail.python.org> http://hg.python.org/cpython/rev/204ef3bca9c8 changeset: 89469:204ef3bca9c8 branch: 3.3 parent: 89458:ca5635efe090 user: Serhiy Storchaka date: Mon Mar 03 21:17:17 2014 +0200 summary: Correct comments and improve failure reports in test_fileinput (closes #20501). Thanks Vajrasky Kok and Zachary Ware. files: Lib/test/test_fileinput.py | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -269,9 +269,12 @@ with FileInput(files=TESTFN, openhook=hook_encoded('ascii'), bufsize=8) as fi: - self.assertEqual(fi.readline(), 'A\n') - self.assertEqual(fi.readline(), 'B\n') - self.assertEqual(fi.readline(), 'C\n') + try: + self.assertEqual(fi.readline(), 'A\n') + self.assertEqual(fi.readline(), 'B\n') + self.assertEqual(fi.readline(), 'C\n') + except UnicodeDecodeError: + self.fail('Read to end of file') with self.assertRaises(UnicodeDecodeError): # Read to the end of file. list(fi) @@ -854,8 +857,8 @@ self.assertFalse(kwargs) def test_modes(self): - # Unlikely UTF-7 is locale encoding with open(TESTFN, 'wb') as f: + # UTF-7 is a convenient, seldom used encoding f.write(b'A\nB\r\nC\rD+IKw-') self.addCleanup(safe_unlink, TESTFN) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 3 20:21:14 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 3 Mar 2014 20:21:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Correct_comments_and_improve_failure_reports_in_test=5Ff?= =?utf-8?q?ileinput_=28closes_=2320501=29=2E?= Message-ID: <3fdB6L2mVMz7LkH@mail.python.org> http://hg.python.org/cpython/rev/c47cc6351ce7 changeset: 89470:c47cc6351ce7 parent: 89467:a9058b772807 parent: 89469:204ef3bca9c8 user: Serhiy Storchaka date: Mon Mar 03 21:19:19 2014 +0200 summary: Correct comments and improve failure reports in test_fileinput (closes #20501). Thanks Vajrasky Kok and Zachary Ware. files: Lib/test/test_fileinput.py | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -271,9 +271,12 @@ with FileInput(files=TESTFN, openhook=hook_encoded('ascii'), bufsize=8) as fi: - self.assertEqual(fi.readline(), 'A\n') - self.assertEqual(fi.readline(), 'B\n') - self.assertEqual(fi.readline(), 'C\n') + try: + self.assertEqual(fi.readline(), 'A\n') + self.assertEqual(fi.readline(), 'B\n') + self.assertEqual(fi.readline(), 'C\n') + except UnicodeDecodeError: + self.fail('Read to end of file') with self.assertRaises(UnicodeDecodeError): # Read to the end of file. list(fi) @@ -856,8 +859,8 @@ self.assertFalse(kwargs) def test_modes(self): - # Unlikely UTF-7 is locale encoding with open(TESTFN, 'wb') as f: + # UTF-7 is a convenient, seldom used encoding f.write(b'A\nB\r\nC\rD+IKw-') self.addCleanup(safe_unlink, TESTFN) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 3 21:43:08 2014 From: python-checkins at python.org (ethan.furman) Date: Mon, 3 Mar 2014 21:43:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_issue20653=3A_improv?= =?utf-8?q?e_functional_API_docs=3B_minor_code_changes?= Message-ID: <3fdCwr61JYz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/b637064cc696 changeset: 89471:b637064cc696 user: Ethan Furman date: Mon Mar 03 12:42:52 2014 -0800 summary: Close issue20653: improve functional API docs; minor code changes files: Doc/library/enum.rst | 34 +++++++++++++++++++++++++++ Lib/enum.py | 39 ++++++++++++++++++++++--------- 2 files changed, 61 insertions(+), 12 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -374,6 +374,9 @@ With pickle protocol version 4 it is possible to easily pickle enums nested in other classes. +It is possible to modify how Enum members are pickled/unpickled by defining +:meth:`__reduce_ex__` in the enumeration class. + Functional API -------------- @@ -420,6 +423,12 @@ >>> Animals = Enum('Animals', 'ant bee cat dog', module=__name__) +.. warning:: + + If :param module: is not supplied, and Enum cannot determine what it is, + the new Enum members will not be unpicklable; to keep errors closer to + the source, pickling will be disabled. + The new pickle protocol 4 also, in some circumstances, relies on :attr:`__qualname__` being set to the location where pickle will be able to find the class. For example, if the class was made available in class @@ -427,6 +436,31 @@ >>> Animals = Enum('Animals', 'ant bee cat dog', qualname='SomeData.Animals') +The complete signature is:: + + Enum(value='NewEnumName', names=<...>, *, module='...', qualname='...', type=) + +:param value: What the new Enum class will record as its name. + +:param names: The Enum members. This can be a whitespace or comma seperated +string:: + + 'red green blue', 'red,green,blue', 'red, green, blue' + +(values will start at 1), or an iterator of name, value pairs:: + + [('cyan', 4), ('magenta', 5), ('yellow', 6)] + +or a mapping:: + + {'chartruese': 7, 'sea_green': 11, 'rosemary': 42} + +:param module: name of module where new Enum class can be found. + +:param qualname: where in module new Enum class can be found. + +:param type: type to mix in to new Enum class. + Derived Enumerations -------------------- diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -115,14 +115,21 @@ # Reverse value->name map for hashable values. enum_class._value2member_map_ = {} - # check for a supported pickle protocols, and if not present sabotage - # pickling, since it won't work anyway. - # if new class implements its own __reduce_ex__, do not sabotage - if classdict.get('__reduce_ex__') is None: + # If a custom type is mixed into the Enum, and it does not know how + # to pickle itself, pickle.dumps will succeed but pickle.loads will + # fail. Rather than have the error show up later and possibly far + # from the source, sabotage the pickle protocol for this class so + # that pickle.dumps also fails. + # + # However, if the new class implements its own __reduce_ex__, do not + # sabotage -- it's on them to make sure it works correctly. We use + # __reduce_ex__ instead of any of the others as it is preferred by + # pickle over __reduce__, and it handles all pickle protocols. + if '__reduce_ex__' not in classdict: if member_type is not object: methods = ('__getnewargs_ex__', '__getnewargs__', '__reduce_ex__', '__reduce__') - if not any(map(member_type.__dict__.get, methods)): + if not any(m in member_type.__dict__ for m in methods): _make_class_unpicklable(enum_class) # instantiate them, checking for duplicates as we go @@ -193,14 +200,22 @@ to an enumeration member (i.e. Color(3)) and for the functional API (i.e. Color = Enum('Color', names='red green blue')). - When used for the functional API: `module`, if set, will be stored in - the new class' __module__ attribute; `qualname`, if set, will be stored - in the new class' __qualname__ attribute; `type`, if set, will be mixed - in as the first base class. + When used for the functional API: - Note: if `module` is not set this routine will attempt to discover the - calling module by walking the frame stack; if this is unsuccessful - the resulting class will not be pickleable. + `value` will be the name of the new class. + + `names` should be either a string of white-space/comma delimited names + (values will start at 1), or an iterator/mapping of name, value pairs. + + `module` should be set to the module this class is being created in; + if it is not set, an attempt to find that module will be made, but if + it fails the class will not be picklable. + + `qualname` should be set to the actual location this class can be found + at in its module; by default it is set to the global scope. If this is + not correct, unpickling will fail in some circumstances. + + `type`, if set, will be mixed in as the first base class. """ if names is None: # simple value lookup -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 4 00:02:05 2014 From: python-checkins at python.org (ethan.furman) Date: Tue, 4 Mar 2014 00:02:05 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue20653=3A_fix_ReST_for?= =?utf-8?q?_Enum?= Message-ID: <3fdH194TlXz7Lr8@mail.python.org> http://hg.python.org/cpython/rev/54ab95407288 changeset: 89472:54ab95407288 user: Ethan Furman date: Mon Mar 03 15:02:04 2014 -0800 summary: Issue20653: fix ReST for Enum files: Doc/library/enum.rst | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -425,7 +425,7 @@ .. warning:: - If :param module: is not supplied, and Enum cannot determine what it is, + If ``module`` is not supplied, and Enum cannot determine what it is, the new Enum members will not be unpicklable; to keep errors closer to the source, pickling will be disabled. @@ -440,26 +440,26 @@ Enum(value='NewEnumName', names=<...>, *, module='...', qualname='...', type=) -:param value: What the new Enum class will record as its name. +:value: What the new Enum class will record as its name. -:param names: The Enum members. This can be a whitespace or comma seperated -string:: +:names: The Enum members. This can be a whitespace or comma seperated string + (values will start at 1):: - 'red green blue', 'red,green,blue', 'red, green, blue' + 'red green blue' | 'red,green,blue' | 'red, green, blue' -(values will start at 1), or an iterator of name, value pairs:: + or an iterator of (name, value) pairs:: [('cyan', 4), ('magenta', 5), ('yellow', 6)] -or a mapping:: + or a mapping:: {'chartruese': 7, 'sea_green': 11, 'rosemary': 42} -:param module: name of module where new Enum class can be found. +:module: name of module where new Enum class can be found. -:param qualname: where in module new Enum class can be found. +:qualname: where in module new Enum class can be found. -:param type: type to mix in to new Enum class. +:type: type to mix in to new Enum class. Derived Enumerations -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Mar 4 09:51:58 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 04 Mar 2014 09:51:58 +0100 Subject: [Python-checkins] Daily reference leaks (54ab95407288): sum=0 Message-ID: results for 54ab95407288 on branch "default" -------------------------------------------- test_site leaked [0, -2, 2] references, sum=0 test_site leaked [0, -2, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogINlA7F', '-x'] From python-checkins at python.org Tue Mar 4 11:39:56 2014 From: python-checkins at python.org (nick.coghlan) Date: Tue, 4 Mar 2014 11:39:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2320839=3A_pkgutil?= =?utf-8?q?=2Efind=5Floader_now_uses_importlib=2Eutil=2Efind=5Fspec?= Message-ID: <3fdZVN6xh3z7LrG@mail.python.org> http://hg.python.org/cpython/rev/ea827c809765 changeset: 89473:ea827c809765 user: Nick Coghlan date: Tue Mar 04 20:39:42 2014 +1000 summary: Close #20839: pkgutil.find_loader now uses importlib.util.find_spec files: Doc/library/pkgutil.rst | 17 +++++++++-------- Lib/pkgutil.py | 17 +++++------------ Lib/test/test_pkgutil.py | 19 +++++++++++++++++++ Misc/NEWS | 4 ++++ 4 files changed, 37 insertions(+), 20 deletions(-) diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -74,15 +74,17 @@ Retrieve a :pep:`302` module loader for the given *fullname*. - This is a convenience wrapper around :func:`importlib.find_loader` that - sets the *path* argument correctly when searching for submodules, and - also ensures parent packages (if any) are imported before searching for - submodules. + This is a backwards compatibility wrapper around + :func:`importlib.util.find_spec` that converts most failures to + :exc:`ImportError` and only returns the loader rather than the full + :class:`ModuleSpec`. .. versionchanged:: 3.3 Updated to be based directly on :mod:`importlib` rather than relying on the package internal PEP 302 import emulation. + .. versionchanged:: 3.4 + Updated to be based on :pep:`451` .. function:: get_importer(path_item) @@ -109,14 +111,13 @@ not already imported, its containing package (if any) is imported, in order to establish the package ``__path__``. - This function uses :func:`iter_importers`, and is thus subject to the same - limitations regarding platform-specific special import locations such as the - Windows registry. - .. versionchanged:: 3.3 Updated to be based directly on :mod:`importlib` rather than relying on the package internal PEP 302 import emulation. + .. versionchanged:: 3.4 + Updated to be based on :pep:`451` + .. function:: iter_importers(fullname='') diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -470,29 +470,22 @@ def find_loader(fullname): """Find a PEP 302 "loader" object for fullname - This is s convenience wrapper around :func:`importlib.find_loader` that - sets the *path* argument correctly when searching for submodules, and - also ensures parent packages (if any) are imported before searching for - submodules. + This is a backwards compatibility wrapper around + importlib.util.find_spec that converts most failures to ImportError + and only returns the loader rather than the full spec """ if fullname.startswith('.'): msg = "Relative module name {!r} not supported".format(fullname) raise ImportError(msg) - path = None - pkg_name = fullname.rpartition(".")[0] - if pkg_name: - pkg = importlib.import_module(pkg_name) - path = getattr(pkg, "__path__", None) - if path is None: - return None try: - return importlib.find_loader(fullname, path) + spec = importlib.util.find_spec(fullname) except (ImportError, AttributeError, TypeError, ValueError) as ex: # This hack fixes an impedance mismatch between pkgutil and # importlib, where the latter raises other errors for cases where # pkgutil previously raised ImportError msg = "Error while finding loader for {!r} ({}: {})" raise ImportError(msg.format(fullname, type(ex), ex)) from ex + return spec.loader def extend_path(path, name): diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -334,6 +334,25 @@ self.assertIsNotNone(pkgutil.get_loader("test.support")) self.assertEqual(len(w.warnings), 0) + def test_get_loader_handles_missing_loader_attribute(self): + global __loader__ + this_loader = __loader__ + del __loader__ + try: + with check_warnings() as w: + self.assertIsNotNone(pkgutil.get_loader(__name__)) + self.assertEqual(len(w.warnings), 0) + finally: + __loader__ = this_loader + + + def test_find_loader_avoids_emulation(self): + with check_warnings() as w: + self.assertIsNotNone(pkgutil.find_loader("sys")) + self.assertIsNotNone(pkgutil.find_loader("os")) + self.assertIsNotNone(pkgutil.find_loader("test.support")) + self.assertEqual(len(w.warnings), 0) + def test_get_importer_avoids_emulation(self): # We use an illegal path so *none* of the path hooks should fire with check_warnings() as w: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,10 @@ Library ------- +- Issue #20839: Don't trigger a DeprecationWarning in the still supported + pkgutil.get_loader() API when __loader__ isn't set on a module (nor + when pkgutil.find_loader() is called directly). + - Issue #20778: Fix modulefinder to work with bytecode-only modules. - Issue #20791: copy.copy() now doesn't make a copy when the input is -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 4 11:50:40 2014 From: python-checkins at python.org (nick.coghlan) Date: Tue, 4 Mar 2014 11:50:40 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_464=3A_drop_the_legacy_mi?= =?utf-8?q?rror_checking_API?= Message-ID: <3fdZkm4SFYz7LnS@mail.python.org> http://hg.python.org/peps/rev/8daf4ec34acb changeset: 5397:8daf4ec34acb user: Nick Coghlan date: Tue Mar 04 20:50:03 2014 +1000 summary: PEP 464: drop the legacy mirror checking API files: pep-0464.txt | 85 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 85 insertions(+), 0 deletions(-) diff --git a/pep-0464.txt b/pep-0464.txt new file mode 100644 --- /dev/null +++ b/pep-0464.txt @@ -0,0 +1,85 @@ +PEP: 464 +Title: Removal of the PyPI Mirror Authenticity API +Version: $Revision$ +Last-Modified: $Date$ +Author: Donald Stufft +BDFL-Delegate: Richard Jones +Discussions-To: distutils-sig at python.org +Status: Draft +Type: Process +Content-Type: text/x-rst +Created: 02-Mar-2014 +Post-History: 03-Mar-2014 +Replaces: 381 + + +Abstract +======== + +This PEP proposes the deprecation and removal of the PyPI Mirror Authenticity +API, this includes the /serverkey URL and all of the URLs under /serversig. + + +Rationale +========= + +The PyPI mirroring infrastructure (defined in PEP 381) provides a means to +mirror the content of PyPI used by the automatic installers, and as a component +of that, it provides a method for verifying the authenticity of the mirrored +content. + +This PEP proposal the removal of this API due to: + +* No known implementations that utilize this API are known, this includes + `pip `_ and + `setuptools `_. +* Because this API uses DSA it is vulnerable to leaking the private key if + there is *any* bias in the random nonce. +* This API solves one small corner of the trust problem, however the problem + itself is much larger and it would be better to have a fully fledged system, + such as `The Update Framework `_, + instead. + +Due to the issues it has and the lack of use it is the opinion of this PEP +that it does not provide any practical benefit to justify the additional +complexity. + + +Plan for Deprecation & Removal +============================== + +Immediately upon the acceptance of this PEP the Mirror Authenticity API will +be considered deprecated and mirroring agents and installation tools should +stop accessing it. + +Instead of actually removing it from the current code base (PyPI 1.0) the +current work to replace PyPI 1.0 with a new code base (PyPI 2.0) will simply +not implement this API. This would cause the API to be "removed" when the +switch from 1.0 to 2.0 occurs. + +If PyPI 2.0 has not been deployed in place of PyPI 1.0 by Sept 01 2014 then +this PEP will be implemented in the PyPI 1.0 code base instead (by removing +the associated code). + +No changes will be required in the installers, however PEP 381 compliant +mirroring clients, such as +`bandersnatch `_ and +`pep381client `_ will need to be +updated to no longer attempt to mirror the /serversig URLs. + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Mar 4 12:15:12 2014 From: python-checkins at python.org (nick.coghlan) Date: Tue, 4 Mar 2014 12:15:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Workflow_automation_proposal_?= =?utf-8?q?updates?= Message-ID: <3fdbH46KPRz7Lrw@mail.python.org> http://hg.python.org/peps/rev/bb1aa7f8efdc changeset: 5398:bb1aa7f8efdc user: Nick Coghlan date: Tue Mar 04 21:15:03 2014 +1000 summary: Workflow automation proposal updates - merge gating makes it easier for others to do continuous integration against the CPython development branches - interesting IRC chat with the CTO of RhodeCode last week, updated accordingly files: pep-0462.txt | 36 ++++++++++++++++++++++++++++++++---- 1 files changed, 32 insertions(+), 4 deletions(-) diff --git a/pep-0462.txt b/pep-0462.txt --- a/pep-0462.txt +++ b/pep-0462.txt @@ -232,6 +232,9 @@ (in the case of ``hg.python.org``, the relevant category is "public open source project"). +The RhodeCode developers have also expressed interest in helping out with +ensuring a python.org RhodeCode deployment is successful. + .. _RhodeCode: https://rhodecode.com/ .. _business source license: https://rhodecode.com/licenses @@ -411,13 +414,21 @@ the other sprint participants than it is on things that are sufficiently mechanical that a computer can (and should) handle them. -Finally, with most of the ways to make a mistake when committing a change -automated out of existence, there are substantially fewer new things to +With most of the ways to make a mistake when committing a change +automated out of existence, there are also substantially fewer new things to learn when a contributor is nominated to become a core developer. This should have a dual benefit, both in making the existing core developers more comfortable with granting that additional level of responsibility, and in making new contributors more comfortable with exercising it. +Finally, a more stable default branch in CPython makes it easier for +other Python projects to conduct continuous integration directly against the +main repo, rather than having to wait until we get into the release +candidate phase. At the moment, setting up such a system isn't particularly +attractive, as it would need to include an additional mechanism to wait +until CPython's own Buildbot fleet had indicate that the build was in a +usable state. + Technical Challenges ==================== @@ -434,8 +445,18 @@ User account management ----------------------- -Ideally, RhodeCode's user account management would be integrated with -the existing account management in Roundup. +Ideally we'd be able to offer a single account that spans all python.org +services, including RhodeCode, Roundup/Rietveld, PyPI and the back end for +the new python.org site, but actually implementing that would be a distinct +infrastructure project, independent of this PEP. + +A potentially simpler near term solution would be if RhodeCode's user +account management could be integrated with the existing account management +in Roundup, similar to what was done for Rietveld. However, if that also +turns out to be impractical in the near term, we would merely end up with +another identity silo to be integrated at a later date, suggesting that +this doesn't need to be considered a blocker for an initial RhodeCode +deployment. Preserving existing SSH access and links for Mercurial repositories @@ -784,6 +805,9 @@ workflow changes are not expected to require any significant changes in Roundup or Buildbot). +Unfortunately, the lead RhodeCode developers aren't able to attend PyCon US +this year, or we would have invited them, too. + Acknowledgements ================ @@ -793,6 +817,10 @@ Taylor for additional technical feedback following publication of the initial draft. +Thanks also to Marcin Kuzminski (CTO of RhodeCode) for getting in touch to +express their interest in helping to ensure the success of a RhodeCode +deployment if we choose to proceed down that path. + Copyright ========= -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Mar 4 18:20:00 2014 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 4 Mar 2014 18:20:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Attempt_to_clean_up_PEP_257_f?= =?utf-8?q?ootnotes=2E?= Message-ID: <3fdlN01Nxlz7Lk8@mail.python.org> http://hg.python.org/peps/rev/4b5a1100c7af changeset: 5399:4b5a1100c7af user: Guido van Rossum date: Tue Mar 04 09:19:58 2014 -0800 summary: Attempt to clean up PEP 257 footnotes. files: pep-0257.txt | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/pep-0257.txt b/pep-0257.txt --- a/pep-0257.txt +++ b/pep-0257.txt @@ -290,13 +290,10 @@ .. [2] PEP 258, Docutils Design Specification, Goodger (http://www.python.org/dev/peps/pep-0258/) -.. [3] Guido van Rossum, Python's creator and Benevolent Dictator For - Life. - .. _Docutils: http://docutils.sourceforge.net/ .. _Python Style Guide: - (http://www.python.org/dev/peps/pep-0008/) + http://www.python.org/dev/peps/pep-0008/ .. _Doc-SIG: http://www.python.org/sigs/doc-sig/ -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Mar 4 20:49:52 2014 From: python-checkins at python.org (donald.stufft) Date: Tue, 4 Mar 2014 20:49:52 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_the_Post-History_date_of_?= =?utf-8?q?PEP464?= Message-ID: <3fdphw2LYSz7LmR@mail.python.org> http://hg.python.org/peps/rev/ac37e8e47178 changeset: 5400:ac37e8e47178 user: Donald Stufft date: Tue Mar 04 14:49:47 2014 -0500 summary: Fix the Post-History date of PEP464 files: pep-0464.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0464.txt b/pep-0464.txt --- a/pep-0464.txt +++ b/pep-0464.txt @@ -9,7 +9,7 @@ Type: Process Content-Type: text/x-rst Created: 02-Mar-2014 -Post-History: 03-Mar-2014 +Post-History: 04-Mar-2014 Replaces: 381 -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Mar 4 21:24:47 2014 From: python-checkins at python.org (donald.stufft) Date: Tue, 4 Mar 2014 21:24:47 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Slight_rewording_of_PEP464?= Message-ID: <3fdqTC3vfvz7Ljn@mail.python.org> http://hg.python.org/peps/rev/08150dafffd5 changeset: 5401:08150dafffd5 user: Donald Stufft date: Tue Mar 04 15:24:43 2014 -0500 summary: Slight rewording of PEP464 files: pep-0464.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0464.txt b/pep-0464.txt --- a/pep-0464.txt +++ b/pep-0464.txt @@ -28,9 +28,9 @@ of that, it provides a method for verifying the authenticity of the mirrored content. -This PEP proposal the removal of this API due to: +This PEP proposes the removal of this API due to: -* No known implementations that utilize this API are known, this includes +* There are no known implementations that utilize this API, this includes `pip `_ and `setuptools `_. * Because this API uses DSA it is vulnerable to leaking the private key if -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Mar 4 23:08:26 2014 From: python-checkins at python.org (victor.stinner) Date: Tue, 4 Mar 2014 23:08:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio=2C_Tulip_issue_158?= =?utf-8?q?=3A_Task=2E=5Fstep=28=29_now_also_sets_self_to_None_if_an?= Message-ID: <3fdsmp11VJz7LjN@mail.python.org> http://hg.python.org/cpython/rev/f4a2d0e4e0be changeset: 89474:f4a2d0e4e0be user: Victor Stinner date: Tue Mar 04 23:07:08 2014 +0100 summary: asyncio, Tulip issue 158: Task._step() now also sets self to None if an exception is raised. self is set to None to break a reference cycle. files: Lib/asyncio/tasks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -325,7 +325,7 @@ 'Task got bad yield: {!r}'.format(result))) finally: self.__class__._current_tasks.pop(self._loop) - self = None + self = None # Needed to break cycles when an exception occurs. def _wakeup(self, future): try: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 5 00:26:45 2014 From: python-checkins at python.org (kristjan.jonsson) Date: Wed, 5 Mar 2014 00:26:45 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_pickling_o?= =?utf-8?q?f_rangeiter=2E__rangeiter=5Fsetstate_would_not_allow_setting_it?= Message-ID: <3fdvW955psz7LnD@mail.python.org> http://hg.python.org/cpython/rev/24d4e52f4f87 changeset: 89475:24d4e52f4f87 branch: 3.3 parent: 89469:204ef3bca9c8 user: Kristj?n Valur J?nsson date: Tue Mar 04 23:19:24 2014 +0000 summary: Fix pickling of rangeiter. rangeiter_setstate would not allow setting it to the exhausted state. files: Lib/test/test_range.py | 12 ++++++++++++ Objects/rangeobject.c | 2 +- 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_range.py b/Lib/test/test_range.py --- a/Lib/test/test_range.py +++ b/Lib/test/test_range.py @@ -379,6 +379,18 @@ it = pickle.loads(d) self.assertEqual(list(it), data[1:]) + def test_exhausted_iterator_pickling(self): + r = range(20) + i = iter(r) + while True: + r = next(i) + if r == 19: + break + d = pickle.dumps(i) + i2 = pickle.loads(d) + self.assertEqual(list(i), []) + self.assertEqual(list(i2), []) + def test_odd_bug(self): # This used to raise a "SystemError: NULL result without error" # because the range validation step was eating the exception diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -1000,7 +1000,7 @@ long index = PyLong_AsLong(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0 || index >= r->len) { + if (index < 0 || index > r->len) { PyErr_SetString(PyExc_ValueError, "index out of range"); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 5 00:26:46 2014 From: python-checkins at python.org (kristjan.jonsson) Date: Wed, 5 Mar 2014 00:26:46 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E3?= Message-ID: <3fdvWB6dH9z7LnD@mail.python.org> http://hg.python.org/cpython/rev/23d9daed4b14 changeset: 89476:23d9daed4b14 parent: 89474:f4a2d0e4e0be parent: 89475:24d4e52f4f87 user: Kristj?n Valur J?nsson date: Tue Mar 04 23:22:15 2014 +0000 summary: Merge with 3.3 files: Lib/test/test_range.py | 12 ++++++++++++ Objects/rangeobject.c | 2 +- 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_range.py b/Lib/test/test_range.py --- a/Lib/test/test_range.py +++ b/Lib/test/test_range.py @@ -380,6 +380,18 @@ it = pickle.loads(d) self.assertEqual(list(it), data[1:]) + def test_exhausted_iterator_pickling(self): + r = range(20) + i = iter(r) + while True: + r = next(i) + if r == 19: + break + d = pickle.dumps(i) + i2 = pickle.loads(d) + self.assertEqual(list(i), []) + self.assertEqual(list(i2), []) + def test_odd_bug(self): # This used to raise a "SystemError: NULL result without error" # because the range validation step was eating the exception diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -807,7 +807,7 @@ long index = PyLong_AsLong(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0 || index >= r->len) { + if (index < 0 || index > r->len) { PyErr_SetString(PyExc_ValueError, "index out of range"); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 5 03:47:46 2014 From: python-checkins at python.org (larry.hastings) Date: Wed, 5 Mar 2014 03:47:46 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Updated_429_=283=2E4_release_?= =?utf-8?q?schedule=29=2C_including_adding_a_third_RC=2E?= Message-ID: <3fdzz66NF5z7Lw1@mail.python.org> http://hg.python.org/peps/rev/2cf89e9e50a3 changeset: 5402:2cf89e9e50a3 user: Larry Hastings date: Tue Mar 04 18:47:44 2014 -0800 summary: Updated 429 (3.4 release schedule), including adding a third RC. files: pep-0429.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0429.txt b/pep-0429.txt --- a/pep-0429.txt +++ b/pep-0429.txt @@ -44,12 +44,13 @@ - 3.4.0 beta 2: January 5, 2014 - 3.4.0 beta 3: January 26, 2014 - 3.4.0 candidate 1: February 10, 2014 +- 3.4.0 candidate 2: February 23, 2014 (Beta 1 is also "feature freeze"--no new features beyond this point.) The anticipated schedule for future releases: -- 3.4.0 candidate 2: February 23, 2014 +- 3.4.0 candidate 3: March 9, 2014 - 3.4.0 final: March 16, 2014 .. don't forget to update final date above as well -- Repository URL: http://hg.python.org/peps From solipsis at pitrou.net Wed Mar 5 09:55:23 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 05 Mar 2014 09:55:23 +0100 Subject: [Python-checkins] Daily reference leaks (23d9daed4b14): sum=0 Message-ID: results for 23d9daed4b14 on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloga65uPA', '-x'] From python-checkins at python.org Wed Mar 5 16:24:30 2014 From: python-checkins at python.org (kristjan.jonsson) Date: Wed, 5 Mar 2014 16:24:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Make_the_vario?= =?utf-8?q?us_iterators=27_=22setstate=22_sliently_and_consistently_clip_t?= =?utf-8?q?he?= Message-ID: <3ffJmG2K2Yz7Llf@mail.python.org> http://hg.python.org/cpython/rev/3b2c28061184 changeset: 89477:3b2c28061184 branch: 3.3 parent: 89475:24d4e52f4f87 user: Kristj?n Valur J?nsson date: Wed Mar 05 13:47:57 2014 +0000 summary: Make the various iterators' "setstate" sliently and consistently clip the index. This avoids the possibility of setting an iterator to an invalid state. files: Lib/test/test_range.py | 12 ++++++++++ Modules/arraymodule.c | 2 + Objects/bytearrayobject.c | 10 ++++++-- Objects/bytesobject.c | 10 ++++++-- Objects/listobject.c | 2 + Objects/rangeobject.c | 31 +++++++++++++++++++++++--- Objects/tupleobject.c | 4 +- Objects/unicodeobject.c | 10 ++++++-- 8 files changed, 66 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_range.py b/Lib/test/test_range.py --- a/Lib/test/test_range.py +++ b/Lib/test/test_range.py @@ -380,6 +380,18 @@ self.assertEqual(list(it), data[1:]) def test_exhausted_iterator_pickling(self): + r = range(2**65, 2**65+2) + i = iter(r) + while True: + r = next(i) + if r == 2**65+1: + break + d = pickle.dumps(i) + i2 = pickle.loads(d) + self.assertEqual(list(i), []) + self.assertEqual(list(i2), []) + + def test_large_exhausted_iterator_pickling(self): r = range(20) i = iter(r) while True: diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -2798,6 +2798,8 @@ return NULL; if (index < 0) index = 0; + else if (index > Py_SIZE(it->ao)) + index = Py_SIZE(it->ao); /* iterator exhausted */ it->index = index; Py_RETURN_NONE; } diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -3043,9 +3043,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyByteArray_GET_SIZE(it->it_seq)) + index = PyByteArray_GET_SIZE(it->it_seq); /* iterator exhausted */ + it->it_index = index; + } Py_RETURN_NONE; } diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -2996,9 +2996,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyBytes_GET_SIZE(it->it_seq)) + index = PyBytes_GET_SIZE(it->it_seq); /* iterator exhausted */ + it->it_index = index; + } Py_RETURN_NONE; } diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -2803,6 +2803,8 @@ if (it->it_seq != NULL) { if (index < 0) index = 0; + else if (index > PyList_GET_SIZE(it->it_seq)) + index = PyList_GET_SIZE(it->it_seq); /* iterator exhausted */ it->it_index = index; } Py_RETURN_NONE; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -1000,10 +1000,11 @@ long index = PyLong_AsLong(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0 || index > r->len) { - PyErr_SetString(PyExc_ValueError, "index out of range"); - return NULL; - } + /* silently clip the index value */ + if (index < 0) + index = 0; + else if (index > r->len) + index = r->len; /* exhausted iterator */ r->index = index; Py_RETURN_NONE; } @@ -1178,6 +1179,28 @@ static PyObject * longrangeiter_setstate(longrangeiterobject *r, PyObject *state) { + int cmp; + + /* clip the value */ + PyObject *zero = PyLong_FromLong(0); + if (zero == NULL) + return NULL; + cmp = PyObject_RichCompareBool(state, zero, Py_LT); + if (cmp > 0) { + Py_CLEAR(r->index); + r->index = zero; + Py_RETURN_NONE; + } + Py_DECREF(zero); + if (cmp < 0) + return NULL; + + cmp = PyObject_RichCompareBool(r->len, state, Py_LT); + if (cmp < 0) + return NULL; + if (cmp > 0) + state = r->len; + Py_CLEAR(r->index); r->index = state; Py_INCREF(r->index); diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -998,8 +998,8 @@ if (it->it_seq != NULL) { if (index < 0) index = 0; - else if (it->it_seq != NULL && index > PyTuple_GET_SIZE(it->it_seq)) - index = PyTuple_GET_SIZE(it->it_seq); + else if (index > PyTuple_GET_SIZE(it->it_seq)) + index = PyTuple_GET_SIZE(it->it_seq); /* exhausted iterator */ it->it_index = index; } Py_RETURN_NONE; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -14420,9 +14420,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyUnicode_GET_LENGTH(it->it_seq)) + index = PyUnicode_GET_LENGTH(it->it_seq); /* iterator truncated */ + it->it_index = index; + } Py_RETURN_NONE; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 5 16:24:31 2014 From: python-checkins at python.org (kristjan.jonsson) Date: Wed, 5 Mar 2014 16:24:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Make_the_various_iterators=27_=22setstate=22_sliently_an?= =?utf-8?q?d_consistently_clip_the?= Message-ID: <3ffJmH59Sqz7Lts@mail.python.org> http://hg.python.org/cpython/rev/08cf35d4ef49 changeset: 89478:08cf35d4ef49 parent: 89476:23d9daed4b14 parent: 89477:3b2c28061184 user: Kristj?n Valur J?nsson date: Wed Mar 05 15:23:07 2014 +0000 summary: Make the various iterators' "setstate" sliently and consistently clip the index. This avoids the possibility of setting an iterator to an invalid state. files: Lib/test/test_range.py | 12 ++++++++++ Modules/arraymodule.c | 2 + Objects/bytearrayobject.c | 10 ++++++-- Objects/bytesobject.c | 10 ++++++-- Objects/listobject.c | 2 + Objects/rangeobject.c | 31 +++++++++++++++++++++++--- Objects/tupleobject.c | 4 +- Objects/unicodeobject.c | 10 ++++++-- 8 files changed, 66 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_range.py b/Lib/test/test_range.py --- a/Lib/test/test_range.py +++ b/Lib/test/test_range.py @@ -381,6 +381,18 @@ self.assertEqual(list(it), data[1:]) def test_exhausted_iterator_pickling(self): + r = range(2**65, 2**65+2) + i = iter(r) + while True: + r = next(i) + if r == 2**65+1: + break + d = pickle.dumps(i) + i2 = pickle.loads(d) + self.assertEqual(list(i), []) + self.assertEqual(list(i2), []) + + def test_large_exhausted_iterator_pickling(self): r = range(20) i = iter(r) while True: diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -2838,6 +2838,8 @@ return NULL; if (index < 0) index = 0; + else if (index > Py_SIZE(it->ao)) + index = Py_SIZE(it->ao); /* iterator exhausted */ it->index = index; Py_RETURN_NONE; } diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -3025,9 +3025,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyByteArray_GET_SIZE(it->it_seq)) + index = PyByteArray_GET_SIZE(it->it_seq); /* iterator exhausted */ + it->it_index = index; + } Py_RETURN_NONE; } diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -2937,9 +2937,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyBytes_GET_SIZE(it->it_seq)) + index = PyBytes_GET_SIZE(it->it_seq); /* iterator exhausted */ + it->it_index = index; + } Py_RETURN_NONE; } diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -2811,6 +2811,8 @@ if (it->it_seq != NULL) { if (index < 0) index = 0; + else if (index > PyList_GET_SIZE(it->it_seq)) + index = PyList_GET_SIZE(it->it_seq); /* iterator exhausted */ it->it_index = index; } Py_RETURN_NONE; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -807,10 +807,11 @@ long index = PyLong_AsLong(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0 || index > r->len) { - PyErr_SetString(PyExc_ValueError, "index out of range"); - return NULL; - } + /* silently clip the index value */ + if (index < 0) + index = 0; + else if (index > r->len) + index = r->len; /* exhausted iterator */ r->index = index; Py_RETURN_NONE; } @@ -985,6 +986,28 @@ static PyObject * longrangeiter_setstate(longrangeiterobject *r, PyObject *state) { + int cmp; + + /* clip the value */ + PyObject *zero = PyLong_FromLong(0); + if (zero == NULL) + return NULL; + cmp = PyObject_RichCompareBool(state, zero, Py_LT); + if (cmp > 0) { + Py_CLEAR(r->index); + r->index = zero; + Py_RETURN_NONE; + } + Py_DECREF(zero); + if (cmp < 0) + return NULL; + + cmp = PyObject_RichCompareBool(r->len, state, Py_LT); + if (cmp < 0) + return NULL; + if (cmp > 0) + state = r->len; + Py_CLEAR(r->index); r->index = state; Py_INCREF(r->index); diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -1011,8 +1011,8 @@ if (it->it_seq != NULL) { if (index < 0) index = 0; - else if (it->it_seq != NULL && index > PyTuple_GET_SIZE(it->it_seq)) - index = PyTuple_GET_SIZE(it->it_seq); + else if (index > PyTuple_GET_SIZE(it->it_seq)) + index = PyTuple_GET_SIZE(it->it_seq); /* exhausted iterator */ it->it_index = index; } Py_RETURN_NONE; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -15196,9 +15196,13 @@ Py_ssize_t index = PyLong_AsSsize_t(state); if (index == -1 && PyErr_Occurred()) return NULL; - if (index < 0) - index = 0; - it->it_index = index; + if (it->it_seq != NULL) { + if (index < 0) + index = 0; + else if (index > PyUnicode_GET_LENGTH(it->it_seq)) + index = PyUnicode_GET_LENGTH(it->it_seq); /* iterator truncated */ + it->it_index = index; + } Py_RETURN_NONE; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 00:54:30 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 6 Mar 2014 00:54:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio=3A_Synchronize_wit?= =?utf-8?q?h_Tulip?= Message-ID: <3ffX4k67LWz7Lkm@mail.python.org> http://hg.python.org/cpython/rev/bf94d79cecfe changeset: 89479:bf94d79cecfe user: Victor Stinner date: Thu Mar 06 00:52:53 2014 +0100 summary: asyncio: Synchronize with Tulip * Issue #159: Fix windows_utils.socketpair() - Use "127.0.0.1" (IPv4) or "::1" (IPv6) host instead of "localhost", because "localhost" may be a different IP address - Reject also invalid arguments: only AF_INET/AF_INET6 with SOCK_STREAM (and proto=0) are supported * Reject add/remove reader/writer when event loop is closed. * Fix ResourceWarning warnings files: Lib/asyncio/selector_events.py | 8 ++ Lib/asyncio/windows_utils.py | 17 +++++- Lib/test/test_asyncio/test_events.py | 27 ++++++++++ Lib/test/test_asyncio/test_windows_utils.py | 27 ++++++++- 4 files changed, 73 insertions(+), 6 deletions(-) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -136,6 +136,8 @@ def add_reader(self, fd, callback, *args): """Add a reader callback.""" + if self._selector is None: + raise RuntimeError('Event loop is closed') handle = events.Handle(callback, args, self) try: key = self._selector.get_key(fd) @@ -151,6 +153,8 @@ def remove_reader(self, fd): """Remove a reader callback.""" + if self._selector is None: + return False try: key = self._selector.get_key(fd) except KeyError: @@ -171,6 +175,8 @@ def add_writer(self, fd, callback, *args): """Add a writer callback..""" + if self._selector is None: + raise RuntimeError('Event loop is closed') handle = events.Handle(callback, args, self) try: key = self._selector.get_key(fd) @@ -186,6 +192,8 @@ def remove_writer(self, fd): """Remove a writer callback.""" + if self._selector is None: + return False try: key = self._selector.get_key(fd) except KeyError: diff --git a/Lib/asyncio/windows_utils.py b/Lib/asyncio/windows_utils.py --- a/Lib/asyncio/windows_utils.py +++ b/Lib/asyncio/windows_utils.py @@ -36,12 +36,25 @@ Origin: https://gist.github.com/4325783, by Geert Jansen. Public domain. """ + if family == socket.AF_INET: + host = '127.0.0.1' + elif family == socket.AF_INET6: + host = '::1' + else: + raise ValueError("Ony AF_INET and AF_INET6 socket address families " + "are supported") + if type != socket.SOCK_STREAM: + raise ValueError("Only SOCK_STREAM socket type is supported") + if proto != 0: + raise ValueError("Only protocol zero is supported") + # We create a connected TCP socket. Note the trick with setblocking(0) # that prevents us from having to create a thread. lsock = socket.socket(family, type, proto) - lsock.bind(('localhost', 0)) + lsock.bind((host, 0)) lsock.listen(1) - addr, port = lsock.getsockname() + # On IPv6, ignore flow_info and scope_id + addr, port = lsock.getsockname()[:2] csock = socket.socket(family, type, proto) csock.setblocking(False) try: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1326,6 +1326,30 @@ self.assertIn('address must be resolved', str(cm.exception)) + def test_remove_fds_after_closing(self): + loop = self.create_event_loop() + callback = lambda: None + r, w = test_utils.socketpair() + self.addCleanup(r.close) + self.addCleanup(w.close) + loop.add_reader(r, callback) + loop.add_writer(w, callback) + loop.close() + self.assertFalse(loop.remove_reader(r)) + self.assertFalse(loop.remove_writer(w)) + + def test_add_fds_after_closing(self): + loop = self.create_event_loop() + callback = lambda: None + r, w = test_utils.socketpair() + self.addCleanup(r.close) + self.addCleanup(w.close) + loop.close() + with self.assertRaises(RuntimeError): + loop.add_reader(r, callback) + with self.assertRaises(RuntimeError): + loop.add_writer(w, callback) + class SubprocessTestsMixin: @@ -1632,6 +1656,9 @@ def test_create_datagram_endpoint(self): raise unittest.SkipTest( "IocpEventLoop does not have create_datagram_endpoint()") + + def test_remove_fds_after_closing(self): + raise unittest.SkipTest("IocpEventLoop does not have add_reader()") else: from asyncio import selectors diff --git a/Lib/test/test_asyncio/test_windows_utils.py b/Lib/test/test_asyncio/test_windows_utils.py --- a/Lib/test/test_asyncio/test_windows_utils.py +++ b/Lib/test/test_asyncio/test_windows_utils.py @@ -1,8 +1,10 @@ """Tests for window_utils""" +import socket import sys import test.support import unittest +from test.support import IPV6_ENABLED from unittest import mock if sys.platform != 'win32': @@ -16,23 +18,40 @@ class WinsocketpairTests(unittest.TestCase): - def test_winsocketpair(self): - ssock, csock = windows_utils.socketpair() - + def check_winsocketpair(self, ssock, csock): csock.send(b'xxx') self.assertEqual(b'xxx', ssock.recv(1024)) - csock.close() ssock.close() + def test_winsocketpair(self): + ssock, csock = windows_utils.socketpair() + self.check_winsocketpair(ssock, csock) + + @unittest.skipUnless(IPV6_ENABLED, 'IPv6 not supported or enabled') + def test_winsocketpair_ipv6(self): + ssock, csock = windows_utils.socketpair(family=socket.AF_INET6) + self.check_winsocketpair(ssock, csock) + @mock.patch('asyncio.windows_utils.socket') def test_winsocketpair_exc(self, m_socket): + m_socket.AF_INET = socket.AF_INET + m_socket.SOCK_STREAM = socket.SOCK_STREAM m_socket.socket.return_value.getsockname.return_value = ('', 12345) m_socket.socket.return_value.accept.return_value = object(), object() m_socket.socket.return_value.connect.side_effect = OSError() self.assertRaises(OSError, windows_utils.socketpair) + def test_winsocketpair_invalid_args(self): + self.assertRaises(ValueError, + windows_utils.socketpair, family=socket.AF_UNSPEC) + self.assertRaises(ValueError, + windows_utils.socketpair, type=socket.SOCK_DGRAM) + self.assertRaises(ValueError, + windows_utils.socketpair, proto=1) + + class PipeTests(unittest.TestCase): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 01:02:35 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 6 Mar 2014 01:02:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio=2C_Tulip_issue_157?= =?utf-8?q?=3A_Improve_test=5Fevents=2Epy=2C_avoid_run=5Fbriefly=28=29_whi?= =?utf-8?q?ch_is?= Message-ID: <3ffXG30bCYz7Lkm@mail.python.org> http://hg.python.org/cpython/rev/56c346e9ae4d changeset: 89480:56c346e9ae4d user: Victor Stinner date: Thu Mar 06 01:00:36 2014 +0100 summary: asyncio, Tulip issue 157: Improve test_events.py, avoid run_briefly() which is not reliable files: Lib/asyncio/test_utils.py | 15 +- Lib/test/test_asyncio/test_events.py | 129 +++++++------- 2 files changed, 71 insertions(+), 73 deletions(-) diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py --- a/Lib/asyncio/test_utils.py +++ b/Lib/asyncio/test_utils.py @@ -21,10 +21,11 @@ except ImportError: # pragma: no cover ssl = None -from . import tasks from . import base_events from . import events +from . import futures from . import selectors +from . import tasks if sys.platform == 'win32': # pragma: no cover @@ -52,18 +53,14 @@ gen.close() -def run_until(loop, pred, timeout=None): - if timeout is not None: - deadline = time.time() + timeout +def run_until(loop, pred, timeout=30): + deadline = time.time() + timeout while not pred(): if timeout is not None: timeout = deadline - time.time() if timeout <= 0: - return False - loop.run_until_complete(tasks.sleep(timeout, loop=loop)) - else: - run_briefly(loop) - return True + raise futures.TimeoutError() + loop.run_until_complete(tasks.sleep(0.001, loop=loop)) def run_once(loop): diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -56,6 +56,7 @@ class MyBaseProto(asyncio.Protocol): + connected = None done = None def __init__(self, loop=None): @@ -63,12 +64,15 @@ self.state = 'INITIAL' self.nbytes = 0 if loop is not None: + self.connected = asyncio.Future(loop=loop) self.done = asyncio.Future(loop=loop) def connection_made(self, transport): self.transport = transport assert self.state == 'INITIAL', self.state self.state = 'CONNECTED' + if self.connected: + self.connected.set_result(None) def data_received(self, data): assert self.state == 'CONNECTED', self.state @@ -330,7 +334,8 @@ def test_reader_callback(self): r, w = test_utils.socketpair() - bytes_read = [] + r.setblocking(False) + bytes_read = bytearray() def reader(): try: @@ -340,37 +345,40 @@ # at least on Linux -- see man select. return if data: - bytes_read.append(data) + bytes_read.extend(data) else: self.assertTrue(self.loop.remove_reader(r.fileno())) r.close() self.loop.add_reader(r.fileno(), reader) self.loop.call_soon(w.send, b'abc') - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: len(bytes_read) >= 3) self.loop.call_soon(w.send, b'def') - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: len(bytes_read) >= 6) self.loop.call_soon(w.close) self.loop.call_soon(self.loop.stop) self.loop.run_forever() - self.assertEqual(b''.join(bytes_read), b'abcdef') + self.assertEqual(bytes_read, b'abcdef') def test_writer_callback(self): r, w = test_utils.socketpair() w.setblocking(False) - self.loop.add_writer(w.fileno(), w.send, b'x'*(256*1024)) - test_utils.run_briefly(self.loop) - def remove_writer(): - self.assertTrue(self.loop.remove_writer(w.fileno())) + def writer(data): + w.send(data) + self.loop.stop() - self.loop.call_soon(remove_writer) - self.loop.call_soon(self.loop.stop) + data = b'x' * 1024 + self.loop.add_writer(w.fileno(), writer, data) self.loop.run_forever() + + self.assertTrue(self.loop.remove_writer(w.fileno())) + self.assertFalse(self.loop.remove_writer(w.fileno())) + w.close() - data = r.recv(256*1024) + read = r.recv(len(data) * 2) r.close() - self.assertGreaterEqual(len(data), 200) + self.assertEqual(read, data) def _basetest_sock_client_ops(self, httpd, sock): sock.setblocking(False) @@ -464,10 +472,10 @@ self.assertFalse(self.loop.remove_signal_handler(signal.SIGKILL)) # Now set a handler and handle it. self.loop.add_signal_handler(signal.SIGINT, my_handler) - test_utils.run_briefly(self.loop) + os.kill(os.getpid(), signal.SIGINT) - test_utils.run_briefly(self.loop) - self.assertEqual(caught, 1) + test_utils.run_until(self.loop, lambda: caught) + # Removing it should restore the default handler. self.assertTrue(self.loop.remove_signal_handler(signal.SIGINT)) self.assertEqual(signal.getsignal(signal.SIGINT), @@ -623,7 +631,7 @@ self.assertIn(str(httpd.address), cm.exception.strerror) def test_create_server(self): - proto = MyProto() + proto = MyProto(self.loop) f = self.loop.create_server(lambda: proto, '0.0.0.0', 0) server = self.loop.run_until_complete(f) self.assertEqual(len(server.sockets), 1) @@ -633,14 +641,11 @@ client = socket.socket() client.connect(('127.0.0.1', port)) client.sendall(b'xxx') - test_utils.run_briefly(self.loop) - test_utils.run_until(self.loop, lambda: proto is not None, 10) - self.assertIsInstance(proto, MyProto) - self.assertEqual('INITIAL', proto.state) - test_utils.run_briefly(self.loop) + + self.loop.run_until_complete(proto.connected) self.assertEqual('CONNECTED', proto.state) - test_utils.run_until(self.loop, lambda: proto.nbytes > 0, - timeout=10) + + test_utils.run_until(self.loop, lambda: proto.nbytes > 0) self.assertEqual(3, proto.nbytes) # extra info is available @@ -650,7 +655,7 @@ # close connection proto.transport.close() - test_utils.run_briefly(self.loop) # windows iocp + self.loop.run_until_complete(proto.done) self.assertEqual('CLOSED', proto.state) @@ -672,27 +677,22 @@ @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'No UNIX Sockets') def test_create_unix_server(self): - proto = MyProto() + proto = MyProto(loop=self.loop) server, path = self._make_unix_server(lambda: proto) self.assertEqual(len(server.sockets), 1) client = socket.socket(socket.AF_UNIX) client.connect(path) client.sendall(b'xxx') - test_utils.run_briefly(self.loop) - test_utils.run_until(self.loop, lambda: proto is not None, 10) - self.assertIsInstance(proto, MyProto) - self.assertEqual('INITIAL', proto.state) - test_utils.run_briefly(self.loop) + self.loop.run_until_complete(proto.connected) self.assertEqual('CONNECTED', proto.state) - test_utils.run_until(self.loop, lambda: proto.nbytes > 0, - timeout=10) + test_utils.run_until(self.loop, lambda: proto.nbytes > 0) self.assertEqual(3, proto.nbytes) # close connection proto.transport.close() - test_utils.run_briefly(self.loop) # windows iocp + self.loop.run_until_complete(proto.done) self.assertEqual('CLOSED', proto.state) @@ -735,12 +735,10 @@ client, pr = self.loop.run_until_complete(f_c) client.write(b'xxx') - test_utils.run_briefly(self.loop) - self.assertIsInstance(proto, MyProto) - test_utils.run_briefly(self.loop) + self.loop.run_until_complete(proto.connected) self.assertEqual('CONNECTED', proto.state) - test_utils.run_until(self.loop, lambda: proto.nbytes > 0, - timeout=10) + + test_utils.run_until(self.loop, lambda: proto.nbytes > 0) self.assertEqual(3, proto.nbytes) # extra info is available @@ -774,12 +772,9 @@ client, pr = self.loop.run_until_complete(f_c) client.write(b'xxx') - test_utils.run_briefly(self.loop) - self.assertIsInstance(proto, MyProto) - test_utils.run_briefly(self.loop) + self.loop.run_until_complete(proto.connected) self.assertEqual('CONNECTED', proto.state) - test_utils.run_until(self.loop, lambda: proto.nbytes > 0, - timeout=10) + test_utils.run_until(self.loop, lambda: proto.nbytes > 0) self.assertEqual(3, proto.nbytes) # close connection @@ -1044,15 +1039,9 @@ self.assertEqual('INITIALIZED', client.state) transport.sendto(b'xxx') - for _ in range(1000): - if server.nbytes: - break - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: server.nbytes) self.assertEqual(3, server.nbytes) - for _ in range(1000): - if client.nbytes: - break - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: client.nbytes) # received self.assertEqual(8, client.nbytes) @@ -1097,11 +1086,11 @@ self.loop.run_until_complete(connect()) os.write(wpipe, b'1') - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: proto.nbytes >= 1) self.assertEqual(1, proto.nbytes) os.write(wpipe, b'2345') - test_utils.run_briefly(self.loop) + test_utils.run_until(self.loop, lambda: proto.nbytes >= 5) self.assertEqual(['INITIAL', 'CONNECTED'], proto.state) self.assertEqual(5, proto.nbytes) @@ -1166,14 +1155,19 @@ self.assertEqual('CONNECTED', proto.state) transport.write(b'1') - test_utils.run_briefly(self.loop) - data = os.read(rpipe, 1024) + + data = bytearray() + def reader(data): + chunk = os.read(rpipe, 1024) + data += chunk + return len(data) + + test_utils.run_until(self.loop, lambda: reader(data) >= 1) self.assertEqual(b'1', data) transport.write(b'2345') - test_utils.run_briefly(self.loop) - data = os.read(rpipe, 1024) - self.assertEqual(b'2345', data) + test_utils.run_until(self.loop, lambda: reader(data) >= 5) + self.assertEqual(b'12345', data) self.assertEqual('CONNECTED', proto.state) os.close(rpipe) @@ -1225,14 +1219,21 @@ self.assertEqual('CONNECTED', proto.state) transport.write(b'1') - test_utils.run_briefly(self.loop) - data = os.read(master, 1024) + + data = bytearray() + def reader(data): + chunk = os.read(master, 1024) + data += chunk + return len(data) + + test_utils.run_until(self.loop, lambda: reader(data) >= 1, + timeout=10) self.assertEqual(b'1', data) transport.write(b'2345') - test_utils.run_briefly(self.loop) - data = os.read(master, 1024) - self.assertEqual(b'2345', data) + test_utils.run_until(self.loop, lambda: reader(data) >= 5, + timeout=10) + self.assertEqual(b'12345', data) self.assertEqual('CONNECTED', proto.state) os.close(master) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 02:02:17 2014 From: python-checkins at python.org (thomas.wouters) Date: Thu, 6 Mar 2014 02:02:17 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_PEP_463_to_Chris_Angel?= =?utf-8?q?ico=27s_latest_=28and_supposedly_final=29_version=2E?= Message-ID: <3ffYZx3Gzyz7Lkj@mail.python.org> http://hg.python.org/peps/rev/59653081cdf6 changeset: 5403:59653081cdf6 user: Thomas Wouters date: Thu Mar 06 02:02:00 2014 +0100 summary: Update PEP 463 to Chris Angelico's latest (and supposedly final) version. files: pep-0463.txt | 41 +++++++++++++++++++++++++++++++++++++++- 1 files changed, 40 insertions(+), 1 deletions(-) diff --git a/pep-0463.txt b/pep-0463.txt --- a/pep-0463.txt +++ b/pep-0463.txt @@ -250,7 +250,8 @@ alternatives listed above must (by the nature of functions) evaluate their default values eagerly. The preferred form, using the colon, parallels try/except by using "except exception_list:", and parallels lambda by having -"keyword name_list: subexpression". Using the arrow introduces a token many +"keyword name_list: subexpression"; it also can be read as mapping Exception +to the default value, dict-style. Using the arrow introduces a token many programmers will not be familiar with, and which currently has no similar meaning, but is otherwise quite readable. The English word "pass" has a vaguely similar meaning (consider the common usage "pass by value/reference" @@ -271,6 +272,18 @@ Using the preferred order, subexpressions will always be evaluated from left to right, no matter how the syntax is nested. +Keeping the existing notation, but shifting the mandatory parentheses, we +have the following suggestion:: + + value = expr except (Exception: default) + value = expr except(Exception: default) + +This is reminiscent of a function call, or a dict initializer. The colon +cannot be confused with introducing a suite, but on the other hand, the new +syntax guarantees lazy evaluation, which a dict does not. The potential +to reduce confusion is considered unjustified by the corresponding potential +to increase it. + Example usage ============= @@ -854,6 +867,32 @@ expression to achieve this. +Common objections +================= + +Colons always introduce suites +------------------------------ + +While it is true that many of Python's syntactic elements use the colon to +introduce a statement suite (if, while, with, for, etcetera), this is not +by any means the sole use of the colon. Currently, Python syntax includes +four cases where a colon introduces a subexpression: + +* dict display - { ... key:value ... } +* slice notation - [start:stop:step] +* function definition - parameter : annotation +* lambda - arg list: return value + +This proposal simply adds a fifth: + +* except-expression - exception list: result + +Style guides and PEP 8 should recommend not having the colon at the end of +a wrapped line, which could potentially look like the introduction of a +suite, but instead advocate wrapping before the exception list, keeping the +colon clearly between two expressions. + + Copyright ========= -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Thu Mar 6 05:16:30 2014 From: python-checkins at python.org (terry.reedy) Date: Thu, 6 Mar 2014 05:16:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE1NjE4?= =?utf-8?q?=3A_Make_turtle=2Epy_itself_work_when_run_from_a_module_with?= Message-ID: <3ffdv20dk0z7M2k@mail.python.org> http://hg.python.org/cpython/rev/1be39dc4409a changeset: 89481:1be39dc4409a branch: 2.7 parent: 89468:d37f963394aa user: Terry Jan Reedy date: Wed Mar 05 23:15:57 2014 -0500 summary: Issue #15618: Make turtle.py itself work when run from a module with from __future__ import unicode_literals. Initial patch by Juancarlo A?ez. The demos at the end of turtle.py appear to be the only test, so I changed some of the strings to unicode with a u prefix. If os.path.isfile or the Tk image function have problems with Unicode input, that would be different issue. files: Lib/lib-tk/turtle.py | 18 +++++++++--------- Misc/ACKS | 1 + Misc/NEWS | 5 ++++- 3 files changed, 14 insertions(+), 10 deletions(-) diff --git a/Lib/lib-tk/turtle.py b/Lib/lib-tk/turtle.py --- a/Lib/lib-tk/turtle.py +++ b/Lib/lib-tk/turtle.py @@ -835,7 +835,7 @@ if isinstance(data, list): data = tuple(data) elif type_ == "image": - if isinstance(data, str): + if isinstance(data, basestring): if data.lower().endswith(".gif") and isfile(data): data = TurtleScreen._image(data) # else data assumed to be Photoimage @@ -1098,7 +1098,7 @@ """ if len(color) == 1: color = color[0] - if isinstance(color, str): + if isinstance(color, basestring): if self._iscolorstring(color) or color == "": return color else: @@ -2602,7 +2602,7 @@ def _cc(self, args): """Convert colortriples to hexstrings. """ - if isinstance(args, str): + if isinstance(args, basestring): return args try: r, g, b = args @@ -3228,7 +3228,7 @@ """ #print "dot-1:", size, color if not color: - if isinstance(size, (str, tuple)): + if isinstance(size, (basestring, tuple)): color = self._colorstr(size) size = self._pensize + max(self._pensize, 4) else: @@ -3913,7 +3913,7 @@ down() # some text write("startstart", 1) - write("start", 1) + write(u"start", 1) color("red") # staircase for i in range(5): @@ -3988,7 +3988,7 @@ tri = getturtle() tri.resizemode("auto") turtle = Turtle() - turtle.resizemode("auto") + turtle.resizemode(u"auto") turtle.shape("turtle") turtle.reset() turtle.left(90) @@ -3998,7 +3998,7 @@ turtle.lt(30) turtle.down() turtle.speed(6) - turtle.color("blue","orange") + turtle.color("blue",u"orange") turtle.pensize(2) tri.speed(6) setheading(towards(turtle)) @@ -4013,9 +4013,9 @@ tri.stamp() switchpen() count += 1 - tri.write("CAUGHT! ", font=("Arial", 16, "bold"), align="right") + tri.write("CAUGHT! ", font=("Arial", 16, "bold"), align=u"right") tri.pencolor("black") - tri.pencolor("red") + tri.pencolor(u"red") def baba(xdummy, ydummy): clearscreen() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -33,6 +33,7 @@ Erik Anders?n Oliver Andrich Ross Andrus +Juancarlo A?ez Chris Angelico Ankur Ankan Heidi Annexstad diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,9 @@ Library ------- +- Issue #15618: Make turtle.py compatible with 'from __future__ import + unicode_literals'. Initial patch by Juancarlo A?ez. + - Issue #20501: fileinput module no longer reads whole file into memory when using fileinput.hook_encoded. @@ -53,7 +56,7 @@ Based on patch by Stephen Tu. - Issue #8478: Untokenizer.compat processes first token from iterator input. - Patch based on lines from Georg Brandl, Eric Snow, and Gareth Rees. + Patch based on lines from Georg Brandl, Eric Snow, and Gareth Rees. - Issue #20594: Avoid name clash with the libc function posix_close. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Thu Mar 6 10:03:05 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 06 Mar 2014 10:03:05 +0100 Subject: [Python-checkins] Daily reference leaks (56c346e9ae4d): sum=0 Message-ID: results for 56c346e9ae4d on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogfXg3mv', '-x'] From python-checkins at python.org Thu Mar 6 10:37:28 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 6 Mar 2014 10:37:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwMjgz?= =?utf-8?q?=3A_RE_pattern_methods_now_accept_the_string_keyword_parameters?= Message-ID: <3ffn1N07fgz7M0f@mail.python.org> http://hg.python.org/cpython/rev/52743dc788e6 changeset: 89482:52743dc788e6 branch: 3.3 parent: 89477:3b2c28061184 user: Serhiy Storchaka date: Thu Mar 06 11:28:32 2014 +0200 summary: Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. files: Lib/test/test_re.py | 16 ++++++ Misc/NEWS | 4 + Modules/_sre.c | 84 +++++++++++++++++++++++++------- 3 files changed, 84 insertions(+), 20 deletions(-) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1076,6 +1076,22 @@ self.assertEqual(out.getvalue().splitlines(), ['literal 102 ', 'literal 111 ', 'literal 111 ']) + def test_keyword_parameters(self): + # Issue #20283: Accepting the string keyword parameter. + pat = re.compile(r'(ab)') + self.assertEqual( + pat.match(string='abracadabra', pos=7, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.search(string='abracadabra', pos=3, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.findall(string='abracadabra', pos=3, endpos=10), ['ab']) + self.assertEqual( + pat.split(string='abracadabra', maxsplit=1), + ['', 'ab', 'racadabra']) + self.assertEqual( + pat.scanner(string='abracadabra', pos=3, endpos=10).search().span(), + (7, 9)) + def run_re_tests(): from test.re_tests import tests, SUCCEED, FAIL, SYNTAX_ERROR diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,6 +22,10 @@ Library ------- +- Issue #20283: RE pattern methods now accept the string keyword parameters + as documented. The pattern and source keyword parameters are left as + deprecated aliases. + - Issue #20778: Fix modulefinder to work with bytecode-only modules. - Issue #20791: copy.copy() now doesn't make a copy when the input is diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -1877,17 +1877,45 @@ } static PyObject* +fix_string_param(PyObject *string, PyObject *string2, const char *oldname) +{ + if (string2 != NULL) { + if (string != NULL) { + PyErr_Format(PyExc_TypeError, + "Argument given by name ('%s') and position (1)", + oldname); + return NULL; + } + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "The '%s' keyword parameter name is deprecated. " + "Use 'string' instead.", oldname) < 0) + return NULL; + return string2; + } + if (string == NULL) { + PyErr_SetString(PyExc_TypeError, + "Required argument 'string' (pos 1) not found"); + return NULL; + } + return string; +} + +static PyObject* pattern_match(PatternObject* self, PyObject* args, PyObject* kw) { SRE_STATE state; int status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:match", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:match", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -1919,12 +1947,16 @@ SRE_STATE state; int status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:search", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:search", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -2052,12 +2084,16 @@ int status; Py_ssize_t i, b, e; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:findall", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:findall", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -2180,11 +2216,15 @@ Py_ssize_t i; void* last; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t maxsplit = 0; - static char* kwlist[] = { "source", "maxsplit", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|n:split", kwlist, - &string, &maxsplit)) + static char* kwlist[] = { "string", "maxsplit", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|On$O:split", kwlist, + &string, &maxsplit, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); @@ -3882,12 +3922,16 @@ ScannerObject* self; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:scanner", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:scanner", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; /* create scanner object */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 10:37:29 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 6 Mar 2014 10:37:29 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320283=3A_RE_pattern_methods_now_accept_the_stri?= =?utf-8?q?ng_keyword_parameters?= Message-ID: <3ffn1P345Nz7M0F@mail.python.org> http://hg.python.org/cpython/rev/f4d7abcf8080 changeset: 89483:f4d7abcf8080 parent: 89480:56c346e9ae4d parent: 89482:52743dc788e6 user: Serhiy Storchaka date: Thu Mar 06 11:36:15 2014 +0200 summary: Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. files: Lib/test/test_re.py | 18 +++ Misc/NEWS | 4 + Modules/_sre.c | 145 +++++++++++++++++-------------- 3 files changed, 102 insertions(+), 65 deletions(-) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1205,6 +1205,24 @@ self.assertEqual(out.getvalue().splitlines(), ['literal 102 ', 'literal 111 ', 'literal 111 ']) + def test_keyword_parameters(self): + # Issue #20283: Accepting the string keyword parameter. + pat = re.compile(r'(ab)') + self.assertEqual( + pat.match(string='abracadabra', pos=7, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.fullmatch(string='abracadabra', pos=7, endpos=9).span(), (7, 9)) + self.assertEqual( + pat.search(string='abracadabra', pos=3, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.findall(string='abracadabra', pos=3, endpos=10), ['ab']) + self.assertEqual( + pat.split(string='abracadabra', maxsplit=1), + ['', 'ab', 'racadabra']) + self.assertEqual( + pat.scanner(string='abracadabra', pos=3, endpos=10).search().span(), + (7, 9)) + class PatternReprTests(unittest.TestCase): def check(self, pattern, expected): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,10 @@ Library ------- +- Issue #20283: RE pattern methods now accept the string keyword parameters + as documented. The pattern and source keyword parameters are left as + deprecated aliases. + - Issue #20839: Don't trigger a DeprecationWarning in the still supported pkgutil.get_loader() API when __loader__ isn't set on a module (nor when pkgutil.find_loader() is called directly). diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -526,59 +526,49 @@ return sre_ucs4_search(state, pattern); } -/*[clinic input] -module _sre -class _sre.SRE_Pattern "PatternObject *" "&Pattern_Type" - -_sre.SRE_Pattern.match as pattern_match - - pattern: object - pos: Py_ssize_t = 0 - endpos: Py_ssize_t(c_default="PY_SSIZE_T_MAX") = sys.maxsize - -Matches zero or more characters at the beginning of the string. -[clinic start generated code]*/ - -PyDoc_STRVAR(pattern_match__doc__, -"match($self, /, pattern, pos=0, endpos=sys.maxsize)\n" -"--\n" -"\n" -"Matches zero or more characters at the beginning of the string."); - -#define PATTERN_MATCH_METHODDEF \ - {"match", (PyCFunction)pattern_match, METH_VARARGS|METH_KEYWORDS, pattern_match__doc__}, - static PyObject * -pattern_match_impl(PatternObject *self, PyObject *pattern, Py_ssize_t pos, Py_ssize_t endpos); +fix_string_param(PyObject *string, PyObject *string2, const char *oldname) +{ + if (string2 != NULL) { + if (string != NULL) { + PyErr_Format(PyExc_TypeError, + "Argument given by name ('%s') and position (1)", + oldname); + return NULL; + } + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "The '%s' keyword parameter name is deprecated. " + "Use 'string' instead.", oldname) < 0) + return NULL; + return string2; + } + if (string == NULL) { + PyErr_SetString(PyExc_TypeError, + "Required argument 'string' (pos 1) not found"); + return NULL; + } + return string; +} static PyObject * pattern_match(PatternObject *self, PyObject *args, PyObject *kwargs) { - PyObject *return_value = NULL; - static char *_keywords[] = {"pattern", "pos", "endpos", NULL}; - PyObject *pattern; + static char *_keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + PyObject *string = NULL; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; - - if (!PyArg_ParseTupleAndKeywords(args, kwargs, - "O|nn:match", _keywords, - &pattern, &pos, &endpos)) - goto exit; - return_value = pattern_match_impl(self, pattern, pos, endpos); - -exit: - return return_value; -} - -static PyObject * -pattern_match_impl(PatternObject *self, PyObject *pattern, Py_ssize_t pos, Py_ssize_t endpos) -/*[clinic end generated code: output=1528eafdb8b025ad input=26f9fd31befe46b9]*/ -{ + PyObject *pattern = NULL; SRE_STATE state; Py_ssize_t status; - PyObject *string; - - string = state_init(&state, (PatternObject *)self, pattern, pos, endpos); + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, + "|Onn$O:match", _keywords, + &string, &pos, &endpos, &pattern)) + return NULL; + string = fix_string_param(string, pattern, "pattern"); + if (!string) + return NULL; + string = state_init(&state, (PatternObject *)self, string, pos, endpos); if (!string) return NULL; @@ -603,12 +593,16 @@ SRE_STATE state; Py_ssize_t status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:fullmatch", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:fullmatch", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -637,12 +631,16 @@ SRE_STATE state; Py_ssize_t status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:search", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:search", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -718,12 +716,16 @@ Py_ssize_t status; Py_ssize_t i, b, e; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:findall", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:findall", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -840,11 +842,15 @@ Py_ssize_t i; void* last; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t maxsplit = 0; - static char* kwlist[] = { "source", "maxsplit", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|n:split", kwlist, - &string, &maxsplit)) + static char* kwlist[] = { "string", "maxsplit", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|On$O:split", kwlist, + &string, &maxsplit, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); @@ -1292,6 +1298,10 @@ return result; } +PyDoc_STRVAR(pattern_match_doc, +"match(string[, pos[, endpos]]) -> match object or None.\n\ + Matches zero or more characters at the beginning of the string"); + PyDoc_STRVAR(pattern_fullmatch_doc, "fullmatch(string[, pos[, endpos]]) -> match object or None.\n\ Matches against all of the string"); @@ -1329,7 +1339,8 @@ PyDoc_STRVAR(pattern_doc, "Compiled regular expression objects"); static PyMethodDef pattern_methods[] = { - PATTERN_MATCH_METHODDEF + {"match", (PyCFunction) pattern_match, METH_VARARGS|METH_KEYWORDS, + pattern_match_doc}, {"fullmatch", (PyCFunction) pattern_fullmatch, METH_VARARGS|METH_KEYWORDS, pattern_fullmatch_doc}, {"search", (PyCFunction) pattern_search, METH_VARARGS|METH_KEYWORDS, @@ -2654,12 +2665,16 @@ ScannerObject* self; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:scanner", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "source", NULL }; + if (!PyArg_ParseTupleAndKeywords(args, kw, "|Onn$O:scanner", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; /* create scanner object */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 11:25:56 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 6 Mar 2014 11:25:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwMjgz?= =?utf-8?q?=3A_RE_pattern_methods_now_accept_the_string_keyword_parameters?= Message-ID: <3ffp5J3w3lz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/52256a5861fa changeset: 89484:52256a5861fa branch: 2.7 parent: 89481:1be39dc4409a user: Serhiy Storchaka date: Thu Mar 06 12:24:29 2014 +0200 summary: Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. files: Lib/test/test_re.py | 13 ++++ Misc/NEWS | 4 + Modules/_sre.c | 97 +++++++++++++++++++++++++++----- 3 files changed, 98 insertions(+), 16 deletions(-) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -933,6 +933,19 @@ self.assertEqual(out.getvalue().splitlines(), ['literal 102', 'literal 111', 'literal 111']) + def test_keyword_parameters(self): + # Issue #20283: Accepting the string keyword parameter. + pat = re.compile(r'(ab)') + self.assertEqual( + pat.match(string='abracadabra', pos=7, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.search(string='abracadabra', pos=3, endpos=10).span(), (7, 9)) + self.assertEqual( + pat.findall(string='abracadabra', pos=3, endpos=10), ['ab']) + self.assertEqual( + pat.split(string='abracadabra', maxsplit=1), + ['', 'ab', 'racadabra']) + def run_re_tests(): from test.re_tests import tests, SUCCEED, FAIL, SYNTAX_ERROR diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,10 @@ Library ------- +- Issue #20283: RE pattern methods now accept the string keyword parameters + as documented. The pattern and source keyword parameters are left as + deprecated aliases. + - Issue #15618: Make turtle.py compatible with 'from __future__ import unicode_literals'. Initial patch by Juancarlo A?ez. diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -1875,18 +1875,62 @@ PyObject_DEL(self); } +static int +check_args_size(const char *name, PyObject* args, PyObject* kw, int n) +{ + Py_ssize_t m = PyTuple_GET_SIZE(args) + (kw ? PyDict_Size(kw) : 0); + if (m <= n) + return 1; + PyErr_Format(PyExc_TypeError, + "%s() takes at most %d positional arguments (%zd given)", + name, n, m); + return 0; +} + +static PyObject* +fix_string_param(PyObject *string, PyObject *string2, const char *oldname) +{ + if (string2 != NULL) { + char buf[100]; + if (string != NULL) { + PyErr_Format(PyExc_TypeError, + "Argument given by name ('%s') and position (1)", + oldname); + return NULL; + } + sprintf(buf, "The '%s' keyword parameter name is deprecated. " + "Use 'string' instead.", oldname); + if (PyErr_Warn(PyExc_DeprecationWarning, buf) < 0) + return NULL; + return string2; + } + if (string == NULL) { + PyErr_SetString(PyExc_TypeError, + "Required argument 'string' (pos 1) not found"); + return NULL; + } + return string; +} + static PyObject* pattern_match(PatternObject* self, PyObject* args, PyObject* kw) { SRE_STATE state; int status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:match", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!check_args_size("match", args, kw, 3)) + return NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kw, "|OnnO:match", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -1920,12 +1964,19 @@ SRE_STATE state; int status; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:search", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "pattern", NULL }; + if (!check_args_size("search", args, kw, 3)) + return NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kw, "|OnnO:search", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "pattern"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -2055,12 +2106,19 @@ int status; Py_ssize_t i, b, e; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:findall", kwlist, - &string, &start, &end)) + static char* kwlist[] = { "string", "pos", "endpos", "source", NULL }; + if (!check_args_size("findall", args, kw, 3)) + return NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kw, "|OnnO:findall", kwlist, + &string, &start, &end, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, start, end); @@ -2185,11 +2243,18 @@ Py_ssize_t i; void* last; - PyObject* string; + PyObject *string = NULL, *string2 = NULL; Py_ssize_t maxsplit = 0; - static char* kwlist[] = { "source", "maxsplit", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|n:split", kwlist, - &string, &maxsplit)) + static char* kwlist[] = { "string", "maxsplit", "source", NULL }; + if (!check_args_size("split", args, kw, 2)) + return NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kw, "|OnO:split", kwlist, + &string, &maxsplit, &string2)) + return NULL; + + string = fix_string_param(string, string2, "source"); + if (!string) return NULL; string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 16:09:34 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 16:09:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_Windows_build_?= =?utf-8?q?uses_ASLR_and_DEP=2E?= Message-ID: <3ffwNZ5Kysz7LjX@mail.python.org> http://hg.python.org/cpython/rev/285ae56e4713 changeset: 89485:285ae56e4713 parent: 89483:f4d7abcf8080 user: R David Murray date: Mon Mar 03 08:26:49 2014 -0500 summary: whatsnew: Windows build uses ASLR and DEP. files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1547,6 +1547,11 @@ values from :envvar:`PYTHONIOENCODING` if it exists. (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`.) +* The Windows build now uses `Address Space Layout Randomization + `_ and `Data Execution Prevention + `_. (Contributed by + Christian Heimes in :issue:`16632`.) + .. _other-improvements-3.4: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 16:09:36 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 16:09:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_tweak_a_couple?= =?utf-8?q?_importlib_porting_notes=2E?= Message-ID: <3ffwNc0D59z7Lk2@mail.python.org> http://hg.python.org/cpython/rev/78f063002259 changeset: 89486:78f063002259 user: R David Murray date: Mon Mar 03 08:56:43 2014 -0500 summary: whatsnew: tweak a couple importlib porting notes. I had missed that there was already an entry for the Frozen package change, so I combined mine with the existing one. Also added the info about the reload change to the entry for the other changes to reload, but I'm missing an issue number for the rest of the attribute behavior changes so I left myself an XXX as a reminder to find it. files: Doc/whatsnew/3.4.rst | 15 +++++++-------- Misc/NEWS | 2 +- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1883,10 +1883,15 @@ * Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. + Note that this restores a pre-3.3 behavior in that it means a module is + re-found when re-loaded (:issue:`19413` and XXX). * Frozen packages no longer set ``__path__`` to a list containing the package - name but an empty list instead. Determing if a module is a package should be - done using ``hasattr(module, '__path__')``. + name, they now set it to an empty list. The previous behavior could cause + the import system to do the wrong thing on submodule imports if there was + also a directory with the same name as the frozen package. The correct way + to determine if a module is a package or not is to use``hasattr(module, + '__path__')`` (:issue:`18065`). * :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path it would write to is a symlink or a non-regular file. This is to act as a @@ -1984,12 +1989,6 @@ similarly to ``__kwdefaults__``. (Contributed by Yury Selivanov in :issue:`20625`). -* Frozen packages used to have their ``__path__`` set to the package name, - but this could cause the import system to do the wrong thing on submodule - imports if there was also a directory with the same name as the frozen - package. The ``__path__`` for frozen packages is now set to ``[]`` - (:issue:`18065`). - * :attr:`hashlib.hash.name` now always returns the identifier in lower case. Previously some builtin hashes had uppercase names, but now that it is a formal public interface the naming has been made consistent (:issue:`18532`). diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1463,7 +1463,7 @@ - Issue #19085: Added basic tests for all tkinter widget options. -- Issue 19384: Fix test_py_compile for root user, patch by Claudiu Popa. +- Issue #19384: Fix test_py_compile for root user, patch by Claudiu Popa. Documentation ------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 16:09:37 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 16:09:37 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_improve_PyZipF?= =?utf-8?q?ile_filterfuc_entry=2C_and_its_docs_=28=2319274=29=2E?= Message-ID: <3ffwNd2hS6z7Lkj@mail.python.org> http://hg.python.org/cpython/rev/064ee489982e changeset: 89487:064ee489982e user: R David Murray date: Wed Mar 05 10:48:01 2014 -0500 summary: whatsnew: improve PyZipFile filterfuc entry, and its docs (#19274). files: Doc/library/zipfile.rst | 26 ++++++++++++++++++++------ Doc/whatsnew/3.4.rst | 12 +++++++----- 2 files changed, 27 insertions(+), 11 deletions(-) diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -401,18 +401,32 @@ ``2``, only files with that optimization level (see :func:`compile`) are added to the archive, compiling if necessary. - If the pathname is a file, the filename must end with :file:`.py`, and + If *pathname* is a file, the filename must end with :file:`.py`, and just the (corresponding :file:`\*.py[co]`) file is added at the top level - (no path information). If the pathname is a file that does not end with + (no path information). If *pathname* is a file that does not end with :file:`.py`, a :exc:`RuntimeError` will be raised. If it is a directory, and the directory is not a package directory, then all the files :file:`\*.py[co]` are added at the top level. If the directory is a package directory, then all :file:`\*.py[co]` are added under the package name as a file path, and if any subdirectories are package directories, - all of these are added recursively. *basename* is intended for internal - use only. When *filterfunc(pathname)* is given, it will be called for every - invocation. When it returns a false value, that path and its subpaths will - be ignored. + all of these are added recursively. + + *basename* is intended for internal use only. + + *filterfunc*, if given, must be a function taking a single string + argument. It will be passed each path (including each individual full + file path) before it is added to the archive. If *filterfunc* returns a + false value, the path will not be added, and if it is a directory its + contents will be ignored. For example, if our test files are all either + in ``test`` directories or start with the string ``test_``, we can use a + *filterfunc* to exclude them:: + + >>> zf = PyZipFile('myprog.zip') + >>> def notests(s): + ... fn = os.path.basename(s) + ... return (not (fn == 'test' or fn.startswith('test_'))) + >>> zf.writepy('myprog', filterfunc=notests) + The :meth:`writepy` method makes archives with file names like this:: diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1415,11 +1415,13 @@ Ariel Poliak and Serhiy Storchaka in :issue:`14377`.) -zipfile.PyZipfile ------------------ - -Add a filter function to ignore some packages (tests for instance), -:meth:`~zipfile.PyZipFile.writepy`. +zipfile +------- + +The :meth:`~zipfile.PyZipFile.writepy` method of the +:class:`~zipfile.PyZipFile` class has a new *filterfunc* option that can be +used to control which directories and files are added to the archive. For +example, this could be used to exclude test files from the archive. (Contributed by Christian Tismer in :issue:`19274`.) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 17:07:02 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 6 Mar 2014 17:07:02 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_tracemalloc=2Epy=3A_fix_in?= =?utf-8?q?dentation?= Message-ID: <3ffxft34mtz7LjW@mail.python.org> http://hg.python.org/cpython/rev/56fbf7a7173a changeset: 89488:56fbf7a7173a user: Victor Stinner date: Thu Mar 06 17:06:04 2014 +0100 summary: tracemalloc.py: fix indentation files: Lib/tracemalloc.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/tracemalloc.py b/Lib/tracemalloc.py --- a/Lib/tracemalloc.py +++ b/Lib/tracemalloc.py @@ -119,12 +119,12 @@ previous = old_group.pop(traceback, None) if previous is not None: stat = StatisticDiff(traceback, - stat.size, stat.size - previous.size, - stat.count, stat.count - previous.count) + stat.size, stat.size - previous.size, + stat.count, stat.count - previous.count) else: stat = StatisticDiff(traceback, - stat.size, stat.size, - stat.count, stat.count) + stat.size, stat.size, + stat.count, stat.count) statistics.append(stat) for traceback, stat in old_group.items(): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 17:07:03 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 6 Mar 2014 17:07:03 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_tracemalloc=2Epy=3A_add_co?= =?utf-8?q?mments_to_show_the_format_of_tuples?= Message-ID: <3ffxfv4y0Zz7LkD@mail.python.org> http://hg.python.org/cpython/rev/b07400659dba changeset: 89489:b07400659dba user: Victor Stinner date: Thu Mar 06 17:06:41 2014 +0100 summary: tracemalloc.py: add comments to show the format of tuples files: Lib/tracemalloc.py | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Lib/tracemalloc.py b/Lib/tracemalloc.py --- a/Lib/tracemalloc.py +++ b/Lib/tracemalloc.py @@ -141,6 +141,7 @@ __slots__ = ("_frame",) def __init__(self, frame): + # frame is a tuple: (filename: str, lineno: int) self._frame = frame @property @@ -177,6 +178,8 @@ def __init__(self, frames): Sequence.__init__(self) + # frames is a tuple of frame tuples: see Frame constructor for the + # format of a frame tuple self._frames = frames def __len__(self): @@ -241,6 +244,8 @@ __slots__ = ("_trace",) def __init__(self, trace): + # trace is a tuple: (size, traceback), see Traceback constructor + # for the format of the traceback tuple self._trace = trace @property @@ -268,6 +273,7 @@ class _Traces(Sequence): def __init__(self, traces): Sequence.__init__(self) + # traces is a tuple of trace tuples: see Trace constructor self._traces = traces def __len__(self): @@ -338,6 +344,8 @@ """ def __init__(self, traces, traceback_limit): + # traces is a tuple of trace tuples: see _Traces constructor for + # the exact format self.traces = _Traces(traces) self.traceback_limit = traceback_limit -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 17:44:33 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 17:44:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2311558=3A_Better_message?= =?utf-8?q?_if_attach_called_on_non-multipart=2E?= Message-ID: <3ffyV920zGz7LjX@mail.python.org> http://hg.python.org/cpython/rev/302c8fdb17e3 changeset: 89490:302c8fdb17e3 user: R David Murray date: Thu Mar 06 11:44:17 2014 -0500 summary: #11558: Better message if attach called on non-multipart. Original patch by Varun Sharma. files: Lib/email/message.py | 6 +++++- Lib/test/test_email/test_email.py | 8 ++++++++ Misc/ACKS | 1 + Misc/NEWS | 4 ++++ 4 files changed, 18 insertions(+), 1 deletions(-) diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -203,7 +203,11 @@ if self._payload is None: self._payload = [payload] else: - self._payload.append(payload) + try: + self._payload.append(payload) + except AttributeError: + raise TypeError("Attach is not valid on a message with a" + " non-multipart payload") def get_payload(self, i=None, decode=False): """Return a reference to the payload. diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -124,6 +124,14 @@ msg.set_payload([]) self.assertEqual(msg.get_payload(), []) + def test_attach_when_payload_is_string(self): + msg = Message() + msg['Content-Type'] = 'multipart/mixed' + msg.set_payload('string payload') + sub_msg = MIMEMessage(Message()) + self.assertRaisesRegex(TypeError, "[Aa]ttach.*non-multipart", + msg.attach, sub_msg) + def test_get_charsets(self): eq = self.assertEqual diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1188,6 +1188,7 @@ Ha Shao Mark Shannon Richard Shapiro +Varun Sharma Vlad Shcherbina Justin Sheehy Charlie Shepherd diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,10 @@ Library ------- +- Issue #11558: ``email.message.Message.attach`` now returns a more + useful error message if ``attach`` is called on a message for which + ``is_multipart`` is False. + - Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 19:41:11 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 19:41:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_ipaddress_is?= =?utf-8?q?=5Fglobal?= Message-ID: <3fg14l6Lk2z7LjR@mail.python.org> http://hg.python.org/cpython/rev/fa3efccb9888 changeset: 89491:fa3efccb9888 user: R David Murray date: Thu Mar 06 11:51:37 2014 -0500 summary: whatsnew: ipaddress is_global Also tweaked the docs a bit and added the missing entry for the ipv6 is_global. files: Doc/library/ipaddress.rst | 18 +++++++++++++----- Doc/whatsnew/3.4.rst | 4 ++++ 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -154,16 +154,16 @@ .. attribute:: is_private ``True`` if the address is allocated for private networks. See - iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry + iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ (for IPv6). .. attribute:: is_global ``True`` if the address is allocated for public networks. See - iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry + iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ (for IPv6). - .. versionadded:: 3.4 + .. versionadded:: 3.4 .. attribute:: is_unspecified @@ -184,6 +184,9 @@ ``True`` if the address is reserved for link-local usage. See :RFC:`3927`. +.. _iana-ipv4-special-registry: http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml +.. _iana-ipv6-special-registry: http://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml + .. class:: IPv6Address(address) @@ -218,18 +221,23 @@ The long form of the address representation, with all leading zeroes and groups consisting entirely of zeroes included. + + For the following attributes, see the corresponding documention of the + :class:`IPv4Address` class: + .. attribute:: packed .. attribute:: version .. attribute:: max_prefixlen .. attribute:: is_multicast .. attribute:: is_private + .. attribute:: is_global .. attribute:: is_unspecified .. attribute:: is_reserved .. attribute:: is_loopback .. attribute:: is_link_local - Refer to the corresponding attribute documentation in - :class:`IPv4Address` + .. versionadded:: 3.4 + is_global .. attribute:: is_site_local diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -901,6 +901,10 @@ by the normal standard library requirements to maintain backwards compatibility. +A new :attr:`~ipaddress.IPv4Address.is_global` property is ``True`` if +an address is globally routeable. (Contributed by Peter Moody in +:issue:`17400`.) + logging ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 19:41:13 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 19:41:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_rewrite_resour?= =?utf-8?q?ce_entry=3A_prlimit_and_new_constants=2E?= Message-ID: <3fg14n0sLDz7LjR@mail.python.org> http://hg.python.org/cpython/rev/4f946e19f016 changeset: 89492:4f946e19f016 user: R David Murray date: Thu Mar 06 12:47:31 2014 -0500 summary: whatsnew: rewrite resource entry: prlimit and new constants. files: Doc/whatsnew/3.4.rst | 17 +++++++++++++++-- 1 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1101,8 +1101,21 @@ resource -------- -New :func:`resource.prlimit` function and Linux specific constants. -(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.) +New :func:`~resource.prlimit` function, available on Linux platforms with a +kernel version of 2.6.36 or later and glibc of 2.13 or later, provides the +ability to query or set the resource limits for processes other than the one +making the call. (Contributed by Christian Heimes in :issue:`16595`.) + +On Linux kernel version 2.6.36 or later, there are there are also some new +Linux specific constants: :attr:`~resource.RLIMIT_MSGQUEUE`, +:attr:`~resource.RLIMIT_NICE`, :attr:`~resource.RLIMIT_RTPRIO`, +:attr:`~resource.RLIMIT_RTTIME`, and :attr:`~resource.RLIMIT_SIGPENDING`. +(Contributed by Christian Heimes in :issue:`19324`.) + +On FreeBSD version 9 and later, there some new FreeBSD specific constants: +:attr:`~resource.RLIMIT_SBSIZE`, :attr:`~resource.RLIMIT_SWAP`, and +:attr:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in +:issue:`19343`.) select -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 19:41:14 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 19:41:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_redundant_NEWS_entr?= =?utf-8?q?y=2E?= Message-ID: <3fg14p31jYz7Llq@mail.python.org> http://hg.python.org/cpython/rev/fe2042d02d85 changeset: 89493:fe2042d02d85 user: R David Murray date: Thu Mar 06 12:58:25 2014 -0500 summary: Remove redundant NEWS entry. files: Misc/NEWS | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1419,8 +1419,6 @@ - Issue #19324: Expose Linux-specific constants in resource module. -- Issue #17400: ipaddress should make it easy to identify rfc6598 addresses. - - Load SSL's error strings in hashlib. - Issue #18527: Upgrade internal copy of zlib to 1.2.8. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 19:41:15 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 6 Mar 2014 19:41:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_note_in_deprec?= =?utf-8?q?ation_section_about_html=2Eparser_strict=2E?= Message-ID: <3fg14q5J0Pz7LkY@mail.python.org> http://hg.python.org/cpython/rev/964c4899efc4 changeset: 89494:964c4899efc4 user: R David Murray date: Thu Mar 06 13:40:53 2014 -0500 summary: whatsnew: note in deprecation section about html.parser strict. files: Doc/whatsnew/3.4.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1759,6 +1759,8 @@ not have been exposed and is hopefully not in use; it is deprecated and will mostly likely be removed in Python 3.5. +* The *strict* argument of :class:`~html.parser.HTMLParser` is deprecated. + Deprecated Functions and Types in the C API ------------------------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 20:28:34 2014 From: python-checkins at python.org (zach.ware) Date: Thu, 6 Mar 2014 20:28:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Make_pywlaunch?= =?utf-8?q?er=2Evcxproj_depend_on_make=5Fversioninfo=2Evcxproj=2E?= Message-ID: <3fg27Q6N92z7Ljh@mail.python.org> http://hg.python.org/cpython/rev/7a7ff2015099 changeset: 89495:7a7ff2015099 branch: 3.3 parent: 89482:52743dc788e6 user: Zachary Ware date: Thu Mar 06 13:21:45 2014 -0600 summary: Make pywlauncher.vcxproj depend on make_versioninfo.vcxproj. pylauncher.vcxproj already has the same dependency, and adding it to the pywlauncher project eliminates the possibility of the pywlauncher build failing when building the solution in parallel. In particular, pywlauncher may fail to build when doing a "Rebuild Solution" from the VS Build menu. files: PCbuild/pywlauncher.vcxproj | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/PCbuild/pywlauncher.vcxproj b/PCbuild/pywlauncher.vcxproj --- a/PCbuild/pywlauncher.vcxproj +++ b/PCbuild/pywlauncher.vcxproj @@ -240,6 +240,11 @@ + + + {f0e0541e-f17d-430b-97c4-93adf0dd284e} + + -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 6 20:28:36 2014 From: python-checkins at python.org (zach.ware) Date: Thu, 6 Mar 2014 20:28:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Make_pywlauncher=2Evcxproj_depend_on_make=5Fversioninfo?= =?utf-8?b?LnZjeHByb2ou?= Message-ID: <3fg27S19fHz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/47f37a688c4c changeset: 89496:47f37a688c4c parent: 89494:964c4899efc4 parent: 89495:7a7ff2015099 user: Zachary Ware date: Thu Mar 06 13:28:08 2014 -0600 summary: Make pywlauncher.vcxproj depend on make_versioninfo.vcxproj. pylauncher.vcxproj already has the same dependency, and adding it to the pywlauncher project eliminates the possibility of the pywlauncher build failing when building the solution in parallel. In particular, pywlauncher may fail to build when doing a "Rebuild Solution" from the VS Build menu. Merge with 3.3. files: PCbuild/pywlauncher.vcxproj | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/PCbuild/pywlauncher.vcxproj b/PCbuild/pywlauncher.vcxproj --- a/PCbuild/pywlauncher.vcxproj +++ b/PCbuild/pywlauncher.vcxproj @@ -240,6 +240,11 @@ + + + {f0e0541e-f17d-430b-97c4-93adf0dd284e} + + -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 7 09:37:18 2014 From: python-checkins at python.org (larry.hastings) Date: Fri, 7 Mar 2014 09:37:18 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_New_hacks_for_threefourtoo?= =?utf-8?q?l!__Now_it_has=2E=2E=2E_*auto-graft*!?= Message-ID: <3fgMdV1M3rz7Ljh@mail.python.org> http://hg.python.org/release/rev/8e3ce80c9f24 changeset: 80:8e3ce80c9f24 user: Larry Hastings date: Fri Mar 07 00:37:13 2014 -0800 summary: New hacks for threefourtool! Now it has... *auto-graft*! files: 3.4/threefourtool.py | 86 ++++++++++++++++++++++++------- 1 files changed, 65 insertions(+), 21 deletions(-) diff --git a/3.4/threefourtool.py b/3.4/threefourtool.py --- a/3.4/threefourtool.py +++ b/3.4/threefourtool.py @@ -28,7 +28,7 @@ def system(s): print(s) - os.system(s) + return os.system(s) def yes_no(): while True: @@ -124,7 +124,7 @@ current_directory = os.getcwd() try: os.chdir("/home/larry/src/python/3.4") - p = subprocess.Popen(["/usr/bin/hg", "log", "-v"], stdout=subprocess.PIPE) + p = subprocess.Popen(["/home/larry/src/hg/hg", "log", "-v"], stdout=subprocess.PIPE) # with open("/home/larry/src/python/logtxt", "rt", encoding="utf-8") as f: # output = f.read() @@ -565,7 +565,7 @@ read_changesets() os.chdir("/home/larry/src/python/3.4") print("Updating to 3.4 branch:") - os.system("hg update -r 3.4") + os.system("/home/larry/src/hg/hg update -r 3.4") print() u = self.unfinished.get print("Picking revisions ", u('picked revisions')) @@ -604,7 +604,7 @@ atexit.register(delete_files, patch_path, commit_message_path) def detect_new_revision(): - output = subprocess.check_output(['/usr/bin/hg', 'summary']).decode('utf-8').split('\n') + output = subprocess.check_output(['/home/larry/src/hg/hg', 'summary']).decode('utf-8').split('\n') line = output[0] assert line.startswith('parent:') line = line[len('parent:'):] @@ -613,6 +613,11 @@ r = r.split()[0].strip() u['threefour picked revision'] = r + show_graft = False + def toggle_graft(): + nonlocal show_graft + show_graft = not show_graft + show_patch = False def toggle_patch(): nonlocal show_patch @@ -625,33 +630,72 @@ print("_" * 79) print() + def remove_orig_and_rej(): + for dirpath, dirnames, filenames in os.walk("."): + for filename in filenames: + if not filename.endswith((".orig", ".rej")): + continue + path = os.path.join(dirpath, filename) + os.unlink(path) + + changes = subprocess.check_output(['/home/larry/src/hg/hg', 'stat', '-mard']).decode('utf-8') + try_auto_graft = not bool(changes) + if try_auto_graft: + print("Attempting auto-graft:") + else: + print("Skipping auto-graft, there are outstanding changes.") + while u.get('default picked revision'): commands = [] - commands.append(("Update to appropriate revision in 3.4 branch", "hg update -r {threefour graft here}")) - commands.append(("Graft revision", "hg graft {default picked revision}")) + commands.append(("Update to appropriate revision in 3.4 branch", "/home/larry/src/hg/hg update -r {threefour graft here}")) + commands.append(("Graft revision", "/home/larry/src/hg/hg graft --tool internal:merge {default picked revision}")) + commands.append(("Handle graft conflict", toggle_graft)) + + if show_graft: + commands.append(("Graft #1: List conflicts", "/home/larry/src/hg/hg diff resolve -l")) + commands.append(("Graft #2: Mark all conflicts as resolved", "/home/larry/src/hg/hg diff resolve -m")) + commands.append(("Graft #3: Resume the graft", "/home/larry/src/hg/hg diff graft --continue")) + commands.append(("Graft #4: Clean up .orig and .rej files", remove_orig_and_rej)) + commands.append(("Patch revision (only if graft fails)", toggle_patch)) - if show_patch: - commands.append(("[graft failed step 1] Generate patch", "/usr/bin/hg diff -r {default diff from} -r {default picked revision} > {patch path}")) - commands.append(("[graft failed step 2] Inspect patch", "{EDITOR} {patch path}")) - commands.append(("[graft failed step 3] Apply patch", "/usr/bin/patch -p1 < {patch path}")) - commands.append(("[graft failed step 4] Check in patch", "/usr/bin/hg ci --user '{user}' --date '{date}' --logfile '{commit message path}'")) + commands.append(("Patch #1: Generate patch", "/home/larry/src/hg/hg diff -r {default diff from} -r {default picked revision} > {patch path}")) + commands.append(("Patch #2: Inspect patch", "{EDITOR} {patch path}")) + commands.append(("Patch #3: Apply patch", "/usr/bin/patch -p1 < {patch path}")) + commands.append(("Patch #4: Check in patch", "/home/larry/src/hg/hg ci --user '{user}' --date '{date}' --logfile '{commit message path}'")) if u.get('threefour rebase from'): commands.append(("Detect new revision", detect_new_revision)) - c = "hg rebase --source {threefour rebase from} --dest {threefour picked revision}" + c = "/home/larry/src/hg/hg rebase --source {threefour rebase from} --dest {threefour picked revision}" commands.append(("Rebase subsequent revisions after grafted revision", c)) - commands.append(("Update to head of 3.4 branch", "hg update -r 3.4")) + commands.append(("Update to head of 3.4 branch", "/home/larry/src/hg/hg update -r 3.4")) commands.append(("Mark revision as picked", mark_as_picked)) - commands.append(("Print details of picked revision", "hg log -r {default picked revision}")) + commands.append(("Print details of picked revision", "/home/larry/src/hg/hg log -v -r {default picked revision}")) print() total = len(u['original picked revisions']) current = len(u['picked revisions']) print("Picking revision {default picked revision}".format_map(u) + " ({}/{}):".format(total-current, total)) + + if try_auto_graft: + for i in (0, 1): + text, cmd = commands[i] + return_code = system(cmd.format_map(u)) + u['commands run'].add(text) + if return_code: + print("*" * 80) + print("*" * 80) + print("** Process return code:", return_code) + print("*" * 80) + print("*" * 80) + break + else: + mark_as_picked() + return + self._run_command(commands, u) def finish(self): @@ -678,7 +722,7 @@ """ Recreate the 3.4 branch from scratch. """ - if (not force) and (yes_no() == 'n'): + if (not force) and (yes_no() != 'y'): sys.exit() os.chdir("/home/larry/src/python") while True: @@ -689,11 +733,11 @@ os.rename("/home/larry/src/python/3.4", "/home/larry/src/python/bad3.4") continue break - os.system("hg clone trunk 3.4") + os.system("/home/larry/src/hg/hg clone trunk 3.4") os.chdir("/home/larry/src/python/3.4") - os.system("hg update -r e64ae8b82672") - os.system("hg branch 3.4") - os.system("hg commit -m 'Created release branch for 3.4.'") + os.system("/home/larry/src/hg/hg update -r e64ae8b82672") + os.system("/home/larry/src/hg/hg branch 3.4") + os.system("/home/larry/src/hg/hg commit -m 'Created release branch for 3.4.'") def tar(self): time = now() @@ -707,10 +751,10 @@ remove_dir(tardir) os.chdir("/home/larry/src/python") - system("hg clone 3.4 " + tardir) + system("/home/larry/src/hg/hg clone 3.4 " + tardir) os.chdir(tardir) - system("hg update -r 3.4") + system("/home/larry/src/hg/hg update -r 3.4") remove_dir(".hg") for prefix in ('.hg', '.bzr', '.git'): for filename in glob.glob(prefix + '*'): -- Repository URL: http://hg.python.org/release From solipsis at pitrou.net Fri Mar 7 09:53:47 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 07 Mar 2014 09:53:47 +0100 Subject: [Python-checkins] Daily reference leaks (47f37a688c4c): sum=0 Message-ID: results for 47f37a688c4c on branch "default" -------------------------------------------- test_site leaked [0, -2, 2] references, sum=0 test_site leaked [0, -2, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog6i6sHX', '-x'] From python-checkins at python.org Fri Mar 7 12:05:26 2014 From: python-checkins at python.org (larry.hastings) Date: Fri, 7 Mar 2014 12:05:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Even_more_updates_for_my_t?= =?utf-8?q?hreefour_tool=2E?= Message-ID: <3fgQwQ0V72z7Ll1@mail.python.org> http://hg.python.org/release/rev/da4d4ef62db4 changeset: 81:da4d4ef62db4 user: Larry Hastings date: Fri Mar 07 03:05:26 2014 -0800 summary: Even more updates for my threefour tool. files: 3.4/dryparse/dryparse.py | 10 ++- 3.4/threefourtool.py | 116 +++++++++++++++++++------- 2 files changed, 92 insertions(+), 34 deletions(-) diff --git a/3.4/dryparse/dryparse.py b/3.4/dryparse/dryparse.py --- a/3.4/dryparse/dryparse.py +++ b/3.4/dryparse/dryparse.py @@ -373,6 +373,7 @@ if is_option and (self.type is not bool) and (not self.value_usage): self.value_usage = option_value_usage_formatter(self.type.__name__) + # print("DryArgument name", repr(name), "type", self.type, "default", repr(default), "annotations", repr(annotations), "is_option", is_option ) class OptionError(RuntimeError): pass @@ -604,7 +605,14 @@ nonlocal needs_value argument, is_bool = analyze_option(option) if is_bool: - argument.value = not argument.value + if value is unspecified: + if argument.value is not unspecified: + value = not argument.value + elif argument.default is not unspecified: + value = not argument.default + else: + value = True + argument.set_value(value) else: if value is unspecified: needs_value = argument diff --git a/3.4/threefourtool.py b/3.4/threefourtool.py --- a/3.4/threefourtool.py +++ b/3.4/threefourtool.py @@ -14,6 +14,7 @@ from dryparse import dryparse import glob import os +import os.path import pprint import pickle import shutil @@ -23,7 +24,22 @@ import time +def which(cmd, path="PATH"): + """Find cmd on PATH.""" + if os.path.exists(cmd): + return cmd + if cmd[0] == '/': + return None + for segment in os.getenv(path).split(":"): + program = os.path.normpath(os.path.join(segment, cmd)) + if os.path.exists(program): + return program + return None + + outgoing = "/home/larry/src/python/34outgoing" +# path_to_hg = "/home/larry/src/hg/hg" +hg = which('hg') def system(s): @@ -124,7 +140,7 @@ current_directory = os.getcwd() try: os.chdir("/home/larry/src/python/3.4") - p = subprocess.Popen(["/home/larry/src/hg/hg", "log", "-v"], stdout=subprocess.PIPE) + p = subprocess.Popen([hg, "log", "-v"], stdout=subprocess.PIPE) # with open("/home/larry/src/python/logtxt", "rt", encoding="utf-8") as f: # output = f.read() @@ -276,6 +292,7 @@ # print(" r", rev, "p", parents, "rev to print", rev_to_print) if rev_to_print in seen: continue + print(fields.get('branch'), "printing r", rev, "rev_to_print", rev_to_print) seen.add(rev_to_print) print_rev(rev_to_print, printer) @@ -301,14 +318,8 @@ branch = fields.get('branch') if branch != '3.4': return False - revs = get_user_date_to_revs(fields) - assert (rev, branch) in revs - # print("is 34?", rev, revs) - for r2, branch in revs: - if not branch: - # print(rev, "->", r2) - return r2 - return False + + return default_from_34.get(rev, False) class Tool: unfinished_filename = "/home/larry/.34unfinished" @@ -340,10 +351,13 @@ def status(self): """ Regenerate the status webpage. + + The status webpage is written to the "34outgoing" directory. """ f = open(outgoing + "/merge.status.html", "wt") read_changesets() + print("changesets in 3.4", default_from_34) def printer(*a): print(*a, file=f) @@ -455,9 +469,15 @@ else: cmd() - def pick(self, picked_revision, *picked_revisions): + def pick(self, picked_revision, *picked_revisions, + rebase:('-r', 'Rebase so that all revisions are in chronological order')=True): """ - Cherry-pick a revision from default to 3.4. + Cherry-pick one or more revisions from default to 3.4. + + You can stop in the middle of a 'pick' session with Ctrl-C. The current + state will be pickled, and you can resume it with the 'finish' command, + or abandon it with the 'abandon' command. You can also resume a pick + session by running 'pick' with exactly the same arguments. """ pr = [picked_revision] pr.extend(picked_revisions) @@ -477,7 +497,8 @@ picked_revisions.sort(key=to_default_index) if self.unfinished: - if self.unfinished['original picked revisions'] == picked_revisions: + if ((self.unfinished['original picked revisions'] == picked_revisions) and + (self.unfinished['rebase'] == rebase)): return self.finish() sys.exit("You have unfinished business!\n\nUse the 'finish' command to finish it,\nor the 'abandon' command to abandon it.") @@ -485,6 +506,7 @@ 'function': '_pick', 'picked revisions': picked_revisions, 'original picked revisions': list(picked_revisions), + 'rebase': rebase, } self.finish() @@ -515,7 +537,9 @@ # print("r", r) if r == r_34_first_revision: break - r_default = default_from_34[r] + r_default = default_from_34.get(r) + if r_default is None: + continue i = branches[None].index(r_default) if i >= index: break @@ -565,7 +589,7 @@ read_changesets() os.chdir("/home/larry/src/python/3.4") print("Updating to 3.4 branch:") - os.system("/home/larry/src/hg/hg update -r 3.4") + os.system(hg + " update -r 3.4") print() u = self.unfinished.get print("Picking revisions ", u('picked revisions')) @@ -584,6 +608,7 @@ u = self.unfinished u['EDITOR'] = os.getenv('EDITOR') + u['hg'] = hg patch_path = "/tmp/patch.{default diff from}.to.{default picked revision}.diff".format_map(u) u['patch path'] = patch_path @@ -604,7 +629,7 @@ atexit.register(delete_files, patch_path, commit_message_path) def detect_new_revision(): - output = subprocess.check_output(['/home/larry/src/hg/hg', 'summary']).decode('utf-8').split('\n') + output = subprocess.check_output([hg, 'summary']).decode('utf-8').split('\n') line = output[0] assert line.startswith('parent:') line = line[len('parent:'):] @@ -638,42 +663,47 @@ path = os.path.join(dirpath, filename) os.unlink(path) - changes = subprocess.check_output(['/home/larry/src/hg/hg', 'stat', '-mard']).decode('utf-8') + changes = subprocess.check_output([hg, 'stat', '-mard']).decode('utf-8') try_auto_graft = not bool(changes) if try_auto_graft: print("Attempting auto-graft:") else: print("Skipping auto-graft, there are outstanding changes.") + rebase = u['rebase'] + while u.get('default picked revision'): commands = [] - commands.append(("Update to appropriate revision in 3.4 branch", "/home/larry/src/hg/hg update -r {threefour graft here}")) - commands.append(("Graft revision", "/home/larry/src/hg/hg graft --tool internal:merge {default picked revision}")) + if rebase: + commands.append(("Update to appropriate revision in 3.4 branch", "{hg} update -r {threefour graft here}")) + else: + commands.append(("Update to current revision in 3.4 branch", "{hg} update -r 3.4")) + commands.append(("Graft revision", "{hg} graft --tool internal:merge {default picked revision}")) commands.append(("Handle graft conflict", toggle_graft)) if show_graft: - commands.append(("Graft #1: List conflicts", "/home/larry/src/hg/hg diff resolve -l")) - commands.append(("Graft #2: Mark all conflicts as resolved", "/home/larry/src/hg/hg diff resolve -m")) - commands.append(("Graft #3: Resume the graft", "/home/larry/src/hg/hg diff graft --continue")) + commands.append(("Graft #1: List conflicts", "{hg} resolve -l")) + commands.append(("Graft #2: Mark all conflicts as resolved", "{hg} resolve -m")) + commands.append(("Graft #3: Resume the graft", "{hg} graft --continue")) commands.append(("Graft #4: Clean up .orig and .rej files", remove_orig_and_rej)) - commands.append(("Patch revision (only if graft fails)", toggle_patch)) + commands.append(("Patch revision (for ungraftable merge revisions)", toggle_patch)) if show_patch: - commands.append(("Patch #1: Generate patch", "/home/larry/src/hg/hg diff -r {default diff from} -r {default picked revision} > {patch path}")) + commands.append(("Patch #1: Generate patch", "{hg} diff -r {default diff from} -r {default picked revision} > {patch path}")) commands.append(("Patch #2: Inspect patch", "{EDITOR} {patch path}")) commands.append(("Patch #3: Apply patch", "/usr/bin/patch -p1 < {patch path}")) - commands.append(("Patch #4: Check in patch", "/home/larry/src/hg/hg ci --user '{user}' --date '{date}' --logfile '{commit message path}'")) + commands.append(("Patch #4: Check in patch", "{hg} ci --user '{user}' --date '{date}' --logfile '{commit message path}'")) if u.get('threefour rebase from'): commands.append(("Detect new revision", detect_new_revision)) - c = "/home/larry/src/hg/hg rebase --source {threefour rebase from} --dest {threefour picked revision}" + c = "{hg} rebase --source {threefour rebase from} --dest {threefour picked revision}" commands.append(("Rebase subsequent revisions after grafted revision", c)) - commands.append(("Update to head of 3.4 branch", "/home/larry/src/hg/hg update -r 3.4")) + commands.append(("Update to head of 3.4 branch", "{hg} update -r 3.4")) commands.append(("Mark revision as picked", mark_as_picked)) - commands.append(("Print details of picked revision", "/home/larry/src/hg/hg log -v -r {default picked revision}")) + commands.append(("Print details of picked revision", "{hg} log -v -r {default picked revision}")) print() total = len(u['original picked revisions']) @@ -691,6 +721,7 @@ print("** Process return code:", return_code) print("*" * 80) print("*" * 80) + try_auto_graft = False break else: mark_as_picked() @@ -699,6 +730,9 @@ self._run_command(commands, u) def finish(self): + """ + Continue unfinished business (e.g. pick). + """ try: if not self.unfinished: sys.exit("No unfinished business!") @@ -713,6 +747,9 @@ self._save() def abandon(self, *, force:('-f',)=False): + """ + Abandon unfinished business (e.g. pick). + """ if not self.unfinished: sys.exit("No unfinished business!") if force or yes_no() == 'y': @@ -733,13 +770,19 @@ os.rename("/home/larry/src/python/3.4", "/home/larry/src/python/bad3.4") continue break - os.system("/home/larry/src/hg/hg clone trunk 3.4") + os.system(hg + " clone trunk 3.4") os.chdir("/home/larry/src/python/3.4") - os.system("/home/larry/src/hg/hg update -r e64ae8b82672") - os.system("/home/larry/src/hg/hg branch 3.4") - os.system("/home/larry/src/hg/hg commit -m 'Created release branch for 3.4.'") + os.system(hg + " update -r e64ae8b82672") + os.system(hg + " branch 3.4") + os.system(hg + " commit -m 'Created release branch for 3.4.'") def tar(self): + """ + Create a tarball from the current 3.4 tree. + + The tarball is written to the "34outgoing" directory. + Its filename is constructed based on the current time. + """ time = now() tarbase = "python_3.4.0rc2_" + time tardir = "/home/larry/src/python/" + tarbase @@ -751,10 +794,10 @@ remove_dir(tardir) os.chdir("/home/larry/src/python") - system("/home/larry/src/hg/hg clone 3.4 " + tardir) + system(hg + " clone 3.4 " + tardir) os.chdir(tardir) - system("/home/larry/src/hg/hg update -r 3.4") + system(hg + " update -r 3.4") remove_dir(".hg") for prefix in ('.hg', '.bzr', '.git'): for filename in glob.glob(prefix + '*'): @@ -765,11 +808,18 @@ remove_dir(tardir) def rsync(self): + """ + Sync the current "34outgoing" directory to the public server. + """ os.chdir(outgoing) system("rsync -av * midwinter.com:public_html/3.4.status") def asyncio(self): + """ + Check that the 'asyncio' files in 3.4 are exactly the same + as the equivalent ones in trunk. + """ for dir in ("Lib/asyncio", "Lib/test/test_asyncio"): os.chdir("/home/larry/src/python/3.4/" + dir) os.system("diff . /home/larry/src/python/trunk/" + dir) -- Repository URL: http://hg.python.org/release From python-checkins at python.org Fri Mar 7 18:27:19 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 7 Mar 2014 18:27:19 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320812=3A_Add_a_sh?= =?utf-8?q?ort_opener_to_the_Python_2/3_porting_HOWTO=2E?= Message-ID: <3fgbP33DXNz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/2a922153463e changeset: 89497:2a922153463e user: Brett Cannon date: Fri Mar 07 12:27:10 2014 -0500 summary: Issue #20812: Add a short opener to the Python 2/3 porting HOWTO. Thanks to Nick Coghlan for the suggestion. files: Doc/howto/pyporting.rst | 31 ++++++++++++++++++++++++++++- 1 files changed, 30 insertions(+), 1 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -26,6 +26,32 @@ For help with porting, you can email the python-porting_ mailing list with questions. +The Short Version +================= + +* Decide what's the oldest version of Python 2 you want to support (if at all) +* Make sure you have a thorough test suite and use continuous integration + testing to make sure you stay compatible with the versions of Python you care + about +* If you have dependencies, check their Python 3 status using caniusepython3 + (`command-line tool `__, + `web app `__) + +With that done, your options are: + +* If you are dropping Python 2 support, use 2to3_ to port to Python 3 +* If you are keeping Python 2 support, then start writing Python 2/3-compatible + code starting **TODAY** + + + If you have dependencies that have not been ported, reach out to them to port + their project while working to make your code compatible with Python 3 so + you're ready when your dependencies are all ported + + If all your dependencies have been ported (or you have none), go ahead and + port to Python 3 + +* If you are creating a new project that wants to have 2/3 compatibility, + code in Python 3 and then backport to Python 2 + Before You Begin ================ @@ -548,7 +574,10 @@ If you have Python 3 code and have little interest in supporting Python 2 you can use 3to2_ to translate from Python 3 code to Python 2 code. This is only -recommended if you don't plan to heavily support Python 2 users. +recommended if you don't plan to heavily support Python 2 users. Otherwise +write your code for Python 3 and then backport as far back as you want. This +is typically easier than going from Python 2 to 3 as you will have worked out +any difficulties with e.g. bytes/strings, etc. Other Resources -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 7 18:29:35 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 7 Mar 2014 18:29:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwODEz?= =?utf-8?q?=3A_Backport_Python_2/3_HOWTO_updates?= Message-ID: <3fgbRg3Fr3z7Ljv@mail.python.org> http://hg.python.org/cpython/rev/a24085e1b1f5 changeset: 89498:a24085e1b1f5 branch: 3.3 parent: 89495:7a7ff2015099 user: Brett Cannon date: Fri Mar 07 12:28:35 2014 -0500 summary: Issue #20813: Backport Python 2/3 HOWTO updates files: Doc/howto/pyporting.rst | 611 +++++++++++---------------- 1 files changed, 245 insertions(+), 366 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -10,258 +10,211 @@ With Python 3 being the future of Python while Python 2 is still in active use, it is good to have your project available for both major releases of - Python. This guide is meant to help you choose which strategy works best - for your project to support both Python 2 & 3 along with how to execute - that strategy. + Python. This guide is meant to help you figure out how best to support both + Python 2 & 3 simultaneously. If you are looking to port an extension module instead of pure Python code, please see :ref:`cporting-howto`. + If you would like to read one core Python developer's take on why Python 3 + came into existence, you can read Nick Coghlan's `Python 3 Q & A`_. -Choosing a Strategy -=================== + If you prefer to read a (free) book on porting a project to Python 3, + consider reading `Porting to Python 3`_ by Lennart Regebro which should cover + much of what is discussed in this HOWTO. -When a project chooses to support both Python 2 & 3, -a decision needs to be made as to how to go about accomplishing that goal. -The chosen strategy will depend on how large the project's existing -codebase is and how much divergence you want from your current Python 2 codebase -(e.g., changing your code to work simultaneously with Python 2 and 3). + For help with porting, you can email the python-porting_ mailing list with + questions. -If you would prefer to maintain a codebase which is semantically **and** -syntactically compatible with Python 2 & 3 simultaneously, you can write -:ref:`use_same_source`. While this tends to lead to somewhat non-idiomatic -code, it does mean you keep a rapid development process for you, the developer. +The Short Version +================= -If your project is brand-new or does not have a large codebase, then you may -want to consider writing/porting :ref:`all of your code for Python 3 -and use 3to2 ` to port your code for Python 2. +* Decide what's the oldest version of Python 2 you want to support (if at all) +* Make sure you have a thorough test suite and use continuous integration + testing to make sure you stay compatible with the versions of Python you care + about +* If you have dependencies, check their Python 3 status using caniusepython3 + (`command-line tool `__, + `web app `__) -Finally, you do have the option of :ref:`using 2to3 ` to translate -Python 2 code into Python 3 code (with some manual help). This can take the -form of branching your code and using 2to3 to start a Python 3 branch. You can -also have users perform the translation at installation time automatically so -that you only have to maintain a Python 2 codebase. +With that done, your options are: -Regardless of which approach you choose, porting is not as hard or -time-consuming as you might initially think. You can also tackle the problem -piece-meal as a good portion of porting is simply updating your code to follow -current best practices in a Python 2/3 compatible way. +* If you are dropping Python 2 support, use 2to3_ to port to Python 3 +* If you are keeping Python 2 support, then start writing Python 2/3-compatible + code starting **TODAY** + + If you have dependencies that have not been ported, reach out to them to port + their project while working to make your code compatible with Python 3 so + you're ready when your dependencies are all ported + + If all your dependencies have been ported (or you have none), go ahead and + port to Python 3 -Universal Bits of Advice ------------------------- +* If you are creating a new project that wants to have 2/3 compatibility, + code in Python 3 and then backport to Python 2 -Regardless of what strategy you pick, there are a few things you should -consider. -One is make sure you have a robust test suite. You need to make sure everything -continues to work, just like when you support a new minor/feature release of -Python. This means making sure your test suite is thorough and is ported -properly between Python 2 & 3. You will also most likely want to use something -like tox_ to automate testing between both a Python 2 and Python 3 interpreter. +Before You Begin +================ -Two, once your project has Python 3 support, make sure to add the proper -classifier on the Cheeseshop_ (PyPI_). To have your project listed as Python 3 -compatible it must have the -`Python 3 classifier `_ -(from -http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/):: +If your project is on the Cheeseshop_/PyPI_, make sure it has the proper +`trove classifiers`_ to signify what versions of Python it **currently** +supports. At minimum you should specify the major version(s), e.g. +``Programming Language :: Python :: 2`` if your project currently only supports +Python 2. It is preferrable that you be as specific as possible by listing every +major/minor version of Python that you support, e.g. if your project supports +Python 2.6 and 2.7, then you want the classifiers of:: - setup( - name='Your Library', - version='1.0', - classifiers=[ - # make sure to use :: Python *and* :: Python :: 3 so - # that pypi can list the package on the python 3 page - 'Programming Language :: Python', - 'Programming Language :: Python :: 3' - ], - packages=['yourlibrary'], - # make sure to add custom_fixers to the MANIFEST.in - include_package_data=True, - # ... - ) + Programming Language :: Python :: 2 + Programming Language :: Python :: 2.6 + Programming Language :: Python :: 2.7 +Once your project supports Python 3 you will want to go back and add the +appropriate classifiers for Python 3 as well. This is important as setting the +``Programming Language :: Python :: 3`` classifier will lead to your project +being listed under the `Python 3 Packages`_ section of PyPI. -Doing so will cause your project to show up in the -`Python 3 packages list -`_. You will know -you set the classifier properly as visiting your project page on the Cheeseshop -will show a Python 3 logo in the upper-left corner of the page. +Make sure you have a robust test suite. You need to +make sure everything continues to work, just like when you support a new +minor/feature release of Python. This means making sure your test suite is +thorough and is ported properly between Python 2 & 3 (consider using coverage_ +to measure that you have effective test coverage). You will also most likely +want to use something like tox_ to automate testing between all of your +supported versions of Python. You will also want to **port your tests first** so +that you can make sure that you detect breakage during the transition. Tests also +tend to be simpler than the code they are testing so it gives you an idea of how +easy it can be to port code. -Three, the six_ project provides a library which helps iron out differences -between Python 2 & 3. If you find there is a sticky point that is a continual -point of contention in your translation or maintenance of code, consider using -a source-compatible solution relying on six. If you have to create your own -Python 2/3 compatible solution, you can use ``sys.version_info[0] >= 3`` as a -guard. - -Four, read all the approaches. Just because some bit of advice applies to one -approach more than another doesn't mean that some advice doesn't apply to other -strategies. This is especially true of whether you decide to use 2to3 or be -source-compatible; tips for one approach almost always apply to the other. - -Five, drop support for older Python versions if possible. `Python 2.5`_ +Drop support for older Python versions if possible. `Python 2.5`_ introduced a lot of useful syntax and libraries which have become idiomatic in Python 3. `Python 2.6`_ introduced future statements which makes compatibility much easier if you are going from Python 2 to 3. -`Python 2.7`_ continues the trend in the stdlib. So choose the newest version +`Python 2.7`_ continues the trend in the stdlib. Choose the newest version of Python which you believe can be your minimum support version and work from there. -Six, target the newest version of Python 3 that you can. Beyond just the usual +Target the newest version of Python 3 that you can. Beyond just the usual bugfixes, compatibility has continued to improve between Python 2 and 3 as time -has passed. This is especially true for Python 3.3 where the ``u`` prefix for -strings is allowed, making source-compatible Python code easier. +has passed. E.g. Python 3.3 added back the ``u`` prefix for +strings, making source-compatible Python code easier to write. -Seven, make sure to look at the `Other Resources`_ for tips from other people -which may help you out. +Writing Source-Compatible Python 2/3 Code +========================================= -.. _tox: http://codespeak.net/tox/ -.. _Cheeseshop: -.. _PyPI: http://pypi.python.org/ -.. _six: http://packages.python.org/six -.. _Python 2.7: http://www.python.org/2.7.x -.. _Python 2.6: http://www.python.org/2.6.x -.. _Python 2.5: http://www.python.org/2.5.x -.. _Python 2.4: http://www.python.org/2.4.x -.. _Python 2.3: http://www.python.org/2.3.x -.. _Python 2.2: http://www.python.org/2.2.x +Over the years the Python community has discovered that the easiest way to +support both Python 2 and 3 in parallel is to write Python code that works in +either version. While this might sound counter-intuitive at first, it actually +is not difficult and typically only requires following some select +(non-idiomatic) practices and using some key projects to help make bridging +between Python 2 and 3 easier. +Projects to Consider +-------------------- -.. _use_3to2: +The lowest level library for suppoting Python 2 & 3 simultaneously is six_. +Reading through its documentation will give you an idea of where exactly the +Python language changed between versions 2 & 3 and thus what you will want the +library to help you continue to support. -Python 3 and 3to2 -================= +To help automate porting your code over to using six, you can use +modernize_. This project will attempt to rewrite your code to be as modern as +possible while using six to smooth out any differences between Python 2 & 3. -If you are starting a new project or your codebase is small enough, you may -want to consider writing your code for Python 3 and backporting to Python 2 -using 3to2_. Thanks to Python 3 being more strict about things than Python 2 -(e.g., bytes vs. strings), the source translation can be easier and more -straightforward than from Python 2 to 3. Plus it gives you more direct -experience developing in Python 3 which, since it is the future of Python, is a -good thing long-term. +If you want to write your compatible code to feel more like Python 3 there is +the future_ project. It tries to provide backports of objects from Python 3 so +that you can use them from Python 2-compatible code, e.g. replacing the +``bytes`` type from Python 2 with the one from Python 3. +It also provides a translation script like modernize (its translation code is +actually partially based on it) to help start working with a pre-existing code +base. It is also unique in that its translation script will also port Python 3 +code backwards as well as Python 2 code forwards. -A drawback of this approach is that 3to2 is a third-party project. This means -that the Python core developers (and thus this guide) can make no promises -about how well 3to2 works at any time. There is nothing to suggest, though, -that 3to2 is not a high-quality project. +Tips & Tricks +------------- -.. _3to2: https://bitbucket.org/amentajo/lib3to2/overview - - -.. _use_2to3: - -Python 2 and 2to3 -================= - -Included with Python since 2.6, the 2to3_ tool (and :mod:`lib2to3` module) -helps with porting Python 2 to Python 3 by performing various source -translations. This is a perfect solution for projects which wish to branch -their Python 3 code from their Python 2 codebase and maintain them as -independent codebases. You can even begin preparing to use this approach -today by writing future-compatible Python code which works cleanly in -Python 2 in conjunction with 2to3; all steps outlined below will work -with Python 2 code up to the point when the actual use of 2to3 occurs. - -Use of 2to3 as an on-demand translation step at install time is also possible, -preventing the need to maintain a separate Python 3 codebase, but this approach -does come with some drawbacks. While users will only have to pay the -translation cost once at installation, you as a developer will need to pay the -cost regularly during development. If your codebase is sufficiently large -enough then the translation step ends up acting like a compilation step, -robbing you of the rapid development process you are used to with Python. -Obviously the time required to translate a project will vary, so do an -experimental translation just to see how long it takes to evaluate whether you -prefer this approach compared to using :ref:`use_same_source` or simply keeping -a separate Python 3 codebase. - -Below are the typical steps taken by a project which tries to support -Python 2 & 3 while keeping the code directly executable by Python 2. - +To help with writing source-compatible code using one of the projects mentioned +in `Projects to Consider`_, consider following the below suggestions. Some of +them are handled by the suggested projects, so if you do use one of them then +read their documentation first to see which suggestions below will taken care of +for you. Support Python 2.7 ------------------- +////////////////// As a first step, make sure that your project is compatible with `Python 2.7`_. This is just good to do as Python 2.7 is the last release of Python 2 and thus will be used for a rather long time. It also allows for use of the ``-3`` flag -to Python to help discover places in your code which 2to3 cannot handle but are -known to cause issues. +to Python to help discover places in your code where compatibility might be an +issue (the ``-3`` flag is in Python 2.6 but Python 2.7 adds more warnings). Try to Support `Python 2.6`_ and Newer Only -------------------------------------------- +/////////////////////////////////////////// While not possible for all projects, if you can support `Python 2.6`_ and newer **only**, your life will be much easier. Various future statements, stdlib additions, etc. exist only in Python 2.6 and later which greatly assist in -porting to Python 3. But if you project must keep support for `Python 2.5`_ (or -even `Python 2.4`_) then it is still possible to port to Python 3. +supporting Python 3. But if you project must keep support for `Python 2.5`_ then +it is still possible to simultaneously support Python 3. Below are the benefits you gain if you only have to support Python 2.6 and newer. Some of these options are personal choice while others are **strongly** recommended (the ones that are more for personal choice are labeled as such). If you continue to support older versions of Python then you -at least need to watch out for situations that these solutions fix. +at least need to watch out for situations that these solutions fix and handle +them appropriately (which is where library help from e.g. six_ comes in handy). ``from __future__ import print_function`` ''''''''''''''''''''''''''''''''''''''''' -This is a personal choice. 2to3 handles the translation from the print -statement to the print function rather well so this is an optional step. This -future statement does help, though, with getting used to typing -``print('Hello, World')`` instead of ``print 'Hello, World'``. +It will not only get you used to typing ``print()`` as a function instead of a +statement, but it will also give you the various benefits the function has over +the Python 2 statement (six_ provides a function if you support Python 2.5 or +older). ``from __future__ import unicode_literals`` ''''''''''''''''''''''''''''''''''''''''''' -Another personal choice. You can always mark what you want to be a (unicode) -string with a ``u`` prefix to get the same effect. But regardless of whether -you use this future statement or not, you **must** make sure you know exactly -which Python 2 strings you want to be bytes, and which are to be strings. This -means you should, **at minimum** mark all strings that are meant to be text -strings with a ``u`` prefix if you do not use this future statement. Python 3.3 -allows strings to continue to have the ``u`` prefix (it's a no-op in that case) -to make it easier for code to be source-compatible between Python 2 & 3. +If you choose to use this future statement then all string literals in +Python 2 will be assumed to be Unicode (as is already the case in Python 3). +If you choose not to use this future statement then you should mark all of your +text strings with a ``u`` prefix and only support Python 3.3 or newer. But you +are **strongly** advised to do one or the other (six_ provides a function in +case you don't want to use the future statement **and** you want to support +Python 3.2 or older). -Bytes literals -'''''''''''''' +Bytes/string literals +''''''''''''''''''''' -This is a **very** important one. The ability to prefix Python 2 strings that -are meant to contain bytes with a ``b`` prefix help to very clearly delineate -what is and is not a Python 3 string. When you run 2to3 on code, all Python 2 -strings become Python 3 strings **unless** they are prefixed with ``b``. +This is a **very** important one. Prefix Python 2 strings that +are meant to contain bytes with a ``b`` prefix to very clearly delineate +what is and is not a Python 3 text string (six_ provides a function to use for +Python 2.5 compatibility). This point cannot be stressed enough: make sure you know what all of your string -literals in Python 2 are meant to become in Python 3. Any string literal that +literals in Python 2 are meant to be in Python 3. Any string literal that should be treated as bytes should have the ``b`` prefix. Any string literal that should be Unicode/text in Python 2 should either have the ``u`` literal (supported, but ignored, in Python 3.3 and later) or you should have ``from __future__ import unicode_literals`` at the top of the file. But the key -point is you should know how Python 3 will treat everyone one of your string +point is you should know how Python 3 will treat every one one of your string literals and you should mark them as appropriate. There are some differences between byte literals in Python 2 and those in Python 3 thanks to the bytes type just being an alias to ``str`` in Python 2. -Probably the biggest "gotcha" is that indexing results in different values. In -Python 2, the value of ``b'py'[1]`` is ``'y'``, while in Python 3 it's ``121``. -You can avoid this disparity by always slicing at the size of a single element: -``b'py'[1:2]`` is ``'y'`` in Python 2 and ``b'y'`` in Python 3 (i.e., close -enough). +See the `Handle Common "Gotchas"`_ section for what to watch out for. -You cannot concatenate bytes and strings in Python 3. But since Python -2 has bytes aliased to ``str``, it will succeed: ``b'a' + u'b'`` works in -Python 2, but ``b'a' + 'b'`` in Python 3 is a :exc:`TypeError`. A similar issue -also comes about when doing comparisons between bytes and strings. +``from __future__ import absolute_import`` +'''''''''''''''''''''''''''''''''''''''''' +Discussed in more detail below, but you should use this future statement to +prevent yourself from accidentally using implicit relative imports. Supporting `Python 2.5`_ and Newer Only ---------------------------------------- +/////////////////////////////////////// If you are supporting `Python 2.5`_ and newer there are still some features of Python that you can utilize. @@ -271,7 +224,7 @@ '''''''''''''''''''''''''''''''''''''''''' Implicit relative imports (e.g., importing ``spam.bacon`` from within -``spam.eggs`` with the statement ``import bacon``) does not work in Python 3. +``spam.eggs`` with the statement ``import bacon``) do not work in Python 3. This future statement moves away from that and allows the use of explicit relative imports (e.g., ``from . import bacon``). @@ -281,7 +234,7 @@ the statement, but you still want the __future__ statement to prevent implicit relative imports. In `Python 2.7`_ the __future__ statement is not needed. In other words, unless you are only supporting Python 2.7 or a version earlier -than Python 2.5, use the __future__ statement. +than Python 2.5, use this __future__ statement. Mark all Unicode strings with a ``u`` prefix @@ -290,17 +243,65 @@ While Python 2.6 has a ``__future__`` statement to automatically cause Python 2 to treat all string literals as Unicode, Python 2.5 does not have that shortcut. This means you should go through and mark all string literals with a ``u`` -prefix to turn them explicitly into Unicode strings where appropriate. That -leaves all unmarked string literals to be considered byte literals in Python 3. +prefix to turn them explicitly into text strings where appropriate and only +support Python 3.3 or newer. Otherwise use a project like six_ which provides a +function to pass all text string literals through. +Capturing the Currently Raised Exception +'''''''''''''''''''''''''''''''''''''''' + +In Python 2.5 and earlier the syntax to access the current exception is:: + + try: + raise Exception() + except Exception, exc: + # Current exception is 'exc'. + pass + +This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) +to:: + + try: + raise Exception() + except Exception as exc: + # Current exception is 'exc'. + # In Python 3, 'exc' is restricted to the block; in Python 2.6/2.7 it will "leak". + pass + +Because of this syntax change you must change how you capture the current +exception in Python 2.5 and earlier to:: + + try: + raise Exception() + except Exception: + import sys + exc = sys.exc_info()[1] + # Current exception is 'exc'. + pass + +You can get more information about the raised exception from +:func:`sys.exc_info` than simply the current exception instance, but you most +likely don't need it. + +.. note:: + In Python 3, the traceback is attached to the exception instance + through the ``__traceback__`` attribute. If the instance is saved in + a local variable that persists outside of the ``except`` block, the + traceback will create a reference cycle with the current frame and its + dictionary of local variables. This will delay reclaiming dead + resources until the next cyclic :term:`garbage collection` pass. + + In Python 2, this problem only occurs if you save the traceback itself + (e.g. the third element of the tuple returned by :func:`sys.exc_info`) + in a variable. + Handle Common "Gotchas" ------------------------ +/////////////////////// -There are a few things that just consistently come up as sticking points for -people which 2to3 cannot handle automatically or can easily be done in Python 2 -to help modernize your code. +These are things to watch out for no matter what version of Python 2 you are +supporting which are not syntactic considerations. ``from __future__ import division`` @@ -357,9 +358,9 @@ the bytes/string dichotomy. Because Python 2 allowed the ``str`` type to hold textual data, people have over the years been rather loose in their delineation of what ``str`` instances held text compared to bytes. In Python 3 you cannot -be so care-free anymore and need to properly handle the difference. The key +be so care-free anymore and need to properly handle the difference. The key to handling this issue is to make sure that **every** string literal in your -Python 2 code is either syntactically of functionally marked as either bytes or +Python 2 code is either syntactically or functionally marked as either bytes or text data. After this is done you then need to make sure your APIs are designed to either handle a specific type or made to be properly polymorphic. @@ -466,14 +467,7 @@ happen to use the ``unicode(self).encode('utf8')`` idiom as the body of your ``__str__()`` method). -There are two ways to solve this issue. One is to use a custom 2to3 fixer. The -blog post at http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ -specifies how to do this. That will allow 2to3 to change all instances of ``def -__unicode(self): ...`` to ``def __str__(self): ...``. This does require that you -define your ``__str__()`` method in Python 2 before your ``__unicode__()`` -method. - -The other option is to use a mixin class. This allows you to only define a +You can use a mixin class to work around this. This allows you to only define a ``__unicode__()`` method for your class and let the mixin derive ``__str__()`` for you (code from http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/):: @@ -516,6 +510,7 @@ Even better is to use the documented attributes the exception provides. + Don't use ``__getslice__`` & Friends '''''''''''''''''''''''''''''''''''' @@ -527,23 +522,23 @@ Updating doctests ''''''''''''''''' -2to3_ will attempt to generate fixes for doctests that it comes across. It's -not perfect, though. If you wrote a monolithic set of doctests (e.g., a single -docstring containing all of your doctests), you should at least consider -breaking the doctests up into smaller pieces to make it more manageable to fix. -Otherwise it might very well be worth your time and effort to port your tests -to :mod:`unittest`. +Don't forget to make them Python 2/3 compatible as well. If you wrote a +monolithic set of doctests (e.g., a single docstring containing all of your +doctests), you should at least consider breaking the doctests up into smaller +pieces to make it more manageable to fix. Otherwise it might very well be worth +your time and effort to port your tests to :mod:`unittest`. -Update `map` for imbalanced input sequences -''''''''''''''''''''''''''''''''''''''''''' +Update ``map`` for imbalanced input sequences +''''''''''''''''''''''''''''''''''''''''''''' -With Python 2, `map` would pad input sequences of unequal length with -`None` values, returning a sequence as long as the longest input sequence. +With Python 2, when ``map`` was given more than one input sequence it would pad +the shorter sequences with `None` values, returning a sequence as long as the +longest input sequence. -With Python 3, if the input sequences to `map` are of unequal length, `map` +With Python 3, if the input sequences to ``map`` are of unequal length, ``map`` will stop at the termination of the shortest of the sequences. For full -compatibility with `map` from Python 2.x, also wrap the sequences in +compatibility with ``map`` from Python 2.x, wrap the sequence arguments in :func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes ``list(map(func, itertools.zip_longest(*sequences)))``. @@ -552,176 +547,37 @@ When you run your application's test suite, run it using the ``-3`` flag passed to Python. This will cause various warnings to be raised during execution about -things that 2to3 cannot handle automatically (e.g., modules that have been -removed). Try to eliminate those warnings to make your code even more portable -to Python 3. +things that are semantic changes between Python 2 and 3. Try to eliminate those +warnings to make your code even more portable to Python 3. -Run 2to3 --------- +Alternative Approaches +====================== -Once you have made your Python 2 code future-compatible with Python 3, it's -time to use 2to3_ to actually port your code. +While supporting Python 2 & 3 simultaneously is typically the preferred choice +by people so that they can continue to improve code and have it work for the +most number of users, your life may be easier if you only have to support one +major version of Python going forward. +Supporting Only Python 3 Going Forward From Python 2 Code +--------------------------------------------------------- -Manually -'''''''' +If you have Python 2 code but going forward only want to improve it as Python 3 +code, then you can use 2to3_ to translate your Python 2 code to Python 3 code. +This is only recommended, though, if your current version of your project is +going into maintenance mode and you want all new features to be exclusive to +Python 3. -To manually convert source code using 2to3_, you use the ``2to3`` script that -is installed with Python 2.6 and later.:: - 2to3 +Backporting Python 3 code to Python 2 +------------------------------------- -This will cause 2to3 to write out a diff with all of the fixers applied for the -converted source code. If you would like 2to3 to go ahead and apply the changes -you can pass it the ``-w`` flag:: - - 2to3 -w - -There are other flags available to control exactly which fixers are applied, -etc. - - -During Installation -''''''''''''''''''' - -When a user installs your project for Python 3, you can have either -:mod:`distutils` or Distribute_ run 2to3_ on your behalf. -For distutils, use the following idiom:: - - try: # Python 3 - from distutils.command.build_py import build_py_2to3 as build_py - except ImportError: # Python 2 - from distutils.command.build_py import build_py - - setup(cmdclass = {'build_py': build_py}, - # ... - ) - -For Distribute:: - - setup(use_2to3=True, - # ... - ) - -This will allow you to not have to distribute a separate Python 3 version of -your project. It does require, though, that when you perform development that -you at least build your project and use the built Python 3 source for testing. - - -Verify & Test -------------- - -At this point you should (hopefully) have your project converted in such a way -that it works in Python 3. Verify it by running your unit tests and making sure -nothing has gone awry. If you miss something then figure out how to fix it in -Python 3, backport to your Python 2 code, and run your code through 2to3 again -to verify the fix transforms properly. - - -.. _2to3: http://docs.python.org/py3k/library/2to3.html -.. _Distribute: http://packages.python.org/distribute/ - - -.. _use_same_source: - -Python 2/3 Compatible Source -============================ - -While it may seem counter-intuitive, you can write Python code which is -source-compatible between Python 2 & 3. It does lead to code that is not -entirely idiomatic Python (e.g., having to extract the currently raised -exception from ``sys.exc_info()[1]``), but it can be run under Python 2 -**and** Python 3 without using 2to3_ as a translation step (although the tool -should be used to help find potential portability problems). This allows you to -continue to have a rapid development process regardless of whether you are -developing under Python 2 or Python 3. Whether this approach or using -:ref:`use_2to3` works best for you will be a per-project decision. - -To get a complete idea of what issues you will need to deal with, see the -`What's New in Python 3.0`_. Others have reorganized the data in other formats -such as http://docs.pythonsprints.com/python3_porting/py-porting.html\ . - -The following are some steps to take to try to support both Python 2 & 3 from -the same source code. - - -.. _What's New in Python 3.0: http://docs.python.org/release/3.0/whatsnew/3.0.html - - -Follow The Steps for Using 2to3_ --------------------------------- - -All of the steps outlined in how to -:ref:`port Python 2 code with 2to3 ` apply -to creating a Python 2/3 codebase. This includes trying only support Python 2.6 -or newer (the :mod:`__future__` statements work in Python 3 without issue), -eliminating warnings that are triggered by ``-3``, etc. - -You should even consider running 2to3_ over your code (without committing the -changes). This will let you know where potential pain points are within your -code so that you can fix them properly before they become an issue. - - -Use six_ --------- - -The six_ project contains many things to help you write portable Python code. -You should make sure to read its documentation from beginning to end and use -any and all features it provides. That way you will minimize any mistakes you -might make in writing cross-version code. - - -Capturing the Currently Raised Exception ----------------------------------------- - -One change between Python 2 and 3 that will require changing how you code (if -you support `Python 2.5`_ and earlier) is -accessing the currently raised exception. In Python 2.5 and earlier the syntax -to access the current exception is:: - - try: - raise Exception() - except Exception, exc: - # Current exception is 'exc' - pass - -This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) -to:: - - try: - raise Exception() - except Exception as exc: - # Current exception is 'exc' - # In Python 3, 'exc' is restricted to the block; Python 2.6 will "leak" - pass - -Because of this syntax change you must change to capturing the current -exception to:: - - try: - raise Exception() - except Exception: - import sys - exc = sys.exc_info()[1] - # Current exception is 'exc' - pass - -You can get more information about the raised exception from -:func:`sys.exc_info` than simply the current exception instance, but you most -likely don't need it. - -.. note:: - In Python 3, the traceback is attached to the exception instance - through the ``__traceback__`` attribute. If the instance is saved in - a local variable that persists outside of the ``except`` block, the - traceback will create a reference cycle with the current frame and its - dictionary of local variables. This will delay reclaiming dead - resources until the next cyclic :term:`garbage collection` pass. - - In Python 2, this problem only occurs if you save the traceback itself - (e.g. the third element of the tuple returned by :func:`sys.exc_info`) - in a variable. +If you have Python 3 code and have little interest in supporting Python 2 you +can use 3to2_ to translate from Python 3 code to Python 2 code. This is only +recommended if you don't plan to heavily support Python 2 users. Otherwise +write your code for Python 3 and then backport as far back as you want. This +is typically easier than going from Python 2 to 3 as you will have worked out +any difficulties with e.g. bytes/strings, etc. Other Resources @@ -729,18 +585,41 @@ The authors of the following blog posts, wiki pages, and books deserve special thanks for making public their tips for porting Python 2 code to Python 3 (and -thus helping provide information for this document): +thus helping provide information for this document and its various revisions +over the years): +* http://wiki.python.org/moin/PortingPythonToPy3k * http://python3porting.com/ * http://docs.pythonsprints.com/python3_porting/py-porting.html * http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/ * http://dabeaz.blogspot.com/2011/01/porting-py65-and-my-superboard-to.html * http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ * http://lucumr.pocoo.org/2010/2/11/porting-to-python-3-a-guide/ -* http://wiki.python.org/moin/PortingPythonToPy3k * https://wiki.ubuntu.com/Python/3 If you feel there is something missing from this document that should be added, please email the python-porting_ mailing list. + + +.. _2to3: http://docs.python.org/2/library/2to3.html +.. _3to2: https://pypi.python.org/pypi/3to2 +.. _Cheeseshop: PyPI_ +.. _coverage: https://pypi.python.org/pypi/coverage +.. _future: http://python-future.org/ +.. _modernize: https://github.com/mitsuhiko/python-modernize +.. _Porting to Python 3: http://python3porting.com/ +.. _PyPI: http://pypi.python.org/ +.. _Python 2.2: http://www.python.org/2.2.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 2.6: http://www.python.org/2.6.x +.. _Python 2.7: http://www.python.org/2.7.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 3.3: http://www.python.org/3.3.x +.. _Python 3 Packages: https://pypi.python.org/pypi?:action=browse&c=533&show=all +.. _Python 3 Q & A: http://ncoghlan-devs-python-notes.readthedocs.org/en/latest/python3/questions_and_answers.html .. _python-porting: http://mail.python.org/mailman/listinfo/python-porting +.. _six: https://pypi.python.org/pypi/six +.. _tox: https://pypi.python.org/pypi/tox +.. _trove classifiers: https://pypi.python.org/pypi?%3Aaction=list_classifiers + -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 7 18:29:36 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 7 Mar 2014 18:29:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_null_merge_for_issue_=2320812?= Message-ID: <3fgbRh5Hsdz7LkM@mail.python.org> http://hg.python.org/cpython/rev/c83ce2a1841c changeset: 89499:c83ce2a1841c parent: 89497:2a922153463e parent: 89498:a24085e1b1f5 user: Brett Cannon date: Fri Mar 07 12:29:24 2014 -0500 summary: null merge for issue #20812 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 02:26:11 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 02:26:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzEwMTk3OiBVcGRh?= =?utf-8?q?te_get=5Bstatus=5Doutput_versionchanged_with_actual_version=2E?= Message-ID: <3fgp1b65G7z7LjN@mail.python.org> http://hg.python.org/cpython/rev/34df43c9c74a changeset: 89500:34df43c9c74a branch: 3.3 parent: 89498:a24085e1b1f5 user: R David Murray date: Fri Mar 07 20:04:17 2014 -0500 summary: #10197: Update get[status]output versionchanged with actual version. This was perhaps on the border between a bug fix and a feature since the Python3 docs did not originally say it was unix only. However, the functions never worked anywhere but unix, and the docs were changed to say it was unix only well before the windows support was added. Unfortunately, windows support was added in 3.3.4 as well as 3.4. That leaves us in the uncomfortable position of needing the tag to say "version changed: 3.3.4" :( files: Doc/library/subprocess.rst | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1072,8 +1072,10 @@ >>> subprocess.getstatusoutput('/bin/junk') (256, 'sh: /bin/junk: not found') - .. versionchanged:: 3.3 - Availability: Unix & Windows + Availability: Unix & Windows + + .. versionchanged:: 3.3.4 + Windows support added .. function:: getoutput(cmd) @@ -1086,8 +1088,10 @@ >>> subprocess.getoutput('ls /bin/ls') '/bin/ls' - .. versionchanged:: 3.3 - Availability: Unix & Windows + Availability: Unix & Windows + + .. versionchanged:: 3.3.4 + Windows support added Notes -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 02:26:13 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 02:26:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_=2310197=3A_Update_get=5Bstatus=5Doutput_versionchanged_?= =?utf-8?q?with_actual_version=2E?= Message-ID: <3fgp1d1fGmz7LlL@mail.python.org> http://hg.python.org/cpython/rev/ee277b383d33 changeset: 89501:ee277b383d33 parent: 89499:c83ce2a1841c parent: 89500:34df43c9c74a user: R David Murray date: Fri Mar 07 20:04:41 2014 -0500 summary: #10197: Update get[status]output versionchanged with actual version. This was perhaps on the border between a bug fix and a feature since the Python3 docs did not originally say it was unix only. However, the functions never worked anywhere but unix, and the docs were changed to say it was unix only well before the windows support was added. Unfortunately, windows support was added in 3.3.4 as well as 3.4. That leaves us in the uncomfortable position of needing the tag to say "version changed: 3.3.4" :( files: Doc/library/subprocess.rst | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1091,8 +1091,10 @@ >>> subprocess.getstatusoutput('/bin/junk') (256, 'sh: /bin/junk: not found') - .. versionchanged:: 3.3 - Availability: Unix & Windows + Availability: Unix & Windows + + .. versionchanged:: 3.3.4 + Windows support added .. function:: getoutput(cmd) @@ -1105,8 +1107,10 @@ >>> subprocess.getoutput('ls /bin/ls') '/bin/ls' - .. versionchanged:: 3.3 - Availability: Unix & Windows + Availability: Unix & Windows + + .. versionchanged:: 3.3.4 + Windows support added Notes -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 02:26:14 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 02:26:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_get=5Bstatus?= =?utf-8?q?=7Coutput=5D_works_on_windows=2E?= Message-ID: <3fgp1f4Wmjz7LlX@mail.python.org> http://hg.python.org/cpython/rev/512032de01ab changeset: 89502:512032de01ab user: R David Murray date: Fri Mar 07 20:06:24 2014 -0500 summary: whatsnew: get[status|output] works on windows. Since 3.3.4.... :( files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1246,6 +1246,10 @@ be used to provide the contents of ``stdin`` for the command that is run. (Contributed by Zack Weinberg in :issue:`16624`.) +:func:`~subprocess.getstatus` and :func:`~subprocess.getstatusoutput` now +work on Windows. This change was actually made in 3.3.4. (Contributed +by Tim Golden in :issue:`10197`.) + sunau ----- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 03:02:06 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 03:02:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_cp273_codec_?= =?utf-8?b?KCMxMDkwNzc5Nyk=?= Message-ID: <3fgpq25C68z7LjZ@mail.python.org> http://hg.python.org/cpython/rev/d843a1caba78 changeset: 89503:d843a1caba78 user: R David Murray date: Fri Mar 07 21:00:34 2014 -0500 summary: whatsnew: cp273 codec (#10907797) Also updated the docs and added the aliases mentioned by the references. files: Doc/library/codecs.rst | 4 ++++ Doc/whatsnew/3.4.rst | 3 +++ Lib/encodings/aliases.py | 5 +++++ 3 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -971,6 +971,10 @@ +-----------------+--------------------------------+--------------------------------+ | cp037 | IBM037, IBM039 | English | +-----------------+--------------------------------+--------------------------------+ +| cp273 | 273, IBM273, csIBM273 | German | +| | | | +| | | .. versionadded:: 3.4 | ++-----------------+--------------------------------+--------------------------------+ | cp424 | EBCDIC-CP-HE, IBM424 | Hebrew | +-----------------+--------------------------------+--------------------------------+ | cp437 | 437, IBM437 | English | diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -365,6 +365,9 @@ Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in :issue:`12892`. +* New EBCDIC :ref:`codec ` ``cp273``. (Contributed by + Michael Bierenfeld and Andrew Kuchling in :issue:`1097797`.) + * :class:`bytes`.join() and :class:`bytearray`.join() now accept arbitrary buffer objects as arguments. (Contributed by Antoine Pitrou in :issue:`15958`.) diff --git a/Lib/encodings/aliases.py b/Lib/encodings/aliases.py --- a/Lib/encodings/aliases.py +++ b/Lib/encodings/aliases.py @@ -109,6 +109,11 @@ '1258' : 'cp1258', 'windows_1258' : 'cp1258', + # cp273 codec + '273' : 'cp273', + 'ibm273' : 'cp273', + 'csibm273' : 'cp273', + # cp424 codec '424' : 'cp424', 'csibm424' : 'cp424', -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 03:38:25 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 03:38:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_sunau_supports?= =?utf-8?q?_24_bit_samples=2E?= Message-ID: <3fgqcx3mZJz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/12fb14bcde41 changeset: 89504:12fb14bcde41 user: R David Murray date: Fri Mar 07 21:15:56 2014 -0500 summary: whatsnew: sunau supports 24 bit samples. files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1265,6 +1265,10 @@ called automatically at the end of the block. (Contributed by Serhiy Storchaka in :issue:`18878`.) +:meth:`.AU_write.setsampwidth` now supports 24 bit samples, thus adding +support for writing 24 sample using the module. (Contributed by +Serhiy Storchaka in :issue:`19261`.) + sys --- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 03:38:26 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 03:38:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_inet=5Fpton/in?= =?utf-8?q?et=5Fntop_support_windows_=28=237171=29=2E?= Message-ID: <3fgqcy5cYjz7LlR@mail.python.org> http://hg.python.org/cpython/rev/f82145a516f0 changeset: 89505:f82145a516f0 user: R David Murray date: Fri Mar 07 21:22:39 2014 -0500 summary: whatsnew: inet_pton/inet_ntop support windows (#7171). Added versionchanged to docs. files: Doc/library/socket.rst | 6 ++++++ Doc/whatsnew/3.4.rst | 3 +++ 2 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -633,6 +633,9 @@ Availability: Unix (maybe not all platforms), Windows. + .. versionchanged:: 3.4 + Windows support added + .. function:: inet_ntop(address_family, packed_ip) @@ -649,6 +652,9 @@ Availability: Unix (maybe not all platforms), Windows. + .. versionchanged:: 3.4 + Windows support added + .. XXX: Are sendmsg(), recvmsg() and CMSG_*() available on any diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1178,6 +1178,9 @@ The :data:`~socket.AF_LINK` constant is now available on BSD and OSX. +:func:`~socket.inet_pton` and :func:`~socket.inet_ntop` are now supported +on Windows. (Contributed by Atsuo Ishimoto in :issue:`7171`.) + sqlite3 ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 03:38:28 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 03:38:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_audioop_accept?= =?utf-8?q?s_any_bytes-like=2C_rejects_strings=2E?= Message-ID: <3fgqd00Gcfz7Lm6@mail.python.org> http://hg.python.org/cpython/rev/2776aa0198a9 changeset: 89506:2776aa0198a9 user: R David Murray date: Fri Mar 07 21:35:31 2014 -0500 summary: whatsnew: audioop accepts any bytes-like, rejects strings. files: Doc/library/audioop.rst | 6 ++---- Doc/whatsnew/3.4.rst | 9 ++++++++- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst --- a/Doc/library/audioop.rst +++ b/Doc/library/audioop.rst @@ -12,10 +12,8 @@ .. versionchanged:: 3.4 Support for 24-bit samples was added. - -.. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted by all functions in this - module. Strings no more supported. + All functions now accept any :term:`bytes-like object`. + String input now results in an immediate error. .. index:: single: Intel/DVI ADPCM diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -554,12 +554,16 @@ audioop ------- -The module now supports 24-bit samples. (Contributed by Serhiy Storchaka +:mod:`audioop` now supports 24-bit samples. (Contributed by Serhiy Storchaka in :issue:`12866`.) Added the :func:`~audioop.byteswap` function to convert big-endian samples to little-endian and vice versa (:issue:`19641`). +All :mod:`audioop` functions now accept any :term:`bytes-like object`. Strings +are not accepted: they didn't work before, now they raise an error right away. +(Contributed by Serhiy Storchaka in :issue:`16685`.) + base64 ------ @@ -2050,6 +2054,9 @@ matched the input type, so if your code was depending on the return value being, say, a ``bytearray``, you will need to change your code. +* :mod:`audioop` functions now raise an error immediately if passed string + input, instead of failing randomly later on (:issue:`16685`). + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 07:36:51 2014 From: python-checkins at python.org (nick.coghlan) Date: Sat, 8 Mar 2014 07:36:51 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Improve_descriptions_of_in?= =?utf-8?q?trospection_changes?= Message-ID: <3fgww34ySrz7Lnj@mail.python.org> http://hg.python.org/cpython/rev/0909f73cfa3a changeset: 89507:0909f73cfa3a user: Nick Coghlan date: Sat Mar 08 16:36:37 2014 +1000 summary: Improve descriptions of introspection changes Several of the introspection changes in Python 3.4 are indirect, where inspect module changes affected pydoc, and those in turn affected the help builtin. This update adds versionchanged notes in the key locations, as well as more coverage in the What's New document (in particular, a note in the porting section regarding the expanded domain for inspect.getfullargspec). files: Doc/library/functions.rst | 4 ++ Doc/library/inspect.rst | 5 ++ Doc/library/pydoc.rst | 5 ++ Doc/whatsnew/3.4.rst | 53 ++++++++++++++++++++++---- 4 files changed, 59 insertions(+), 8 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -610,6 +610,10 @@ This function is added to the built-in namespace by the :mod:`site` module. + .. versionchanged:: 3.4 + Changes to :mod:`pydoc` and :mod:`inspect` mean that the reported + signatures for callables are now more comprehensive and consistent. + .. function:: hex(x) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -729,6 +729,11 @@ Consider using the new :ref:`Signature Object ` interface, which provides a better way of introspecting functions. + .. versionchanged:: 3.4 + This function is now based on :func:`signature`, but still ignores + ``__wrapped__`` attributes and includes the already bound first + parameter in the signature output for bound methods. + .. function:: getargvalues(frame) diff --git a/Doc/library/pydoc.rst b/Doc/library/pydoc.rst --- a/Doc/library/pydoc.rst +++ b/Doc/library/pydoc.rst @@ -84,3 +84,8 @@ .. versionchanged:: 3.2 Added the ``-b`` option, deprecated the ``-g`` option. + +.. versionchanged:: 3.4 + :mod:`pydoc` now uses :func:`inspect.signature` rather than + :func:`inspect.getfullargspec` to extract signature information from + callables. diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -133,6 +133,8 @@ a new :mod:`~email.message.Message` subclass (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME handling ` (:issue:`18891`). +* The :mod:`inspect` and :mod:`pydoc` modules are now capable of + correct introspection of a much wider variety of callable objects * The :mod:`ipaddress` module API has been declared stable @@ -384,6 +386,10 @@ and supports the :func:`reversed` builtin. (Contributed by Nick Coghlan and Claudiu Popa in :issue:`18690` and :issue:`19078`.) +* Signatures reported by :func:`help` have been modified and improved in + several cases as a result of the introduction of Argument Clinic and other + changes to the :mod:`inspect` and :mod:`pydoc` modules. + New Modules =========== @@ -871,7 +877,7 @@ inspect ------- -The inspect module now offers a basic :ref:`command line interface +The :mod:`inspect` module now offers a basic :ref:`command line interface ` to quickly display source code and other information for modules, classes and functions. (Contributed by Claudiu Popa and Nick Coghlan in :issue:`18626`) @@ -889,10 +895,14 @@ :func:`~inspect.getfullargspec` and :func:`~inspect.getargspec` now use the :func:`~inspect.signature` API. This allows them to -support much broader range of functions, including some builtins and -callables that follow ``__signature__`` protocol. It is still -recommended to update your code to use :func:`~inspect.signature` -directly. (Contributed by Yury Selivanov in :issue:`17481`) +support a much broader range of callables, including those with +``__signature__`` attributes, those with metadata provided by argument +clinic, :func:`functools.partial` objects and more. Note that, unlike +:func:`~inspect.signature`, these functions still ignore ``__wrapped__`` +attributes, and report the already bound first argument for bound methods, +so it is still necessary to update your code to use +:func:`~inspect.signature` directly if those features are desired. +(Contributed by Yury Selivanov in :issue:`17481`) :func:`~inspect.signature` now supports duck types of CPython functions, which adds support for functions compiled with Cython. (Contributed @@ -1086,11 +1096,25 @@ pydoc ----- -While significant changes have not been made to :mod:`pydoc` directly, +The :mod:`pydoc` module is now based directly on the +:func:`inspect.signature` introspection API, allowing it to provide +signature information for a wider variety of callable objects. This change +also means that ``__wrapped__`` attributes are now taken into account when +display help information (Contributed by Larry Hastings in :issue:`19674`) + +The :mod:`pydoc` module no longer displays the ``self`` parameter for +already bound methods. Instead, it aims to always display the exact current +signature of the supplied callable (Contributed by Larry Hastings in +:issue:`20710`) + +In addition to the changes that have been made to :mod:`pydoc` directly, its handling of custom ``__dir__`` methods and various descriptor -behaviours has been improved substantially by the underlying changes in +behaviours has also been improved substantially by the underlying changes in the :mod:`inspect` module. +As the :func:`help` builtin is based on :mod:`pydoc`, the above changes also +affect the behaviour of :func:`help`. + re -- @@ -1532,7 +1556,12 @@ implemented in C. Some standard library extension modules have been converted to use Argument -Clinic in Python 3.4, and :mod:`inspect` has been updated accordingly. +Clinic in Python 3.4, and :mod:`pydoc` and :mod:`inspect` has been updated +accordingly. + +It is expected that signature metadata for programmatic introspection will +be added to additional callables implemented in C as part of Python 3.4 +maintenance releases. .. note:: The Argument Clinic PEP is not fully up to date with the state of the @@ -1956,6 +1985,14 @@ :func:`inspect.unwrap` to access the first function in the chain that has no ``__wrapped__`` attribute. +* :func:`inspect.getfullargspec` has been reimplemented on top of + :func`inspect.signature` and hence handles a much wider variety of callable + objects than it did in the past. It is expected that additional builtin and + extension module callables will gain signature metadata over the course of + the Python 3.4 series. Code that assumes that + :func:`inspect.getfullargspec` will fail on non-Python callables may need + to be adjusted accordingly. + * :class:`importlib.machinery.PathFinder` now passes on the current working directory to objects in :data:`sys.path_hooks` for the empty string. This results in :data:`sys.path_importer_cache` never containing ``''``, thus -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Mar 8 09:34:54 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 08 Mar 2014 09:34:54 +0100 Subject: [Python-checkins] Daily reference leaks (d843a1caba78): sum=0 Message-ID: results for d843a1caba78 on branch "default" -------------------------------------------- test_site leaked [-2, 0, 2] references, sum=0 test_site leaked [-2, 0, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogUJ5Tpu', '-x'] From python-checkins at python.org Sat Mar 8 18:54:19 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 18:54:19 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Tweak_NEWS_so_some_additio?= =?utf-8?q?nal_issue_numbers_become_links_in_html_version=2E?= Message-ID: <3fhCxl1hN5z7Ljn@mail.python.org> http://hg.python.org/cpython/rev/58a47fc5cdff changeset: 89508:58a47fc5cdff user: R David Murray date: Sat Mar 08 10:06:04 2014 -0500 summary: Tweak NEWS so some additional issue numbers become links in html version. files: Misc/NEWS | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1262,8 +1262,9 @@ - Issue #19523: Closed FileHandler leak which occurred when delay was set. -- Issue #19544 and #6516: Restore support for --user and --group parameters to - sdist command accidentally rolled back as part of the distutils2 rollback. +- Issue #19544 and Issue #6516: Restore support for --user and --group + parameters to sdist command accidentally rolled back as part of the + distutils2 rollback. - Issue #13674: Prevented time.strftime from crashing on Windows when given a year before 1900 and a format of %y. @@ -1609,8 +1610,9 @@ - Issue #19254: Provide an optimized Python implementation of pbkdf2_hmac. -- Issues #19201, #19222, #19223: Add "x" mode (exclusive creation) in opening - file to bz2, gzip and lzma modules. Patches by Tim Heaney and Vajrasky Kok. +- Issues #19201, Issue #19222, Issue #19223: Add "x" mode (exclusive creation) + in opening file to bz2, gzip and lzma modules. Patches by Tim Heaney and + Vajrasky Kok. - Fix a reference count leak in _sre. @@ -6328,7 +6330,7 @@ deallocator calls one of the methods on the type (e.g. when subclassing IOBase). Diagnosis and patch by Davide Rizzo. -- Issue #9611, #9015: FileIO.read() clamps the length to INT_MAX on Windows. +- Issue #9611, Issue #9015: FileIO.read() clamps the length to INT_MAX on Windows. - Issue #9642: Uniformize the tests on the availability of the mbcs codec, add a new HAVE_MBCS define. @@ -6796,7 +6798,7 @@ 'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath', and private attributes of 'smtpd.SMTPChannel'. -- Issue #5905, #13560: time.strftime() is now using the current locale +- Issue #5905, Issue #13560: time.strftime() is now using the current locale encoding, instead of UTF-8, if the wcsftime() function is not available. - Issue #13464: Add a readinto() method to http.client.HTTPResponse. Patch -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 18:54:20 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 18:54:20 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_Wave=5Fwrite_h?= =?utf-8?q?andles_unseekable_files=2E_=28=235202=29?= Message-ID: <3fhCxm3PMvz7LnK@mail.python.org> http://hg.python.org/cpython/rev/b861c7717c79 changeset: 89509:b861c7717c79 user: R David Murray date: Sat Mar 08 11:14:29 2014 -0500 summary: whatsnew: Wave_write handles unseekable files. (#5202) Also clarify the documentation of this feature. files: Doc/library/wave.rst | 31 +++++++++++++++++++++++++------ Doc/whatsnew/3.4.rst | 4 ++++ 2 files changed, 29 insertions(+), 6 deletions(-) diff --git a/Doc/library/wave.rst b/Doc/library/wave.rst --- a/Doc/library/wave.rst +++ b/Doc/library/wave.rst @@ -150,14 +150,30 @@ Wave_write Objects ------------------ +For seekable output streams, the ``wave`` header will automatically be updated +to reflect the number of frames actually written. For unseekable streams, the +*nframes* value must be accurate when the first frame data is written. An +accurate *nframes* value can be achieved either by calling +:meth:`~Wave_write.setnframes` or :meth:`~Wave_write.setparams` with the number +of frames that will be written before :meth:`~Wave_write.close` is called and +then using :meth:`~Wave_write.writeframesraw` to write the frame data, or by +calling :meth:`~Wave_write.writeframes` with all of the frame data to be +written. In the latter case :meth:`~Wave_write.writeframes` will calculate +the number of frames in the data and set *nframes* accordingly before writing +the frame data. + Wave_write objects, as returned by :func:`.open`, have the following methods: +.. versionchanged:: 3.4 + Added support for unseekable files. + .. method:: Wave_write.close() Make sure *nframes* is correct, and close the file if it was opened by - :mod:`wave`. This method is called upon object collection. Can raise an - exception if *nframes* is not correct and a file is not seekable. + :mod:`wave`. This method is called upon object collection. It will raise + an exception if the output stream is not seekable and *nframes* does not + match the number of frames actually written. .. method:: Wave_write.setnchannels(n) @@ -181,8 +197,9 @@ .. method:: Wave_write.setnframes(n) - Set the number of frames to *n*. This will be changed later if more frames are - written. + Set the number of frames to *n*. This will be changed later if the number + of frames actually written is different (this update attempt will + raise an error if the output stream is not seekable). .. method:: Wave_write.setcomptype(type, name) @@ -214,8 +231,10 @@ .. method:: Wave_write.writeframes(data) - Write audio frames and make sure *nframes* is correct. Can raise an - exception if a file is not seekable. + Write audio frames and make sure *nframes* is correct. It will raise an + error if the output stream is not seekable and the total number of frames + that have been written after *data* has been written does not match the + previously set value for *nframes*. .. versionchanged:: 3.4 Any :term:`bytes-like object`\ s are now accepted. diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1439,6 +1439,10 @@ :meth:`wave.open` now supports the context manager protocol. (Contributed by Claudiu Popa in :issue:`17616`.) +:mod:`wave` can now :ref:`write output to unssekable files +`. (Contributed by David Jones, Guilherme Polo, and Serhiy +Storchaka in :issue:`5202`.) + weakref ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 18:54:21 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 18:54:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_sunau/aifc/wav?= =?utf-8?q?e_writeframes=5Braw=5D_accept_any_bytes-like_=28=238311=29?= Message-ID: <3fhCxn4l5Lz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/b72615222c98 changeset: 89510:b72615222c98 user: R David Murray date: Sat Mar 08 11:46:05 2014 -0500 summary: whatsnew: sunau/aifc/wave writeframes[raw] accept any bytes-like (#8311) files: Doc/library/aifc.rst | 4 ++-- Doc/library/sunau.rst | 4 ++-- Doc/library/wave.rst | 4 ++-- Doc/whatsnew/3.4.rst | 14 +++++++++++++- 4 files changed, 19 insertions(+), 7 deletions(-) diff --git a/Doc/library/aifc.rst b/Doc/library/aifc.rst --- a/Doc/library/aifc.rst +++ b/Doc/library/aifc.rst @@ -226,7 +226,7 @@ file parameters have been set. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. .. method:: aifc.writeframesraw(data) @@ -235,7 +235,7 @@ updated. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. .. method:: aifc.close() diff --git a/Doc/library/sunau.rst b/Doc/library/sunau.rst --- a/Doc/library/sunau.rst +++ b/Doc/library/sunau.rst @@ -251,7 +251,7 @@ Write audio frames, without correcting *nframes*. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. .. method:: AU_write.writeframes(data) @@ -259,7 +259,7 @@ Write audio frames and make sure *nframes* is correct. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. .. method:: AU_write.close() diff --git a/Doc/library/wave.rst b/Doc/library/wave.rst --- a/Doc/library/wave.rst +++ b/Doc/library/wave.rst @@ -226,7 +226,7 @@ Write audio frames, without correcting *nframes*. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. .. method:: Wave_write.writeframes(data) @@ -237,7 +237,7 @@ previously set value for *nframes*. .. versionchanged:: 3.4 - Any :term:`bytes-like object`\ s are now accepted. + Any :term:`bytes-like object` is now accepted. Note that it is invalid to set any parameters after calling :meth:`writeframes` diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -540,7 +540,7 @@ aifc ---- -The :meth:`~aifc.getparams` method now returns a namedtuple rather than a +The :meth:`~aifc.aifc.getparams` method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.) :func:`aifc.open` now supports the context manager protocol: when used in a @@ -548,6 +548,10 @@ object will be called automatically at the end of the block. (Contributed by Serhiy Storchacha in :issue:`16486`.) +The :meth:`~aifc.aifc.writeframesraw` and :meth:`~aifc.aifc.writeframes` +methods now accept any :term:`bytes-like object`. (Contributed by Serhiy +Storchaka in :issue:`8311`.) + argparse -------- @@ -1300,6 +1304,10 @@ support for writing 24 sample using the module. (Contributed by Serhiy Storchaka in :issue:`19261`.) +The :meth:`~sunau.AU_write.writeframesraw` and +:meth:`~sunau.AU_write.writeframes` methods now accept any :term:`bytes-like +object`. (Contributed by Serhiy Storchaka in :issue:`8311`.) + sys --- @@ -1443,6 +1451,10 @@ `. (Contributed by David Jones, Guilherme Polo, and Serhiy Storchaka in :issue:`5202`.) +The :meth:`~wave.Wave_write.writeframesraw` and +:meth:`~wave.Wave_write.writeframes` methods now accept any :term:`bytes-like +object`. (Contributed by Serhiy Storchaka in :issue:`8311`.) + weakref ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 18:54:22 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 18:54:22 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_dbm=2Eopen_is_?= =?utf-8?q?context_manager=2E_=28=2319282=29?= Message-ID: <3fhCxp6lS9z7LpF@mail.python.org> http://hg.python.org/cpython/rev/200207e50cbf changeset: 89511:200207e50cbf user: R David Murray date: Sat Mar 08 12:00:52 2014 -0500 summary: whatsnew: dbm.open is context manager. (#19282) files: Doc/whatsnew/3.4.rst | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -617,6 +617,15 @@ differences between single use, reusable and reentrant context managers. +dbm +--- + +:func:`dbm.open` objects now support the context management protocol. When +used in a :keyword:`with` statement, the ``close`` method of the database +object will be called automatically at the end of the block. (Contributed by +Claudiu Popa and Nick Coghlan in :issue:`19282`.) + + dis --- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 18:54:24 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 8 Mar 2014 18:54:24 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_base65_encodin?= =?utf-8?b?Z3MuICgjMTc2MTgp?= Message-ID: <3fhCxr29Zlz7LpR@mail.python.org> http://hg.python.org/cpython/rev/1853679c6f71 changeset: 89512:1853679c6f71 user: R David Murray date: Sat Mar 08 12:53:28 2014 -0500 summary: whatsnew: base65 encodings. (#17618) Also updated the base64 module title and introduction to adjust for the fact that these new encodings are included. Also adjusted the wording about the base64 alphabets (see issue #20837). files: Doc/library/base64.rst | 32 +++++++++++++++++------------ Doc/whatsnew/3.4.rst | 8 +++++++ 2 files changed, 27 insertions(+), 13 deletions(-) diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst --- a/Doc/library/base64.rst +++ b/Doc/library/base64.rst @@ -1,27 +1,33 @@ -:mod:`base64` --- RFC 3548: Base16, Base32, Base64 Data Encodings -================================================================= +:mod:`base64` --- Base16, Base32, Base64, Base85 Data Encodings +=============================================================== .. module:: base64 - :synopsis: RFC 3548: Base16, Base32, Base64 Data Encodings + :synopsis: RFC 3548: Base16, Base32, Base64 Data Encodings; + Base85 and Ascii85 .. index:: pair: base64; encoding single: MIME; base64 encoding -This module provides data encoding and decoding as specified in :rfc:`3548`. -This standard defines the Base16, Base32, and Base64 algorithms for encoding -and decoding arbitrary binary strings into ASCII-only byte strings that can be +This module provides functions for encoding binary data to printable +ASCII characters and decoding such encodings back to binary data. +It provides encoding and decoding functions for the encodings specified in +in :rfc:`3548`, which defines the Base16, Base32, and Base64 algorithms, +and for the de-facto standard Ascii85 and Base85 encodings. + +The :rfc:`3548` encodings are suitable for encoding binary data so that it can safely sent by email, used as parts of URLs, or included as part of an HTTP POST request. The encoding algorithm is not the same as the :program:`uuencode` program. -There are two interfaces provided by this module. The modern interface -supports encoding and decoding ASCII byte string objects using all three -alphabets. Additionally, the decoding functions of the modern interface also -accept Unicode strings containing only ASCII characters. The legacy interface -provides for encoding and decoding to and from file-like objects as well as -byte strings, but only using the Base64 standard alphabet. +There are two :rfc:`3548` interfaces provided by this module. The modern +interface supports encoding and decoding ASCII byte string objects using all +three :rfc:`3548` defined alphabets (normal, URL-safe, and filesystem-safe). +Additionally, the decoding functions of the modern interface also accept +Unicode strings containing only ASCII characters. The legacy interface provides +for encoding and decoding to and from file-like objects as well as byte +strings, but only using the Base64 standard alphabet. .. versionchanged:: 3.3 ASCII-only Unicode strings are now accepted by the decoding functions of @@ -29,7 +35,7 @@ .. versionchanged:: 3.4 Any :term:`bytes-like object`\ s are now accepted by all - encoding and decoding functions in this module. + encoding and decoding functions in this module. Ascii85/Base85 support added. The modern interface provides: diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -583,6 +583,14 @@ :class:`bytes` or :class:`bytearray` instance. (Contributed by Nick Coghlan in :issue:`17839`.) +New functions :func:`~base64.a85encode`, :func:`~base64.a85decode`, +:func:`~base64.b85encode`, and :func:`~base64.b85decode` provide the ability to +encode and decode binary data from and to ``Ascii85`` and the git/mercurial +``Base85`` formats, respectively. The ``a85`` functions have options that can +be sued to make them compatible with the variants of the ``Ascii85`` encoding, +including the Adobe variant. (Contributed by Martin Morrison, the Mercurial +project, Serhiy Storchaka, and Antoine Pitrou in :issue:`17618`.) + colorsys -------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 8 18:56:15 2014 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 8 Mar 2014 18:56:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Improve_the_de?= =?utf-8?q?fault_seeding_in_random_module_to_use_32_bytes_of_entropy_when?= Message-ID: <3fhCzz360kz7LjT@mail.python.org> http://hg.python.org/cpython/rev/1ba8deb4536d changeset: 89513:1ba8deb4536d branch: 2.7 parent: 89484:52256a5861fa user: Raymond Hettinger date: Sat Mar 08 09:56:08 2014 -0800 summary: Improve the default seeding in random module to use 32 bytes of entropy when available. files: Lib/random.py | 2 +- Misc/NEWS | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -108,7 +108,7 @@ if a is None: try: - a = long(_hexlify(_urandom(16)), 16) + a = long(_hexlify(_urandom(32)), 16) except NotImplementedError: import time a = long(time.time() * 256) # use fractional seconds diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,10 @@ as documented. The pattern and source keyword parameters are left as deprecated aliases. +- Improve the random module's default seeding to use 256 bits of entropy + from os.urandom(). This was already done for Python 3, mildly improving + security with a bigger seed space. + - Issue #15618: Make turtle.py compatible with 'from __future__ import unicode_literals'. Initial patch by Juancarlo A?ez. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 03:41:47 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 03:41:47 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_glob=2Eescape?= =?utf-8?q?=2E?= Message-ID: <3fhRfM5Nwjz7Lqr@mail.python.org> http://hg.python.org/cpython/rev/e617f0ac696d changeset: 89514:e617f0ac696d parent: 89512:1853679c6f71 user: R David Murray date: Sat Mar 08 21:23:21 2014 -0500 summary: whatsnew: glob.escape. files: Doc/whatsnew/3.4.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -793,6 +793,14 @@ (Contributed by Antoine Pitrou in :issue:`16351`.) +glob +---- + +A new function :func:`~glob.escape` provides a way to escape special characters +in a filename so that they do not become part of the globbing expansion but are +instead matched literally. (Contributed by Serhiy Storchaka in :issue:`8402`.) + + hashlib ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 03:41:48 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 03:41:48 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_tidy_up_html_n?= =?utf-8?q?otes=2C_add_porting_note_for_expand=5Fcharrefs=2E?= Message-ID: <3fhRfN6j80z7Llx@mail.python.org> http://hg.python.org/cpython/rev/fd53f083768c changeset: 89515:fd53f083768c user: R David Murray date: Sat Mar 08 21:40:29 2014 -0500 summary: whatsnew: tidy up html notes, add porting note for expand_charrefs. files: Doc/whatsnew/3.4.rst | 26 ++++++++++++++++---------- 1 files changed, 16 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -830,16 +830,16 @@ html ---- -Added a new :func:`html.unescape` function that converts HTML5 character -references to the corresponding Unicode characters. -(Contributed by Ezio Melotti in :issue:`2927`) - -Added a new *convert_charrefs* keyword argument to -:class:`~html.parser.HTMLParser` that, when ``True``, automatically converts -all character references. For backward-compatibility, its value defaults -to ``False``, but it will change to ``True`` in future versions, so you -are invited to set it explicitly and update your code to use this new feature. -(Contributed by Ezio Melotti in :issue:`13633`) +New function :func:`~html.unescape` function converts HTML5 character references to +the corresponding Unicode characters. (Contributed by Ezio Melotti in +:issue:`2927`) + +:class:`~html.parser.HTMLParser` accepts a new keyword argument +*convert_charrefs* that, when ``True``, automatically converts all character +references. For backward-compatibility, its value defaults to ``False``, but +it will change to ``True`` in a future version of Python, so you are invited to +set it explicitly and update your code to use this new feature. (Contributed +by Ezio Melotti in :issue:`13633`) The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated. (Contributed by Ezio Melotti in :issue:`15114`) @@ -2135,6 +2135,12 @@ * :mod:`audioop` functions now raise an error immediately if passed string input, instead of failing randomly later on (:issue:`16685`). +* The new *convert_charrefs* argument to :class:`~html.parser.HTMLParser` + currently defaults to ``False`` for backward compatibility, but will + eventually be changed to default to ``True``. It is recommended that you add + this keyword, with the appropriate value, to any + :class:`~html.parser.HTMLParser` calls in your code. + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Mar 9 09:43:31 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 09 Mar 2014 09:43:31 +0100 Subject: [Python-checkins] Daily reference leaks (1853679c6f71): sum=-4 Message-ID: results for 1853679c6f71 on branch "default" -------------------------------------------- test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogpikoPI', '-x'] From python-checkins at python.org Sun Mar 9 09:47:16 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 09:47:16 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_=2Emd5_files_are_not_neede?= =?utf-8?q?d=2E?= Message-ID: <3fhbm436ZBz7Lqv@mail.python.org> http://hg.python.org/release/rev/adec99d760a3 changeset: 82:adec99d760a3 user: Georg Brandl date: Sun Mar 09 09:47:08 2014 +0100 summary: .md5 files are not needed. files: release.py | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/release.py b/release.py --- a/release.py +++ b/release.py @@ -232,10 +232,6 @@ checksum_tgz.hexdigest(), int(os.path.getsize(tgz)), tgz)) print(' %s %8s %s' % ( checksum_xz.hexdigest(), int(os.path.getsize(xz)), xz)) - with open(tgz + '.md5', 'w', encoding="ascii") as fp: - fp.write(checksum_tgz.hexdigest()) - with open(xz + '.md5', 'w', encoding="ascii") as fp: - fp.write(checksum_xz.hexdigest()) print('Signing tarballs') print('List of available private keys:') -- Repository URL: http://hg.python.org/release From python-checkins at python.org Sun Mar 9 10:01:42 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:01:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_PEP_101_a_bit=2E_Remov?= =?utf-8?q?e_outdated_Windows_instructions=2E?= Message-ID: <3fhc4k5Ryjz7Ljk@mail.python.org> http://hg.python.org/peps/rev/95aaea3e65b0 changeset: 5404:95aaea3e65b0 user: Georg Brandl date: Sun Mar 09 10:01:33 2014 +0100 summary: Update PEP 101 a bit. Remove outdated Windows instructions. files: pep-0101.txt | 95 ++++++++++++++------------------------- 1 files changed, 35 insertions(+), 60 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -59,7 +59,7 @@ done by the Release Manager (RM), the designated person performing the release. The roles and their current experts are: - * RM = Release Manager: Georg Brandl (Central Europe) + * RM = Release Manager: Larry Hastings (US) * WE = Windows: Martin von Loewis (Central Europe) * ME = Mac: Ned Deily (US) * DE = Docs: Georg Brandl (Central Europe) @@ -70,13 +70,10 @@ in different timezones, the RM must ensure that the release tag is created in enough time for the Experts to cut binary releases. - IT IS HIGHLY RECOMMENDED THAT YOU AT LEAST TAG THE TREE 24 HOURS - BEFORE A FINAL RELEASE. This will give the Experts enough time to - do their bits before the announcement goes out. - - In any case, the RM MUST wait for the "green light" from the - following experts before updating the web pages and sending the - announcement: WE, DE + You should not make the release public (by updating the website and + sending announcements) before all experts have updated their bits. + In rare cases where the expert for Windows or Mac is MIA, you may add + a message "(Platform) binaries will be provided shortly" and proceed. XXX: We should include a dependency graph to illustrate the steps that can be taken in parallel, or those that depend on other steps. @@ -139,14 +136,14 @@ Create a local clone of the cpython repository (called the "release clone" from now on). - + Also clone the repo at http://hg.python.org/cpython using the server-side clone feature. The name of the new clone should preferably have a "releasing/" prefix. The other experts will use the release clone for making the binaries, so it is important that they have access to it! - Optionally, set up your local release clone to push to the remote + It's best to set up your local release clone to push to the remote release clone by default (by editing .hg/hgrc to that effect). ___ Notify all committers by sending email to python-committers at python.org. @@ -208,10 +205,6 @@ ___ PC/python_nt.rc sets up the DLL version resource for Windows (displayed when you right-click on the DLL and select Properties). - ___ The license.ht file for the distribution on the website - contains what purports to be an HTML-ized copy of the LICENSE - file from the distribution. You'll need to bump the version number to - the one you're releasing. BROKEN ___ Check with the IE (if there is one ) to be sure that Lib/idlelib/NEWS.txt has been similarly updated. @@ -266,7 +259,7 @@ $ hg mv -f PC/python32gen.py PC/python33gen.py ___ Commit these changes to the default branch. - + ___ Now, go back to the previously noted revision and make the maintenance branch *from there*. @@ -285,43 +278,23 @@ order to create the release. There are things you can do while you wait though, so keep reading until you hit the next STOP. - ___ XXX The WE builds the Windows helpfile, using (in Doc/) either + ___ The WE builds the Windows helpfile, using (in Doc/) - $ make htmlhelp (on Unix) - - or - - > make.bat htmlhelp (on Windows) + > make.bat htmlhelp (on Windows) to create suitable input for HTML Help Workshop in build/htmlhelp. HTML - Help Workshop is then fired up on the created python26.hhp file, finally - resulting in an python26.chm file. He then copies the file into the Doc - directories of the build trees (once for each target architecture). + Help Workshop is then fired up on the created python33.hhp file, finally + resulting in an python33.chm file. - XXX The CHM file should also be scp'd to the docs download location. + ___ The WE then generates Windows installer files for each Windows + target architecture (for Python 3.3, this means x86 and AMD64). - ___ XXX The WE then generates Windows installer files for each Windows - target architecture (for Python 2.6, this means x86 and AMD64). He has - one checkout tree per target architecture, and builds the pcbuild.sln - project for the appropriate architecture. He then edits - Tools/msi/config.py to update full_current_version, sets snapshot to - False and runs msi.py with ActivePython 2.5 or Python 2.5 with pywin32. - For that to work, the following prerequisites must be met: + The WE checksums the files (*.msi, *.chm, *-pdb.zip), uploads them to + dinsdale together with gpg signature files, and emails you the location + and md5sums. - - PC\icons.mak must have been run with nmake. - - - The cmd.exe window in which this is run must have Cygwin/bin in its - path (atleast for x86). - - - The cmd.exe window must have MS compiler tools for the target - architecture in its path (VS 2003 for x86, the platform SDK for - AMD64). - - - The cmd.exe window must also have cabarc.exe from the CAB SDK in its - path. - - The WE checksums the files (*.msi and *.chm), uploads them to some place - in the net, and emails you the location and md5sums. + ___ The ME builds Mac installer packages and uploads them to dinsdale together + with gpg signature files. ___ Time to build the source tarball. Be sure to update your clone to the correct branch. E.g. @@ -333,13 +306,18 @@ You should not see any files. I.e. you better not have any uncommitted changes in your working directory. - ___ Use the release script to create the source gzip and bz2 tarballs, md5 - checksums, documentation tar and zip files, and gpg signature files. + ___ Make sure you have an up-to-date Sphinx toolchain installed. + + $ pip install -U Sphinx + + ___ Use the release script to create the source gzip and xz tarballs, + documentation tar and zip files, and gpg signature files. $ .../release/release.py --export X.Y.ZaN - This will leave all the relevant files in a subdirectory called - 'X.Y.ZaN/src', and the built docs in 'X.Y.ZaN/docs' (for final releases). + This can take a while for final releases, and it will leave all the + tarballs and signatures in a subdirectory called 'X.Y.ZaN/src', and the + built docs in 'X.Y.ZaN/docs' (for final releases). ___ scp or rsync all the files to your home directory on dinsdale.python.org. @@ -371,7 +349,7 @@ If you're feeling lucky and have some time to kill, or if you are making a release candidate or final release, run the full test suite: - $ make TESTOPTS='-u all' test + $ make testall If the tests pass, then you can feel good that the tarball is fine. If some of the tests fail, or anything else about the @@ -380,7 +358,7 @@ ___ Now you need to go to dinsdale.python.org and move all the files in place over there. Our policy is that every Python version gets its own - directory, but each directory may contain several releases. + directory, but each directory contains all releases of that version. ___ On dinsdale, cd /data/ftp.python.org/pub/python/X.Y.Z creating it if necessary. Make sure it is owned by group 'webmaster' @@ -409,16 +387,15 @@ ___ Let the DE check if the docs are built and work all right. - ___ If this is a major release: Tell the DE to adapt redirects for + ___ If this is a final major release: Tell the DE to adapt redirects for docs.python.org/X.Y in the Apache config for docs.python.org, update the script Doc/tools/dailybuild.py to point to the right stable/development branches, and to install it and make the initial checkout. The Doc's version_switcher.js script also needs to be updated. - ___ For the extra paranoid, do a completely clean test of the - release. This includes downloading the tarball from - www.python.org. + ___ For the extra paranoid, do a completely clean test of the release. + This includes downloading the tarball from www.python.org. Make sure the md5 checksums match. Then unpack the tarball, and do a clean make test. @@ -436,9 +413,7 @@ don't have that, ask someone on pydotorg at python.org for the proper permissions. It's insane for you not to have it. - I'm not going to go into the details of building the site or pushing it - live. All the directories below are named relative to the data subdirectory - unless otherwise noted. + XXX This is completely out of date for Django based python.org. This page will probably come in handy: @@ -550,7 +525,7 @@ ___ You can delete the remote release clone, or simply reuse it for the next release. - + ___ Send email to python-committers informing them that the release has been published. -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Mar 9 10:22:58 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:22:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogaGd0b3VjaDogcmVt?= =?utf-8?q?ove_meta-variable_from_option_spec?= Message-ID: <3fhcYG6qCQz7LjS@mail.python.org> http://hg.python.org/cpython/rev/6e9827ba455f changeset: 89516:6e9827ba455f branch: 3.3 parent: 89500:34df43c9c74a user: Georg Brandl date: Sun Mar 09 10:22:10 2014 +0100 summary: hgtouch: remove meta-variable from option spec This fifth tuple element is unsupported in an old hg version still used on one buildbot, and it is not really necessary (only used in touch --help). files: Tools/hg/hgtouch.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/hg/hgtouch.py b/Tools/hg/hgtouch.py --- a/Tools/hg/hgtouch.py +++ b/Tools/hg/hgtouch.py @@ -125,6 +125,6 @@ cmdtable = { "touch": (touch, - [('b', 'basedir', '', 'base dir of the tree to apply touching', 'BASEDIR')], + [('b', 'basedir', '', 'base dir of the tree to apply touching')], "hg touch [-b BASEDIR]") } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 10:23:00 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:23:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_merge_with_3=2E3?= Message-ID: <3fhcYJ1P4Pz7Lm3@mail.python.org> http://hg.python.org/cpython/rev/c88553c48d76 changeset: 89517:c88553c48d76 parent: 89515:fd53f083768c parent: 89516:6e9827ba455f user: Georg Brandl date: Sun Mar 09 10:22:46 2014 +0100 summary: merge with 3.3 files: Tools/hg/hgtouch.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/hg/hgtouch.py b/Tools/hg/hgtouch.py --- a/Tools/hg/hgtouch.py +++ b/Tools/hg/hgtouch.py @@ -125,6 +125,6 @@ cmdtable = { "touch": (touch, - [('b', 'basedir', '', 'base dir of the tree to apply touching', 'BASEDIR')], + [('b', 'basedir', '', 'base dir of the tree to apply touching')], "hg touch [-b BASEDIR]") } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 10:23:56 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:23:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMDcz?= =?utf-8?q?5=3A_remove_erroneous_deprecated_marker_from_stringprep_docs?= Message-ID: <3fhcZN2FRKz7LjS@mail.python.org> http://hg.python.org/cpython/rev/fadde95c134e changeset: 89518:fadde95c134e branch: 2.7 parent: 89352:0f0e9b7d4f1d user: Georg Brandl date: Mon Feb 24 09:26:53 2014 +0100 summary: Closes #20735: remove erroneous deprecated marker from stringprep docs files: Doc/library/stringprep.rst | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/library/stringprep.rst b/Doc/library/stringprep.rst --- a/Doc/library/stringprep.rst +++ b/Doc/library/stringprep.rst @@ -4,7 +4,6 @@ .. module:: stringprep :synopsis: String preparation, as per RFC 3453 - :deprecated: .. moduleauthor:: Martin v. L?wis .. sectionauthor:: Martin v. L?wis -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 10:23:57 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:23:57 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?= =?utf-8?q?_merge?= Message-ID: <3fhcZP3YYnz7LjS@mail.python.org> http://hg.python.org/cpython/rev/de632ccaffd6 changeset: 89519:de632ccaffd6 branch: 2.7 parent: 89513:1ba8deb4536d parent: 89518:fadde95c134e user: Georg Brandl date: Sun Mar 09 10:23:24 2014 +0100 summary: merge files: Doc/library/stringprep.rst | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/library/stringprep.rst b/Doc/library/stringprep.rst --- a/Doc/library/stringprep.rst +++ b/Doc/library/stringprep.rst @@ -4,7 +4,6 @@ .. module:: stringprep :synopsis: String preparation, as per RFC 3453 - :deprecated: .. moduleauthor:: Martin v. L?wis .. sectionauthor:: Martin v. L?wis -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 10:32:07 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 10:32:07 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Fix_markup=2E?= Message-ID: <3fhclq5Cm0z7LrC@mail.python.org> http://hg.python.org/devguide/rev/57e7eb25a230 changeset: 671:57e7eb25a230 user: Georg Brandl date: Sun Mar 09 10:32:01 2014 +0100 summary: Fix markup. files: gdb.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/gdb.rst b/gdb.rst --- a/gdb.rst +++ b/gdb.rst @@ -277,7 +277,7 @@ #8 Frame 0x7fffd00024a0, for file /home/david/coding/python-svn/Lib/test/lock_tests.py, line 378, in _check_notify (self=, skipped=[], _mirrorOutput=False, testsRun=39, buffer=False, _original_stderr=, _stdout_buffer=, _stderr_buffer=, _moduleSetUpFailed=False, expectedFailures=[], errors=[], _previousTestClass=, unexpectedSuccesses=[], failures=[], shouldStop=False, failfast=False) at remote 0xc185a0>, _threads=(0,), _cleanups=[], _type_equality_funcs={: , : , : , : , http://hg.python.org/cpython/rev/1f2c16c7ecc2 changeset: 89520:1f2c16c7ecc2 branch: 3.3 parent: 89458:ca5635efe090 user: Georg Brandl date: Sun Mar 02 09:29:00 2014 +0100 summary: Added tag v3.3.5rc2 for changeset ca5635efe090 files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -121,3 +121,4 @@ fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 +ca5635efe090f78806188ac2758f9948596aa8b2 v3.3.5rc2 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:18:28 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:18:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQnVtcCB0byAzLjMu?= =?utf-8?q?5_final=2E?= Message-ID: <3fhdnJ3qKwz7LlV@mail.python.org> http://hg.python.org/cpython/rev/62cf4e77f785 changeset: 89521:62cf4e77f785 branch: 3.3 tag: v3.3.5 user: Georg Brandl date: Sun Mar 09 09:37:14 2014 +0100 summary: Bump to 3.3.5 final. files: Include/patchlevel.h | 6 +++--- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Misc/NEWS | 8 ++++++++ Misc/RPM/python-3.3.spec | 2 +- README | 4 ++-- 6 files changed, 16 insertions(+), 8 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -19,11 +19,11 @@ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 3 #define PY_MICRO_VERSION 5 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 2 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL +#define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.3.5rc2" +#define PY_VERSION "3.3.5" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.5rc2" +__version__ = "3.3.5" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.5rc2" +IDLE_VERSION = "3.3.5" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,14 @@ Python News +++++++++++ +What's New in Python 3.3.5? +=========================== + +*Release date: 09-Mar-2014* + +No changes from release candidate 2. + + What's New in Python 3.3.5 release candidate 2? =============================================== diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.5rc2 +%define version 3.3.5 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,5 +1,5 @@ -This is Python version 3.3.5 release candidate 2 -================================================ +This is Python version 3.3.5 +============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Python Software Foundation. All rights reserved. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:18:29 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:18:29 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Added_tag_v3?= =?utf-8?q?=2E3=2E5_for_changeset_62cf4e77f785?= Message-ID: <3fhdnK58Tsz7Llh@mail.python.org> http://hg.python.org/cpython/rev/993955b807b3 changeset: 89522:993955b807b3 branch: 3.3 user: Georg Brandl date: Sun Mar 09 09:37:32 2014 +0100 summary: Added tag v3.3.5 for changeset 62cf4e77f785 files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -122,3 +122,4 @@ 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 ca5635efe090f78806188ac2758f9948596aa8b2 v3.3.5rc2 +62cf4e77f78564714e7ea3d4bf1479ca1fbd0758 v3.3.5 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:18:30 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:18:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Post-release_u?= =?utf-8?q?pdate=2E?= Message-ID: <3fhdnL6c5hz7Lqv@mail.python.org> http://hg.python.org/cpython/rev/732401b87be6 changeset: 89523:732401b87be6 branch: 3.3 user: Georg Brandl date: Sun Mar 09 11:01:14 2014 +0100 summary: Post-release update. files: Include/patchlevel.h | 2 +- Misc/NEWS | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -23,7 +23,7 @@ #define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.3.5" +#define PY_VERSION "3.3.5+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,18 @@ Python News +++++++++++ +What's New in Python 3.3.6 release candidate 1? +=============================================== + +*Release date: XX-XXX-XXXX* + +Core and Builtins +----------------- + +Library +------- + + What's New in Python 3.3.5? =========================== -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:18:32 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:18:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?= =?utf-8?q?_merge_with_3=2E3=2E5_release_repo?= Message-ID: <3fhdnN1R0Dz7Lrh@mail.python.org> http://hg.python.org/cpython/rev/95d8ca61cdbf changeset: 89524:95d8ca61cdbf branch: 3.3 parent: 89516:6e9827ba455f parent: 89523:732401b87be6 user: Georg Brandl date: Sun Mar 09 11:17:23 2014 +0100 summary: merge with 3.3.5 release repo files: .hgtags | 2 + Include/patchlevel.h | 6 ++-- Lib/distutils/__init__.py | 2 +- Lib/idlelib/idlever.py | 2 +- Misc/NEWS | 28 +++++++++++++++++++++++--- Misc/RPM/python-3.3.spec | 2 +- README | 4 +- 7 files changed, 34 insertions(+), 12 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -121,3 +121,5 @@ fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 +ca5635efe090f78806188ac2758f9948596aa8b2 v3.3.5rc2 +62cf4e77f78564714e7ea3d4bf1479ca1fbd0758 v3.3.5 diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -19,11 +19,11 @@ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 3 #define PY_MICRO_VERSION 5 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 2 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL +#define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.3.5rc2" +#define PY_VERSION "3.3.5+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -13,5 +13,5 @@ # Updated automatically by the Python release process. # #--start constants-- -__version__ = "3.3.5rc2" +__version__ = "3.3.5" #--end constants-- diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,1 +1,1 @@ -IDLE_VERSION = "3.3.5rc2" +IDLE_VERSION = "3.3.5" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,30 @@ Python News +++++++++++ +What's New in Python 3.3.6 release candidate 1? +=============================================== + +*Release date: XX-XXX-XXXX* + +Core and Builtins +----------------- + +Library +------- + +- Issue #20283: RE pattern methods now accept the string keyword parameters + as documented. The pattern and source keyword parameters are left as + deprecated aliases. + + +What's New in Python 3.3.5? +=========================== + +*Release date: 09-Mar-2014* + +No changes from release candidate 2. + + What's New in Python 3.3.5 release candidate 2? =============================================== @@ -22,10 +46,6 @@ Library ------- -- Issue #20283: RE pattern methods now accept the string keyword parameters - as documented. The pattern and source keyword parameters are left as - deprecated aliases. - - Issue #20778: Fix modulefinder to work with bytecode-only modules. - Issue #20791: copy.copy() now doesn't make a copy when the input is diff --git a/Misc/RPM/python-3.3.spec b/Misc/RPM/python-3.3.spec --- a/Misc/RPM/python-3.3.spec +++ b/Misc/RPM/python-3.3.spec @@ -39,7 +39,7 @@ %define name python #--start constants-- -%define version 3.3.5rc2 +%define version 3.3.5 %define libvers 3.3 #--end constants-- %define release 1pydotorg diff --git a/README b/README --- a/README +++ b/README @@ -1,5 +1,5 @@ -This is Python version 3.3.5 release candidate 2 -================================================ +This is Python version 3.3.5 +============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Python Software Foundation. All rights reserved. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:18:33 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:18:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_merge_with_3=2E3?= Message-ID: <3fhdnP3H7qz7Lrq@mail.python.org> http://hg.python.org/cpython/rev/3ae2cd85a908 changeset: 89525:3ae2cd85a908 parent: 89517:c88553c48d76 parent: 89524:95d8ca61cdbf user: Georg Brandl date: Sun Mar 09 11:18:16 2014 +0100 summary: merge with 3.3 files: .hgtags | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -121,6 +121,8 @@ fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 9ec811df548ed154a9bf9815383a916d6df31b98 v3.3.5rc1 +ca5635efe090f78806188ac2758f9948596aa8b2 v3.3.5rc2 +62cf4e77f78564714e7ea3d4bf1479ca1fbd0758 v3.3.5 46535f65e7f3bcdcf176f36d34bc1fed719ffd2b v3.4.0a1 9265a2168e2cb2a84785d8717792acc661e6b692 v3.4.0a2 dd9cdf90a5073510877e9dd5112f8e6cf20d5e89 v3.4.0a3 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 11:28:27 2014 From: python-checkins at python.org (georg.brandl) Date: Sun, 9 Mar 2014 11:28:27 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Small_fixes_to_the_add-to-?= =?utf-8?q?pydotorg_script=2E?= Message-ID: <3fhf0q2fsKz7LlK@mail.python.org> http://hg.python.org/release/rev/31cba510ad2d changeset: 83:31cba510ad2d user: Georg Brandl date: Sun Mar 09 11:28:19 2014 +0100 summary: Small fixes to the add-to-pydotorg script. files: add-to-pydotorg.py | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py --- a/add-to-pydotorg.py +++ b/add-to-pydotorg.py @@ -26,6 +26,7 @@ import re import sys import json +import time import hashlib from os import path @@ -71,7 +72,7 @@ ('-' + release[5:] if release[5:] else '') def sigfile_for(release, rfile): - return download_root + '%s/%s.asc' % (release[:5], rfile) + return download_root + '%s/%s.asc' % (release, rfile) def md5sum_for(release, rfile): return hashlib.md5(open(ftp_root + release[:5] + '/' + rfile, 'rb').read()).hexdigest() @@ -100,13 +101,13 @@ """Return a dictionary with all needed fields for a ReleaseFile object.""" return dict( name = file_desc, - slug = slug_for(release) + '-' + make_slug(file_desc), + slug = slug_for(release) + '-' + make_slug(file_desc)[:40], os = '/api/v1/downloads/os/%s/' % os_pk, release = '/api/v1/downloads/release/%s/' % rel_pk, description = add_desc, is_source = os_pk == 3, url = download_root + '%s/%s' % (release[:5], rfile), - gpg_signature_file = sigfile_for(release, rfile), + gpg_signature_file = sigfile_for(release[:5], rfile), md5_sum = md5sum_for(release, rfile), filesize = filesize_for(release, rfile), download_button = 'tar.xz' in rfile or @@ -165,10 +166,13 @@ data=json.dumps(datadict), headers=headers) if resp.status_code != 201: try: - print resp.json['traceback'] + info = json.loads(resp.text) + print info.get('error_message', 'No error message.') + print info.get('traceback', '') except: pass - raise RuntimeError('creating %s failed: %s' % (objtype, resp.status_code)) + print 'Creating %s failed: %s' % (objtype, resp.status_code) + return -1 newloc = resp.headers['Location'] pk = int(newloc.strip('/').split('/')[-1]) return pk @@ -188,8 +192,9 @@ print 'Creating ReleaseFile object for', rfile file_dict = build_file_dict(rel, rfile, rel_pk, file_desc, os_pk, add_desc) file_pk = post_object('release_file', file_dict) - print 'Created as id =', file_pk - n += 1 + if file_pk >= 0: + print 'Created as id =', file_pk + n += 1 print 'Done - %d files added' % n main() -- Repository URL: http://hg.python.org/release From python-checkins at python.org Sun Mar 9 20:01:26 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 9 Mar 2014 20:01:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_note_that_futu?= =?utf-8?q?re=5Fbuiltin=27s_map_is_not_quite_like_python_3=27s_=28closes_?= =?utf-8?q?=2319363=29?= Message-ID: <3fhsNk2fMKz7Ll2@mail.python.org> http://hg.python.org/cpython/rev/16c5d7c289c6 changeset: 89526:16c5d7c289c6 branch: 2.7 parent: 89519:de632ccaffd6 user: Benjamin Peterson date: Sun Mar 09 14:01:09 2014 -0500 summary: note that future_builtin's map is not quite like python 3's (closes #19363) Patch by Gareth Rees. files: Doc/library/future_builtins.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/future_builtins.rst b/Doc/library/future_builtins.rst --- a/Doc/library/future_builtins.rst +++ b/Doc/library/future_builtins.rst @@ -50,6 +50,11 @@ Works like :func:`itertools.imap`. + .. note:: + + In Python 3, :func:`map` does not accept ``None`` for the + function argument. + .. function:: oct(object) Works like the built-in :func:`oct`, but instead of :meth:`__oct__` it will -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:44 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:44 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_hmac_*digestmo?= =?utf-8?q?d*_accepts_strings=2C_and_default_is_deprecated=2E_=28=2317276?= =?utf-8?q?=29?= Message-ID: <3fhslX10tNz7Lmn@mail.python.org> http://hg.python.org/cpython/rev/c10ec51a2ce4 changeset: 89527:c10ec51a2ce4 parent: 89525:3ae2cd85a908 user: R David Murray date: Sat Mar 08 22:43:12 2014 -0500 summary: whatsnew: hmac *digestmod* accepts strings, and default is deprecated. (#17276) files: Doc/library/hmac.rst | 5 ++--- Doc/whatsnew/3.4.rst | 15 ++++++++++++++- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -23,9 +23,8 @@ defaults to the :data:`hashlib.md5` constructor. .. versionchanged:: 3.4 - Parameter *key* can be a bytes or bytearray object. Parameter *msg* can - be of any type supported by :mod:`hashlib`. - + Parameter *key* can be a bytes or bytearray object. + Parameter *msg* can be of any type supported by :mod:`hashlib`. Paramter *digestmod* can be the name of a hash algorithm. .. deprecated:: 3.4 diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -826,6 +826,12 @@ accepts any type supported by the :mod:`hashlib` module. (Contributed by Jonas Borgstr?m in :issue:`18240`.) +The *digestmod* argument to the :func:`hmac.new` function may now be any hash +digest name recognized by :mod:`hashlib`. In addition, the current behavior in +which the value of *digestmod* defaults to ``MD5`` is deprecated: in a +future version of Python there will be no default value. (Contributed by +Christian Heimes in :issue:`17276`.) + html ---- @@ -1849,6 +1855,9 @@ * The *strict* argument of :class:`~html.parser.HTMLParser` is deprecated. +* The :func:`hmac.new` *digestmod* keyword having a default value (currently + ``MD5``) is deprecated. + Deprecated Functions and Types in the C API ------------------------------------------- @@ -2139,7 +2148,11 @@ currently defaults to ``False`` for backward compatibility, but will eventually be changed to default to ``True``. It is recommended that you add this keyword, with the appropriate value, to any - :class:`~html.parser.HTMLParser` calls in your code. + :class:`~html.parser.HTMLParser` calls in your code (:issue:`13633`). + +* Since the *digestmod* argument to the :func:`hmac.new` function will in the + future have no default, all calls to :func:`hmac.new` should be changed to + explicitly specify a *digestmod*. (:issue:`17276`). Changes in the C API -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:45 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:45 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_hmac_conforms_?= =?utf-8?q?to_pep_247=2E_=28=2318775=29?= Message-ID: <3fhslY3bVrz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/dde3fb877b07 changeset: 89528:dde3fb877b07 user: R David Murray date: Sat Mar 08 23:18:09 2014 -0500 summary: whatsnew: hmac conforms to pep 247. (#18775) files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -832,6 +832,11 @@ future version of Python there will be no default value. (Contributed by Christian Heimes in :issue:`17276`.) +With the addition of :attr:`~hmac.HMAC.block_size` and :attr:`~hmac.HMAC.name` +attributes (and the formal documentation of the :attr:`~hmac.HMAC.digest_size` +attribute), the :mod:`hmac` module now conforms fully to the :pep:`247` API. +(Contributed by Christian Heimes in :issue:`18775`.) + html ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:46 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:46 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_SSLcontext=2El?= =?utf-8?q?oad=5Fverify=5Flocations_cadata_argument_=28=2318138=29?= Message-ID: <3fhslZ51njz7Ls4@mail.python.org> http://hg.python.org/cpython/rev/8e3b3b4a90fb changeset: 89529:8e3b3b4a90fb user: R David Murray date: Sat Mar 08 23:19:45 2014 -0500 summary: whatsnew: SSLcontext.load_verify_locations cadata argument (#18138) files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1269,6 +1269,11 @@ Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orr? and Antoine Pitrou in :issue:`16692`) +:class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_verify_locations` +accepts a new optional argument *cadata*, which can be used to provide PEM or +DER encoded certificates directly via strings or bytes, respectively. +(Contributed by Christian Heimes in :issue:`18138`.) + New function :func:`~ssl.get_default_verify_paths` returns a named tuple of the paths and environment variables that the :meth:`~ssl.SSLContext.set_default_verify_paths` method uses to set -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:48 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:48 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogd2hhdHNuZXc6IHN5cy5fX2lu?= =?utf-8?b?dGVyYWN0aXZlaG9va19fLiAoIzU4NDUp?= Message-ID: <3fhslc1GgNz7Lrp@mail.python.org> http://hg.python.org/cpython/rev/69c451851c71 changeset: 89530:69c451851c71 user: R David Murray date: Sun Mar 09 12:03:30 2014 -0400 summary: whatsnew: sys.__interactivehook__. (#5845) I decided to leave the (slightly expanded) Other Improvements entry as well, since someone looking for command line behavior changes isn't going to look in the 'sys' module section. files: Doc/library/site.rst | 10 +++++++--- Doc/library/sys.rst | 9 +++++---- Doc/whatsnew/3.4.rst | 17 ++++++++++++++++- 3 files changed, 28 insertions(+), 8 deletions(-) diff --git a/Doc/library/site.rst b/Doc/library/site.rst --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -123,9 +123,13 @@ configure the :mod:`rlcompleter` module, if Python is started in :ref:`interactive mode ` and without the :option:`-S` option. The default behavior is enable tab-completion and to use -:file:`~/.python_history` as the history save file. To disable it, override -the :data:`sys.__interactivehook__` attribute in your :mod:`sitecustomize` -or :mod:`usercustomize` module or your :envvar:`PYTHONSTARTUP` file. +:file:`~/.python_history` as the history save file. To disable it, delete (or +override) the :data:`sys.__interactivehook__` attribute in your +:mod:`sitecustomize` or :mod:`usercustomize` module or your +:envvar:`PYTHONSTARTUP` file. + +.. versionchanged:: 3.4 + Activation of rlcompleter and history was made automatic. Module contents diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -692,10 +692,11 @@ .. data:: __interactivehook__ - When present, this function is automatically called (with no arguments) - when the interpreter is launched in :ref:`interactive mode `. - This is done after the :envvar:`PYTHONSTARTUP` file is read, so that you - can set this hook there. + When this attribute exists, its value is automatically called (with no + arguments) when the interpreter is launched in :ref:`interactive mode + `. This is done after the :envvar:`PYTHONSTARTUP` file is + read, so that you can set this hook there. The :mod:`site` module + :ref:`sets this `. .. versionadded:: 3.4 diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1360,6 +1360,19 @@ especially if automated via a test suite. (Contributed by Antoine Pitrou in :issue:`13390`.) +When the Python interpreter starts in :ref:`interactive mode +`, it checks for an :data:`~sys.__interactivehook__` attribute +on the :mod:`sys` module. If the attribute exists, its value is called with no +arguments just before interactive mode is started. The check is made after the +:envvar:`PYTHONSTARTUP` file is read, so it can be set there. The :mod:`site` +module :ref:`sets it ` to a function that enables tab +completion and history saving (in :file:`~/.python-history`) if the platform +supports :mod:`readline`. If you do not want this (new) behavior, you can +override it in :envvar:`PYTHONSTARTUP`, :mod:`sitecustomize`, or +:mod:`usercustomize` by deleting this attribute from :mod:`sys` (or setting it +to some other callable). (Contributed by ?ric Araujo and Antoine Pitrou in +:issue:`5845`.) + textwrap -------- @@ -1694,7 +1707,9 @@ most POSIX systems it can and should be used in the ``#!`` line of system scripts. (Contributed by Christian Heimes in :issue:`16499`.) -* Tab-completion is now enabled by default in the interactive interpreter. +* Tab-completion is now enabled by default in the interactive interpreter + on systems that support :mod:`readline`. History is also enabled by default, + and is written to (and read from) the file :file:`~/.python-history`. (Contributed by Antoine Pitrou and ?ric Araujo in :issue:`5845`.) * Invoking the Python interpreter with ``--version`` now outputs the version to -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:49 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:49 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_SSLSocket=2Ege?= =?utf-8?q?tpeercert_new_values_returned=2E_=28=2318379=29?= Message-ID: <3fhsld2w0Nz7Ll8@mail.python.org> http://hg.python.org/cpython/rev/7885876b6503 changeset: 89531:7885876b6503 user: R David Murray date: Sun Mar 09 12:55:58 2014 -0400 summary: whatsnew: SSLSocket.getpeercert new values returned. (#18379) files: Doc/library/ssl.rst | 4 +--- Doc/whatsnew/3.4.rst | 4 ++++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -841,10 +841,8 @@ .. versionchanged:: 3.4 :exc:`ValueError` is raised when the handshake isn't done. - - .. versionchanged:: 3.4 The returned dictionary includes additional X509v3 extension items - such as ``crlDistributionPoints``, ``caIssuers`` and ``OCSP`` URIs. + such as ``crlDistributionPoints``, ``caIssuers`` and ``OCSP`` URIs. .. method:: SSLSocket.cipher() diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1297,6 +1297,10 @@ :meth:`ssl.SSLContext.set_servername_callback` method. (Contributed by Daniel Black in :issue:`8109`.) +The dictionary returned by :meth:`.SSLSocket.getpeercert` contains additional +``X509v3`` extension items: ``crlDistributionPoints``, ``calIssuers``, and +``OCSP`` URIs. (Contributed by Christian Heimes in :issue:`18379`.) + stat ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:17:50 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:17:50 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_plistlib_new_a?= =?utf-8?q?pi_and_deprecations_=28=2314455=29?= Message-ID: <3fhslf4b12z7Lll@mail.python.org> http://hg.python.org/cpython/rev/728f626ee337 changeset: 89532:728f626ee337 user: R David Murray date: Sun Mar 09 15:09:04 2014 -0400 summary: whatsnew: plistlib new api and deprecations (#14455) Also fixed the versionchanged/deprecation tags in the docs. files: Doc/library/plistlib.rst | 13 ++++++++----- Doc/whatsnew/3.4.rst | 17 ++++++++++++++--- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/Doc/library/plistlib.rst b/Doc/library/plistlib.rst --- a/Doc/library/plistlib.rst +++ b/Doc/library/plistlib.rst @@ -32,6 +32,9 @@ (but only with string keys), :class:`Data`, :class:`bytes`, :class:`bytesarray` or :class:`datetime.datetime` objects. +.. versionchanged:: 3.4 + New API, old API deprecated. Support for binary format plists added. + .. seealso:: `PList manual page `_ @@ -107,7 +110,7 @@ An :exc:`OverflowError` will be raised for integer values that cannot be represented in (binary) plist files. - .. versionadded: 3.4 + .. versionadded:: 3.4 .. function:: dumps(value, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False) @@ -116,7 +119,7 @@ the documentation for :func:`dump` for an explanation of the keyword arguments of this function. - .. versionadded: 3.4 + .. versionadded:: 3.4 The following functions are deprecated: @@ -135,7 +138,7 @@ to ``__getitem_``. This means that you can use attribute access to access items of these dictionaries. - .. deprecated: 3.4 Use :func:`load` instead. + .. deprecated:: 3.4 Use :func:`load` instead. .. function:: writePlist(rootObject, pathOrFile) @@ -143,7 +146,7 @@ Write *rootObject* to an XML plist file. *pathOrFile* may be either a file name or a (writable and binary) file object - .. deprecated: 3.4 Use :func:`dump` instead. + .. deprecated:: 3.4 Use :func:`dump` instead. .. function:: readPlistFromBytes(data) @@ -194,7 +197,7 @@ .. deprecated:: 3.4 Use a :class:`bytes` object instead -The following constants are avaiable: +The following constants are available: .. data:: FMT_XML diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1097,9 +1097,13 @@ plistlib -------- -:mod:`plistlib` now supports binary plist files, and offers the common -``load``/``loads``/``dump``/``dumps`` API pattern for serialization formats -(Contributed by Ronald Oussoren and others in :issue:`14455`). +:mod:`plistlib` now has an API that is similar to the standard pattern for +stdlib serialization protocols, with new :func:`~plistlib.load`, +:func:`~plistlib.dump`, :func:`~plistlib.loads`, and :func:`~plistlib.dumps` +functions. (The older API is now deprecated.) In addition to the already +supported XML plist format (:data:`~plistlib.FMT_XML`), it also now supports +the binary plist format (:data:`~plistlib.FMT_BINARY`). (Contributed by Ronald +Oussoren and others in :issue:`14455`). poplib @@ -1887,6 +1891,13 @@ * The :func:`hmac.new` *digestmod* keyword having a default value (currently ``MD5``) is deprecated. +* The :mod:`plistlib` :func:`~plistlib.readPlist`, + :func:`~plistlib.writePlist`, :func:`~plistlib.readPlistFromBytes`, and + :func:`~plistlib.writePlistToBytes` functions are deprecated in favor of the + corresponding new functions :func:`~plistlib.load`, :func:`~plistlib.dump`, + :func:`~plistlib.loads`, and :func:`~plistlib.dumps`. :func:`~plistlib.Data` + is deprecated in favor of just using the :class:`bytes` constructor. + Deprecated Functions and Types in the C API ------------------------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 20:30:34 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 20:30:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2320871=3A_improve_email_?= =?utf-8?q?policy_test_coverage=2E?= Message-ID: <3fht2L65zPz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/8f7486263212 changeset: 89533:8f7486263212 user: R David Murray date: Sun Mar 09 15:29:24 2014 -0400 summary: #20871: improve email policy test coverage. More tests of the concrete policy methods would probably be a good idea, but this is a start, and it brings line coverage up to 100% for the policy module. Patch by Milan Oberkirch. files: Lib/test/test_email/test_policy.py | 9 +++++++++ Misc/ACKS | 1 + 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_email/test_policy.py b/Lib/test/test_email/test_policy.py --- a/Lib/test/test_email/test_policy.py +++ b/Lib/test/test_email/test_policy.py @@ -319,5 +319,14 @@ self.assertEqual(msg.as_string(), "Subject: testXTo: fooXX") +class TestConcretePolicies(unittest.TestCase): + + def test_header_store_parse_rejects_newlines(self): + instance = email.policy.EmailPolicy() + self.assertRaises(ValueError, + instance.header_store_parse, + 'From', 'spam\negg at foo.py') + + if __name__ == '__main__': unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -951,6 +951,7 @@ James Oakley Elena Oat Jon Oberheide +Milan Oberkirch Pascal Oberndoerfer Jeffrey Ollie Adam Olsen -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 22:48:32 2014 From: python-checkins at python.org (ned.deily) Date: Sun, 9 Mar 2014 22:48:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwODc1?= =?utf-8?q?=3A_Prevent_possible_gzip_=22=27read=27_is_not_defined=22_NameE?= =?utf-8?q?rror=2E?= Message-ID: <3fhx5X2SRpz7LjP@mail.python.org> http://hg.python.org/cpython/rev/c3836de644e0 changeset: 89534:c3836de644e0 branch: 3.3 parent: 89524:95d8ca61cdbf user: Ned Deily date: Sun Mar 09 14:44:34 2014 -0700 summary: Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. files: Lib/gzip.py | 2 +- Lib/test/test_gzip.py | 7 +++++++ Misc/NEWS | 3 +++ 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/gzip.py b/Lib/gzip.py --- a/Lib/gzip.py +++ b/Lib/gzip.py @@ -99,7 +99,7 @@ self._read -= len(prepend) return else: - self._buffer = self._buffer[read:] + prepend + self._buffer = self._buffer[self._read:] + prepend self._length = len(self._buffer) self._read = 0 diff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.py --- a/Lib/test/test_gzip.py +++ b/Lib/test/test_gzip.py @@ -396,6 +396,13 @@ with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f: self.assertEqual(f.read(), b'Test') + def test_prepend_error(self): + # See issue #20875 + with gzip.open(self.filename, "wb") as f: + f.write(data1) + with gzip.open(self.filename, "rb") as f: + f.fileobj.prepend() + class TestOpen(BaseTest): def test_binary_modes(self): uncompressed = data1 * 50 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Issue #20875: Prevent possible gzip "'read' is not defined" NameError. + Patch by Claudiu Popa. + - Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 22:48:33 2014 From: python-checkins at python.org (ned.deily) Date: Sun, 9 Mar 2014 22:48:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320875=3A_Merge_from_3=2E3?= Message-ID: <3fhx5Y4Gnvz7LjP@mail.python.org> http://hg.python.org/cpython/rev/42599de6a4de changeset: 89535:42599de6a4de parent: 89533:8f7486263212 parent: 89534:c3836de644e0 user: Ned Deily date: Sun Mar 09 14:47:58 2014 -0700 summary: Issue #20875: Merge from 3.3 files: Lib/gzip.py | 2 +- Lib/test/test_gzip.py | 7 +++++++ Misc/NEWS | 3 +++ 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/gzip.py b/Lib/gzip.py --- a/Lib/gzip.py +++ b/Lib/gzip.py @@ -96,7 +96,7 @@ self._read -= len(prepend) return else: - self._buffer = self._buffer[read:] + prepend + self._buffer = self._buffer[self._read:] + prepend self._length = len(self._buffer) self._read = 0 diff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.py --- a/Lib/test/test_gzip.py +++ b/Lib/test/test_gzip.py @@ -421,6 +421,13 @@ with gzip.GzipFile(fileobj=io.BytesIO(gzdata)) as f: self.assertEqual(f.read(), b'Test') + def test_prepend_error(self): + # See issue #20875 + with gzip.open(self.filename, "wb") as f: + f.write(data1) + with gzip.open(self.filename, "rb") as f: + f.fileobj.prepend() + class TestOpen(BaseTest): def test_binary_modes(self): uncompressed = data1 * 50 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,9 @@ Library ------- +- Issue #20875: Prevent possible gzip "'read' is not defined" NameError. + Patch by Claudiu Popa. + - Issue #11558: ``email.message.Message.attach`` now returns a more useful error message if ``attach`` is called on a message for which ``is_multipart`` is False. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:50:27 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:50:27 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_SSLContext=2Ev?= =?utf-8?q?erify=5Fflags_and_constants=2E_=28=238813=29?= Message-ID: <3fhySz0KKqz7Ll3@mail.python.org> http://hg.python.org/cpython/rev/1508c4c9e747 changeset: 89536:1508c4c9e747 user: R David Murray date: Sun Mar 09 17:01:34 2014 -0400 summary: whatsnew: SSLContext.verify_flags and constants. (#8813) files: Doc/whatsnew/3.4.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1305,6 +1305,14 @@ ``X509v3`` extension items: ``crlDistributionPoints``, ``calIssuers``, and ``OCSP`` URIs. (Contributed by Christian Heimes in :issue:`18379`.) +If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has an new +attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the +certificate verification process by setting it to some combination of the new +constants :data:`~ssl.VERIFY_DEFAULT`, :data:`~ssl.VERIFY_CRL_CHECK_LEAF`, +:data:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :data:`~ssl.VERIFY_X509_STRICT`. +OpenSSL does not do any CRL verification by default. (Contributed by +Christien Heimes in :issue:`8813`.) + stat ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:50:28 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:50:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_deprecation_of?= =?utf-8?q?_sysconfig_SO_key_=28=2319555=29=2E?= Message-ID: <3fhyT025x6z7LmQ@mail.python.org> http://hg.python.org/cpython/rev/ac2ee9fc353a changeset: 89537:ac2ee9fc353a user: R David Murray date: Sun Mar 09 17:11:00 2014 -0400 summary: whatsnew: deprecation of sysconfig SO key (#19555). files: Doc/whatsnew/3.4.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1906,6 +1906,9 @@ :func:`~plistlib.loads`, and :func:`~plistlib.dumps`. :func:`~plistlib.Data` is deprecated in favor of just using the :class:`bytes` constructor. +* The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by + ``EXT_SUFFIX``. + Deprecated Functions and Types in the C API ------------------------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:50:29 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:50:29 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_more_accurate_?= =?utf-8?q?deprecation_section_titles=2E?= Message-ID: <3fhyT13mpwz7Lls@mail.python.org> http://hg.python.org/cpython/rev/4d9e0a6b0ddc changeset: 89538:4d9e0a6b0ddc user: R David Murray date: Sun Mar 09 17:13:31 2014 -0400 summary: whatsnew: more accurate deprecation section titles. files: Doc/whatsnew/3.4.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1845,8 +1845,8 @@ using ``-Wd``). -Deprecated Python Modules, Functions and Methods ------------------------------------------------- +Deprecations in the Python API +------------------------------ * :meth:`difflib.SequenceMatcher.isbjunk` and :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and @@ -1910,8 +1910,8 @@ ``EXT_SUFFIX``. -Deprecated Functions and Types in the C API -------------------------------------------- +Deprecations in the C API +------------------------- XXX: None so far -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:50:30 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:50:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_venv_with=5Fpi?= =?utf-8?q?p=2C_pyvenv_--without-pip_=28=2319552=29?= Message-ID: <3fhyT2594sz7Lm1@mail.python.org> http://hg.python.org/cpython/rev/a140caad76bc changeset: 89539:a140caad76bc user: R David Murray date: Sun Mar 09 18:09:51 2014 -0400 summary: whatsnew: venv with_pip, pyvenv --without-pip (#19552) files: Doc/whatsnew/3.4.rst | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1507,6 +1507,12 @@ :mod:`venv` now includes activation scripts for the ``csh`` and ``fish`` shells (Contributed by Andrew Svetlov in :issue:`15417`.) +:class:`~venv.EnvBuilder` and the :func:`~venv.create` convenience function +take a new keyword argument *with_pip*, which defaults to ``False``, that +controls whether or not :class:`~venv.EnvBuilder` ensures that ``pip`` is +installed in the virtual environment. (Contributed by Nick Coghlan in +:issue:`19552` as part of the :pep:`453` implementation.) + wave ---- @@ -1772,6 +1778,11 @@ to use copies rather than symlinks even on systems where symlinks are the default. (Contributed by Vinay Sajip in :issue:`18807`.) +* The :ref:`pyvenv ` command also accepts a ``--without-pip`` + option to suppress the otherwise-automatic bootstrapping of pip into + the virtual environment. (Contributed by Nick Coghlan in :issue:`19552` + as part of the :pep:`453` implementation.) + * The encoding name is now optional in the value set for the :envvar:`PYTHONIOENCODING` environment variable. This makes it possible to set just the error handler, without changing the default encoding. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:52:09 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:52:09 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE5OTUzOiBDbGFy?= =?utf-8?q?ify_the_wording_of_the_augmented_assignment_discussion=2E?= Message-ID: <3fhyVx2SWCz7Ljb@mail.python.org> http://hg.python.org/cpython/rev/f9cb5a44879c changeset: 89540:f9cb5a44879c branch: 3.3 parent: 89534:c3836de644e0 user: R David Murray date: Sun Mar 09 18:51:16 2014 -0400 summary: #19953: Clarify the wording of the augmented assignment discussion. Patch by Priya Pappachan, based on suggestions from Terry Reedy and myself. files: Doc/faq/programming.rst | 1 + Doc/reference/datamodel.rst | 12 +++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1103,6 +1103,7 @@ result = [obj.method() for obj in mylist] +.. _faq-augmented-assignment-tuple-error: Why does a_tuple[i] += ['item'] raise an exception when the addition works? --------------------------------------------------------------------------- diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2023,11 +2023,13 @@ ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation in-place (modifying *self*) and return the result (which could be, but does not have to be, *self*). If a specific method is not defined, the augmented - assignment falls back to the normal methods. For instance, to execute the - statement ``x += y``, where *x* is an instance of a class that has an - :meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance - of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)`` - and ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. + assignment falls back to the normal methods. For instance, if *x* is an + instance of a class with an :meth:`__iadd__` method, ``x += y`` is equivalent + to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and ``y.__radd__(x)`` + are considered, as with the evaluation of ``x + y``. In certain situations, + augmented assignment can result in unexpected errors (see + :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in + fact part of the data model. .. method:: object.__neg__(self) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 9 23:52:10 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 9 Mar 2014 23:52:10 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2319953=3A_Clarify_the_wording_of_the_augmented_a?= =?utf-8?q?ssignment_discussion=2E?= Message-ID: <3fhyVy4T16z7Lm1@mail.python.org> http://hg.python.org/cpython/rev/61ceb299a255 changeset: 89541:61ceb299a255 parent: 89539:a140caad76bc parent: 89540:f9cb5a44879c user: R David Murray date: Sun Mar 09 18:51:35 2014 -0400 summary: Merge #19953: Clarify the wording of the augmented assignment discussion. files: Doc/faq/programming.rst | 1 + Doc/reference/datamodel.rst | 12 +++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1103,6 +1103,7 @@ result = [obj.method() for obj in mylist] +.. _faq-augmented-assignment-tuple-error: Why does a_tuple[i] += ['item'] raise an exception when the addition works? --------------------------------------------------------------------------- diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2050,11 +2050,13 @@ ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation in-place (modifying *self*) and return the result (which could be, but does not have to be, *self*). If a specific method is not defined, the augmented - assignment falls back to the normal methods. For instance, to execute the - statement ``x += y``, where *x* is an instance of a class that has an - :meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance - of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)`` - and ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. + assignment falls back to the normal methods. For instance, if *x* is an + instance of a class with an :meth:`__iadd__` method, ``x += y`` is equivalent + to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and ``y.__radd__(x)`` + are considered, as with the evaluation of ``x + y``. In certain situations, + augmented assignment can result in unexpected errors (see + :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in + fact part of the data model. .. method:: object.__neg__(self) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:06 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_pdb_works_for_?= =?utf-8?q?debugging_asyncio_programs_=28=2316596=29=2E?= Message-ID: <3fj26y1Y81z7LkY@mail.python.org> http://hg.python.org/cpython/rev/5c6c96c82afb changeset: 89542:5c6c96c82afb user: R David Murray date: Sun Mar 09 19:40:30 2014 -0400 summary: whatsnew: pdb works for debugging asyncio programs (#16596). files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1066,6 +1066,11 @@ pdb --- +:mod:`pdb` has been enhanced to handle generators, :keyword:`yield`, and +:keyword:`yield from` in a more useful fashion. This is especially helpful when +debugging :mod:`asyncio` based programs. (Contributed by Andrew Svetlov and +Xavier de Gaye in :issue:`16596`.) + The ``print`` command has been removed from :mod:`pdb`, restoring access to the Python :func:`print` function from the pdb command line. Python2's ``pdb`` did not have a ``print`` command; instead, entering ``print`` executed the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:07 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:07 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_SSLContext=2El?= =?utf-8?b?b2FkX2RlZmF1bHRfY2VydHMgKCMxOTI5Miku?= Message-ID: <3fj26z3QFQz7LkB@mail.python.org> http://hg.python.org/cpython/rev/35a5284d5388 changeset: 89543:35a5284d5388 user: R David Murray date: Sun Mar 09 19:41:25 2014 -0400 summary: whatsnew: SSLContext.load_default_certs (#19292). files: Doc/whatsnew/3.4.rst | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1318,6 +1318,15 @@ OpenSSL does not do any CRL verification by default. (Contributed by Christien Heimes in :issue:`8813`.) +New :class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_default_certs` +loads a set of dfault "certificate authority" (CA) certificates from default +locations, which vary according to the platform. It can be used to load both +TLS web server authentication certificates +(``purpose=``:data:`~ssl.Purpose.SERVER_AUTH`) for a client to use to verify a +server, and certificates for a server to use in verifying client certificates +(``purpose=``:data:`~ssl.Purpose.CLIENT_AUTH`). (Contributed by Christian +Heimes in :issue:`19292`.) + stat ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:08 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_allowZip64_def?= =?utf-8?q?aults_to_True_=28=2317201=29=2E?= Message-ID: <3fj2704vP8z7LkR@mail.python.org> http://hg.python.org/cpython/rev/86338438b70d changeset: 89544:86338438b70d user: R David Murray date: Sun Mar 09 19:57:19 2014 -0400 summary: whatsnew: allowZip64 defaults to True (#17201). files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1589,6 +1589,10 @@ example, this could be used to exclude test files from the archive. (Contributed by Christian Tismer in :issue:`19274`.) +The *allowZip64* parameter to :class:`~zipfile.ZipFile` and +:class:`~zipfile.PyZipfile` is now ``True`` by default. (Contributed by +William Mallard in :issue:`17201`.) + CPython Implementation Changes -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:09 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:09 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_ssl=2Ecreate?= =?utf-8?b?X2RlZmF1bHRfY29udGV4dCAoIzE5Njg5KS4=?= Message-ID: <3fj2716GPDz7LlY@mail.python.org> http://hg.python.org/cpython/rev/8b4b6609cd31 changeset: 89545:8b4b6609cd31 user: R David Murray date: Sun Mar 09 19:58:17 2014 -0400 summary: whatsnew: ssl.create_default_context (#19689). files: Doc/whatsnew/3.4.rst | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1327,6 +1327,13 @@ (``purpose=``:data:`~ssl.Purpose.CLIENT_AUTH`). (Contributed by Christian Heimes in :issue:`19292`.) +New function :func:`~ssl.create_default_context` provides a standard way to +obtain an :class:`~ssl.SSLContext` based on the settings Python uses by +default. These settings are more stringent than the defaults provided by the +:class:`~ssl.SSLContext` constructor, and may be adjusted (without notice) in +the future to maintain a balance between compatibility and security. +(Contributed by Christian Heimes in :issue:`19689`.) + stat ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:11 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_cp1125_codec_?= =?utf-8?b?KCMxOTY2OCku?= Message-ID: <3fj2730xsyz7LmK@mail.python.org> http://hg.python.org/cpython/rev/11bc9f669015 changeset: 89546:11bc9f669015 user: R David Murray date: Sun Mar 09 20:13:16 2014 -0400 summary: whatsnew: cp1125 codec (#19668). files: Doc/whatsnew/3.4.rst | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -367,8 +367,11 @@ Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in :issue:`12892`. -* New EBCDIC :ref:`codec ` ``cp273``. (Contributed by - Michael Bierenfeld and Andrew Kuchling in :issue:`1097797`.) +* New German EBCDIC :ref:`codec ` ``cp273``. (Contributed + by Michael Bierenfeld and Andrew Kuchling in :issue:`1097797`.) + +* New Ukrainian :ref:`codec ` ``cp1125``. (Contributed by + Serhiy Storchaka in :issue:`19668`.) * :class:`bytes`.join() and :class:`bytearray`.join() now accept arbitrary buffer objects as arguments. (Contributed by Antoine Pitrou in -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:12 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_=27U=27_mode_d?= =?utf-8?q?eprecation_=28=2315204=29=2E?= Message-ID: <3fj2742w8Tz7Lm2@mail.python.org> http://hg.python.org/cpython/rev/2d5544afb510 changeset: 89547:2d5544afb510 user: R David Murray date: Sun Mar 09 20:33:35 2014 -0400 summary: whatsnew: 'U' mode deprecation (#15204). files: Doc/whatsnew/3.4.rst | 14 +++++++++++++- 1 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1948,6 +1948,11 @@ * The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by ``EXT_SUFFIX``. +* The ``U`` mode accepted by various ``open`` functions is deprecated. + In Python3 it does not do anything useful, and should be replaced by + appropriate uses of :class:`io.TextIOWrapper` (if needed) and its *newline* + argument. + Deprecations in the C API ------------------------- @@ -2242,7 +2247,14 @@ * Since the *digestmod* argument to the :func:`hmac.new` function will in the future have no default, all calls to :func:`hmac.new` should be changed to - explicitly specify a *digestmod*. (:issue:`17276`). + explicitly specify a *digestmod* (:issue:`17276`). + +* Any calls to ``open`` functions that specify ``U`` should be modified. + ``U`` is ineffective in Python3 and will eventually raise an error if used. + Depending on the function, the equivalent of its old Python2 behavior can be + achieved using either a *newline* argument, or if necessary by wrapping the + stream in :mod:`~io.TextIOWrapper` to use its *newline* argument + (:issue:`15204`). Changes in the C API -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:13 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_porting_note_f?= =?utf-8?q?or_sysconfig_=27SO=27_key_deprecation=2E?= Message-ID: <3fj2754rjBz7LlM@mail.python.org> http://hg.python.org/cpython/rev/39b2679ec4e5 changeset: 89548:39b2679ec4e5 user: R David Murray date: Sun Mar 09 20:39:52 2014 -0400 summary: whatsnew: porting note for sysconfig 'SO' key deprecation. files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2249,6 +2249,11 @@ future have no default, all calls to :func:`hmac.new` should be changed to explicitly specify a *digestmod* (:issue:`17276`). +* Calling :func:`sysconfig.get_config_var` with the ``SO`` key, or looking + ``SO`` up in the results of a call to :func:`sysconfig.get_config_vars` + is deprecated. This key should be replaced by ``EXT_SUFFIX`` or + ``SHLIB_SUFFIX``, depending on the context (:issue:`19555`). + * Any calls to ``open`` functions that specify ``U`` should be modified. ``U`` is ineffective in Python3 and will eventually raise an error if used. Depending on the function, the equivalent of its old Python2 behavior can be -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:14 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_auidioop=2Ebyt?= =?utf-8?b?ZXN3YXAgKCMxNjY4NSku?= Message-ID: <3fj2766bJRz7Lm2@mail.python.org> http://hg.python.org/cpython/rev/c375697f062e changeset: 89549:c375697f062e user: R David Murray date: Sun Mar 09 20:42:49 2014 -0400 summary: whatsnew: auidioop.byteswap (#16685). files: Doc/library/audioop.rst | 2 +- Doc/whatsnew/3.4.rst | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst --- a/Doc/library/audioop.rst +++ b/Doc/library/audioop.rst @@ -80,7 +80,7 @@ "Byteswap" all samples in a fragment and returns the modified fragment. Converts big-endian samples to little-endian and vice versa. - .. versionadded: 3.4 + .. versionadded:: 3.4 .. function:: cross(fragment, width) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -570,8 +570,9 @@ :mod:`audioop` now supports 24-bit samples. (Contributed by Serhiy Storchaka in :issue:`12866`.) -Added the :func:`~audioop.byteswap` function to convert big-endian samples -to little-endian and vice versa (:issue:`19641`). +New :func:`~audioop.byteswap` function converts big-endian samples to +little-endian and vice versa (Contributed by Serhiy Storchaka in +:issue:`19641`). All :mod:`audioop` functions now accept any :term:`bytes-like object`. Strings are not accepted: they didn't work before, now they raise an error right away. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:16 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:16 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_give_Hugo_cred?= =?utf-8?q?it_for_his_preliminary_work_on_regex_patch=2E?= Message-ID: <3fj27810g6z7LkY@mail.python.org> http://hg.python.org/cpython/rev/8bef27cdcd0b changeset: 89550:8bef27cdcd0b user: R David Murray date: Sun Mar 09 20:46:32 2014 -0400 summary: whatsnew: give Hugo credit for his preliminary work on regex patch. files: Doc/whatsnew/3.4.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1178,8 +1178,9 @@ The repr of :ref:`regex objects ` now includes the pattern and the flags; the repr of :ref:`match objects ` now -includes the start, end, and the part of the string that matched. -(Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.) +includes the start, end, and the part of the string that matched. (Contributed +by Hugo Lopes Tavares and Serhiy Storchaka in :issue:`13592` and +:issue:`17087`.) resource -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:17 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:17 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_expand_re=2Efu?= =?utf-8?q?llmatch_entry_with_the_motivation_for_the_feature=2E?= Message-ID: <3fj2792K4vz7LmS@mail.python.org> http://hg.python.org/cpython/rev/382691325133 changeset: 89551:382691325133 user: R David Murray date: Sun Mar 09 20:52:28 2014 -0400 summary: whatsnew: expand re.fullmatch entry with the motivation for the feature. files: Doc/whatsnew/3.4.rst | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1172,9 +1172,12 @@ re -- -Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method, -which anchor the pattern at both ends of the string to match. -(Contributed by Matthew Barnett in :issue:`16203`.) +New :func:`~re.fullmatch` function and :meth:`.regex.fullmatch` method anchor +the pattern at both ends of the string to match. This provides a way to be +explicit about the goal of the match, which avoids a class of subtle bugs where +``$`` characters get lost during code changes or the addition of alternatives +to an existing regular expression. (Contributed by Matthew Barnett in +:issue:`16203`.) The repr of :ref:`regex objects ` now includes the pattern and the flags; the repr of :ref:`match objects ` now -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:18 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:18 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_oops=2C_create?= =?utf-8?q?=5Fdefault=5Fcontext_doesn=27t_return_what_the_stdlib_uses=2E?= Message-ID: <3fj27B3hCjz7LkB@mail.python.org> http://hg.python.org/cpython/rev/fcd1eb5c6026 changeset: 89552:fcd1eb5c6026 user: R David Murray date: Sun Mar 09 21:00:14 2014 -0400 summary: whatsnew: oops, create_default_context doesn't return what the stdlib uses. files: Doc/whatsnew/3.4.rst | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1336,11 +1336,12 @@ Heimes in :issue:`19292`.) New function :func:`~ssl.create_default_context` provides a standard way to -obtain an :class:`~ssl.SSLContext` based on the settings Python uses by -default. These settings are more stringent than the defaults provided by the -:class:`~ssl.SSLContext` constructor, and may be adjusted (without notice) in -the future to maintain a balance between compatibility and security. -(Contributed by Christian Heimes in :issue:`19689`.) +obtain an :class:`~ssl.SSLContext` whose settings are intended to be a +reasonable balance between compatibility and security. These settings are +more stringent than the defaults provided by the :class:`~ssl.SSLContext` +constructor, and may be adjusted in the future, without prior deprecation, if +best-practice security requirements change. (Contributed by Christian Heimes +in :issue:`19689`.) stat -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:19 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:19 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_dis=2Estack=5F?= =?utf-8?q?effect_=28=2319722=29=2E?= Message-ID: <3fj27C564qz7LlB@mail.python.org> http://hg.python.org/cpython/rev/4a801f8b7e2d changeset: 89553:4a801f8b7e2d user: R David Murray date: Sun Mar 09 21:07:37 2014 -0400 summary: whatsnew: dis.stack_effect (#19722). files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -696,6 +696,10 @@ (Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816` and Claudiu Popa in :issue:`17916`) +New function :func:`~dis.stack_effect` computes the effect on the Python stack +of a given opcode and argument, information that is not otherwise available. +(Contributed by Larry Hastings in :issue:`19722`.) + doctest ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:35:21 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 02:35:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_tarfile_cli_?= =?utf-8?b?KCMxMzQ3Nyku?= Message-ID: <3fj27F00g1z7Lls@mail.python.org> http://hg.python.org/cpython/rev/5b52db6fc7dc changeset: 89554:5b52db6fc7dc user: R David Murray date: Sun Mar 09 21:19:57 2014 -0400 summary: whatsnew: tarfile cli (#13477). files: Doc/whatsnew/3.4.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1424,6 +1424,14 @@ :issue:`5845`.) +tarfile +------- + +The :mod:`tarfile` module now supports a simple :ref:`tarfile-commandline` when +called as a script directly or via ``-m``. This can be used to create and +extract tarfile archives. (Contributed by Berker Peksag in :issue:`13477`.) + + textwrap -------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 02:59:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 10 Mar 2014 02:59:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_assertEqual_instead_of?= =?utf-8?q?_the_deprecated_assertEquals_=28closes_=2320877=29?= Message-ID: <3fj2gD0ZM6zS0d@mail.python.org> http://hg.python.org/cpython/rev/2bda1065e931 changeset: 89555:2bda1065e931 user: Benjamin Peterson date: Sun Mar 09 20:59:24 2014 -0500 summary: use assertEqual instead of the deprecated assertEquals (closes #20877) files: Lib/test/test_grammar.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -319,8 +319,8 @@ def f(self, *, __kw:1): pass class Ham(Spam): pass - self.assertEquals(Spam.f.__annotations__, {'_Spam__kw': 1}) - self.assertEquals(Ham.f.__annotations__, {'_Spam__kw': 1}) + self.assertEqual(Spam.f.__annotations__, {'_Spam__kw': 1}) + self.assertEqual(Ham.f.__annotations__, {'_Spam__kw': 1}) # Check for SF Bug #1697248 - mixing decorators and a return annotation def null(x): return x @null -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 08:04:25 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 08:04:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Updated_Windows_instructions_?= =?utf-8?q?as_per_MvL=2E?= Message-ID: <3fj9Qx0LbKz7LjQ@mail.python.org> http://hg.python.org/peps/rev/027b1e215e08 changeset: 5405:027b1e215e08 user: Georg Brandl date: Mon Mar 10 08:04:10 2014 +0100 summary: Updated Windows instructions as per MvL. files: pep-0101.txt | 24 ++++++++++++++++++++++++ 1 files changed, 24 insertions(+), 0 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -289,6 +289,30 @@ ___ The WE then generates Windows installer files for each Windows target architecture (for Python 3.3, this means x86 and AMD64). + - He has one checkout tree per target architecture, and builds the + pcbuild.sln project for the appropriate architecture. + + - PC\icons.mak must have been run with nmake. + + - The cmd.exe window in which this is run must have Cygwin/bin in its + path (at least for x86). + + - The cmd.exe window must have MS compiler tools for the target + architecture in its path (VS 2010 for Python 3.3). + + - The WE then edits Tools/msi/config.py (a file only present locally) + to update full_current_version and sets snapshot to false. Currently + for a release config.py looks like + + snapshot=0 + full_current_version="3.3.5rc2" + certname="Python Software Foundation + PCBUILD='PCbuild\\amd64' + + The last line is only present for the amd64 checkout. + + - Now he runs msi.py with ActivePython or Python with pywin32. + The WE checksums the files (*.msi, *.chm, *-pdb.zip), uploads them to dinsdale together with gpg signature files, and emails you the location and md5sums. -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Mar 10 09:06:31 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 09:06:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Add_a_shebang=2C_for_good_?= =?utf-8?q?measure=2E?= Message-ID: <3fjBpb4y25z7LjV@mail.python.org> http://hg.python.org/release/rev/a3391892eb9e changeset: 84:a3391892eb9e user: Georg Brandl date: Mon Mar 10 09:06:18 2014 +0100 summary: Add a shebang, for good measure. files: add-to-pydotorg.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py --- a/add-to-pydotorg.py +++ b/add-to-pydotorg.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python """ Script to add ReleaseFile objects for Python releases on the new pydotorg. To use (RELEASE is something like 3.3.5rc2): -- Repository URL: http://hg.python.org/release From python-checkins at python.org Mon Mar 10 09:19:48 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 10 Mar 2014 09:19:48 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?release=3A_Now_that_it_has_a_shiny_ne?= =?utf-8?q?w_shebang_line=2C_made_add-to-pydotorg=2Epy_executable=2E?= Message-ID: <3fjC5w2LX6z7LjV@mail.python.org> http://hg.python.org/release/rev/39a3e4c740ed changeset: 85:39a3e4c740ed user: Larry Hastings date: Mon Mar 10 01:19:53 2014 -0700 summary: Now that it has a shiny new shebang line, made add-to-pydotorg.py executable. files: add-to-pydotorg.py | 0 1 files changed, 0 insertions(+), 0 deletions(-) diff --git a/add-to-pydotorg.py b/add-to-pydotorg.py old mode 100644 new mode 100755 -- Repository URL: http://hg.python.org/release From solipsis at pitrou.net Mon Mar 10 10:04:50 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 10 Mar 2014 10:04:50 +0100 Subject: [Python-checkins] Daily reference leaks (2bda1065e931): sum=0 Message-ID: results for 2bda1065e931 on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogLDShFw', '-x'] From python-checkins at python.org Mon Mar 10 11:06:52 2014 From: python-checkins at python.org (victor.stinner) Date: Mon, 10 Mar 2014 11:06:52 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_tracemalloc=3A_filter=5Ftr?= =?utf-8?q?aces=28=29_raises_a_TypeError_if_filters_is_not_an_iterable?= Message-ID: <3fjFTS1y2pz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/d083b6ead156 changeset: 89556:d083b6ead156 user: Victor Stinner date: Mon Mar 10 11:05:07 2014 +0100 summary: tracemalloc: filter_traces() raises a TypeError if filters is not an iterable files: Lib/test/test_tracemalloc.py | 2 ++ Lib/tracemalloc.py | 5 ++++- 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_tracemalloc.py b/Lib/test/test_tracemalloc.py --- a/Lib/test/test_tracemalloc.py +++ b/Lib/test/test_tracemalloc.py @@ -346,6 +346,8 @@ self.assertIsNot(snapshot5.traces, snapshot.traces) self.assertEqual(snapshot5.traces, snapshot.traces) + self.assertRaises(TypeError, snapshot.filter_traces, filter1) + def test_snapshot_group_by_line(self): snapshot, snapshot2 = create_snapshots() tb_0 = traceback_lineno('', 0) diff --git a/Lib/tracemalloc.py b/Lib/tracemalloc.py --- a/Lib/tracemalloc.py +++ b/Lib/tracemalloc.py @@ -1,4 +1,4 @@ -from collections import Sequence +from collections import Sequence, Iterable from functools import total_ordering import fnmatch import linecache @@ -382,6 +382,9 @@ is a list of Filter instances. If filters is an empty list, return a new Snapshot instance with a copy of the traces. """ + if not isinstance(filters, Iterable): + raise TypeError("filters must be a list of filters, not %s" + % type(filters).__name__) if filters: include_filters = [] exclude_filters = [] -- Repository URL: http://hg.python.org/cpython From ncoghlan at gmail.com Mon Mar 10 14:04:19 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Mar 2014 23:04:19 +1000 Subject: [Python-checkins] cpython: whatsnew: venv with_pip, pyvenv --without-pip (#19552) In-Reply-To: <3fhyT2594sz7Lm1@mail.python.org> References: <3fhyT2594sz7Lm1@mail.python.org> Message-ID: On 10 Mar 2014 08:51, "r.david.murray" wrote: > > http://hg.python.org/cpython/rev/a140caad76bc > changeset: 89539:a140caad76bc > user: R David Murray > date: Sun Mar 09 18:09:51 2014 -0400 > summary: > whatsnew: venv with_pip, pyvenv --without-pip (#19552) > > files: > Doc/whatsnew/3.4.rst | 11 +++++++++++ > 1 files changed, 11 insertions(+), 0 deletions(-) > > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > --- a/Doc/whatsnew/3.4.rst > +++ b/Doc/whatsnew/3.4.rst > @@ -1507,6 +1507,12 @@ > :mod:`venv` now includes activation scripts for the ``csh`` and ``fish`` > shells (Contributed by Andrew Svetlov in :issue:`15417`.) > > +:class:`~venv.EnvBuilder` and the :func:`~venv.create` convenience function > +take a new keyword argument *with_pip*, which defaults to ``False``, that > +controls whether or not :class:`~venv.EnvBuilder` ensures that ``pip`` is > +installed in the virtual environment. (Contributed by Nick Coghlan in > +:issue:`19552` as part of the :pep:`453` implementation.) > + > > wave > ---- > @@ -1772,6 +1778,11 @@ > to use copies rather than symlinks even on systems where symlinks are the > default. (Contributed by Vinay Sajip in :issue:`18807`.) > > +* The :ref:`pyvenv ` command also accepts a ``--without-pip`` > + option to suppress the otherwise-automatic bootstrapping of pip into > + the virtual environment. (Contributed by Nick Coghlan in :issue:`19552` > + as part of the :pep:`453` implementation.) It just occurred to me that we should probably have a porting note about that CLI change. We decided that changing the default CLI behaviour, while preserving the default module API was the best available compromise, but anyone that is invoking the CLI programmatically and wants to skip bootstrapping pip will still need to change their command. Cheers, Nick. > + > * The encoding name is now optional in the value set for the > :envvar:`PYTHONIOENCODING` environment variable. This makes it possible to > set just the error handler, without changing the default encoding. > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Mar 10 14:09:53 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Mar 2014 23:09:53 +1000 Subject: [Python-checkins] cpython: whatsnew: 'U' mode deprecation (#15204). In-Reply-To: <3fj2742w8Tz7Lm2@mail.python.org> References: <3fj2742w8Tz7Lm2@mail.python.org> Message-ID: On 10 Mar 2014 11:36, "r.david.murray" wrote: > > http://hg.python.org/cpython/rev/2d5544afb510 > changeset: 89547:2d5544afb510 > user: R David Murray > date: Sun Mar 09 20:33:35 2014 -0400 > summary: > whatsnew: 'U' mode deprecation (#15204). > > files: > Doc/whatsnew/3.4.rst | 14 +++++++++++++- > 1 files changed, 13 insertions(+), 1 deletions(-) > > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > --- a/Doc/whatsnew/3.4.rst > +++ b/Doc/whatsnew/3.4.rst > @@ -1948,6 +1948,11 @@ > * The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by > ``EXT_SUFFIX``. > > +* The ``U`` mode accepted by various ``open`` functions is deprecated. > + In Python3 it does not do anything useful, and should be replaced by > + appropriate uses of :class:`io.TextIOWrapper` (if needed) and its *newline* > + argument. > + Huh, I missed that change. Don't we still need U when writing 2/3 compatible code at this point? If so, perhaps we may want to reconsider this particular deprecation in 3.4.1. Cheers, Nick. > > Deprecations in the C API > ------------------------- > @@ -2242,7 +2247,14 @@ > > * Since the *digestmod* argument to the :func:`hmac.new` function will in the > future have no default, all calls to :func:`hmac.new` should be changed to > - explicitly specify a *digestmod*. (:issue:`17276`). > + explicitly specify a *digestmod* (:issue:`17276`). > + > +* Any calls to ``open`` functions that specify ``U`` should be modified. > + ``U`` is ineffective in Python3 and will eventually raise an error if used. > + Depending on the function, the equivalent of its old Python2 behavior can be > + achieved using either a *newline* argument, or if necessary by wrapping the > + stream in :mod:`~io.TextIOWrapper` to use its *newline* argument > + (:issue:`15204`). > > > Changes in the C API > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Mon Mar 10 14:18:03 2014 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 10 Mar 2014 09:18:03 -0400 Subject: [Python-checkins] [Python-Dev] cpython: whatsnew: 'U' mode deprecation (#15204). In-Reply-To: References: <3fj2742w8Tz7Lm2@mail.python.org> Message-ID: <20140310131803.C68BC250CBE@webabinitio.net> On Mon, 10 Mar 2014 23:09:53 +1000, Nick Coghlan wrote: > On 10 Mar 2014 11:36, "r.david.murray" wrote: > > > > http://hg.python.org/cpython/rev/2d5544afb510 > > changeset: 89547:2d5544afb510 > > user: R David Murray > > date: Sun Mar 09 20:33:35 2014 -0400 > > summary: > > whatsnew: 'U' mode deprecation (#15204). > > > > files: > > Doc/whatsnew/3.4.rst | 14 +++++++++++++- > > 1 files changed, 13 insertions(+), 1 deletions(-) > > > > > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > > --- a/Doc/whatsnew/3.4.rst > > +++ b/Doc/whatsnew/3.4.rst > > @@ -1948,6 +1948,11 @@ > > * The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by > > ``EXT_SUFFIX``. > > > > +* The ``U`` mode accepted by various ``open`` functions is deprecated. > > + In Python3 it does not do anything useful, and should be replaced by > > + appropriate uses of :class:`io.TextIOWrapper` (if needed) and its > *newline* > > + argument. > > + > > Huh, I missed that change. Don't we still need U when writing 2/3 > compatible code at this point? If so, perhaps we may want to reconsider > this particular deprecation in 3.4.1. It's a deprecation, not a removal. I believe the idea is to not remove it before 4.0. --David From python-checkins at python.org Mon Mar 10 14:21:04 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 14:21:04 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_add_porting_no?= =?utf-8?q?te_for_pyvenv_--without-pip=2E?= Message-ID: <3fjKnX6N7yz7Llh@mail.python.org> http://hg.python.org/cpython/rev/74961f17cc43 changeset: 89557:74961f17cc43 user: R David Murray date: Mon Mar 10 09:20:37 2014 -0400 summary: whatsnew: add porting note for pyvenv --without-pip. files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2279,6 +2279,10 @@ stream in :mod:`~io.TextIOWrapper` to use its *newline* argument (:issue:`15204`). +* If you use :ref:`pyvenv ` in a script and desire that pip + *not* be installed, you must add ``--wihtout-pip`` to your command + invocation. + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Mon Mar 10 14:25:21 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 10 Mar 2014 14:25:21 +0100 Subject: [Python-checkins] cpython: whatsnew: 'U' mode deprecation (#15204). References: <3fj2742w8Tz7Lm2@mail.python.org> Message-ID: <20140310142521.69c3f957@fsol> On Mon, 10 Mar 2014 23:09:53 +1000 Nick Coghlan wrote: > On 10 Mar 2014 11:36, "r.david.murray" wrote: > > > > http://hg.python.org/cpython/rev/2d5544afb510 > > changeset: 89547:2d5544afb510 > > user: R David Murray > > date: Sun Mar 09 20:33:35 2014 -0400 > > summary: > > whatsnew: 'U' mode deprecation (#15204). > > > > files: > > Doc/whatsnew/3.4.rst | 14 +++++++++++++- > > 1 files changed, 13 insertions(+), 1 deletions(-) > > > > > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > > --- a/Doc/whatsnew/3.4.rst > > +++ b/Doc/whatsnew/3.4.rst > > @@ -1948,6 +1948,11 @@ > > * The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by > > ``EXT_SUFFIX``. > > > > +* The ``U`` mode accepted by various ``open`` functions is deprecated. > > + In Python3 it does not do anything useful, and should be replaced by > > + appropriate uses of :class:`io.TextIOWrapper` (if needed) and its > *newline* > > + argument. > > + > > Huh, I missed that change. Don't we still need U when writing 2/3 > compatible code at this point? Define "need" :-) I rarely if ever see "U" used in 2.x code, so I don't think the need is very pressing in general. Regards Antoine. From ncoghlan at gmail.com Mon Mar 10 14:36:19 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Mar 2014 23:36:19 +1000 Subject: [Python-checkins] cpython: whatsnew: 'U' mode deprecation (#15204). In-Reply-To: <20140310142521.69c3f957@fsol> References: <3fj2742w8Tz7Lm2@mail.python.org> <20140310142521.69c3f957@fsol> Message-ID: On 10 Mar 2014 23:26, "Antoine Pitrou" wrote: > > On Mon, 10 Mar 2014 23:09:53 +1000 > Nick Coghlan wrote: > > On 10 Mar 2014 11:36, "r.david.murray" wrote: > > > > > > +* The ``U`` mode accepted by various ``open`` functions is deprecated. > > > + In Python3 it does not do anything useful, and should be replaced by > > > + appropriate uses of :class:`io.TextIOWrapper` (if needed) and its > > *newline* > > > + argument. > > > + > > > > Huh, I missed that change. Don't we still need U when writing 2/3 > > compatible code at this point? > > Define "need" :-) I rarely if ever see "U" used in 2.x code, so I don't > think the need is very pressing in general. True, normal text mode is much more common. I'm OK with waiting to see if anyone actually complains about the deprecation :) Cheers, Nick. > > Regards > > Antoine. > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Mon Mar 10 19:19:50 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 19:19:50 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Closes_=2315605=3A_update?= =?utf-8?q?_and_refurbish_doc_building_section_of_the_devguide=2E?= Message-ID: <3fjSQG33v7z7LlF@mail.python.org> http://hg.python.org/devguide/rev/f32569ddc2db changeset: 672:f32569ddc2db user: Georg Brandl date: Mon Mar 10 19:19:07 2014 +0100 summary: Closes #15605: update and refurbish doc building section of the devguide. Adapts suggestions by Daniel and Terry, and handles the new situation in 3.4+. files: documenting.rst | 72 ++++++++++++++---------------------- 1 files changed, 29 insertions(+), 43 deletions(-) diff --git a/documenting.rst b/documenting.rst --- a/documenting.rst +++ b/documenting.rst @@ -1442,28 +1442,37 @@ Building the documentation ========================== -You need to have Python 2.4 or higher installed. The toolset used to build -the docs is written in Python and is called Sphinx_. Sphinx is maintained -separately and is not included in this tree. Also needed are docutils_, -supplying the base markup that Sphinx uses; Jinja_, a templating engine; and -optionally Pygments_, a code highlighter. +The toolset used to build the docs is written in Python and is called Sphinx_. +Sphinx is maintained separately and is not included in this tree. Also needed +are docutils_, supplying the base markup that Sphinx uses; Jinja_, a templating +engine; and optionally Pygments_, a code highlighter. To build the documentation, follow the instructions from one of the sections below. You can view the documentation after building the HTML by pointing a browser at the file :file:`Doc/build/html/index.html`. +In the Python 2.7 and 3.3 branches, the Sphinx toolchain will be checked out +using Subversion from ``svn.python.org`` by the Makefile. This toolchain will +need an installed Python 2 to run. -Using make ----------- +In the Python 3.4 and later branches, you are expected to have installed a +recent version of Sphinx on your system, so that the Makefile can find the +``sphinx-build`` command. -On Unix, if you have Subversion installed, run the following from the root of -your :ref:`repository clone `:: + +Using make / make.bat +--------------------- + +On Unix, run the following from the root of your :ref:`repository clone +`:: cd Doc make html -or alternatively ``make -C Doc html``. This checks out the needed toolset -in the :file:`Doc/tools/` directory and builds the output as HTML. +or alternatively ``make -C Doc html``. This builds the output as HTML. + +For Windows users there is a :file:`make.bat` batchfile that tries to work like +``make`` does. Available :command:`make` targets are: @@ -1494,47 +1503,24 @@ * "pydoc-topics", which builds a Python module containing a dictionary with plain text documentation for the labels defined in - :file:`tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic and - keyword help. + :file:`Doc/tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic + and keyword help. -A "make update" updates the Subversion checkouts in :file:`tools/`. (These -Subversion checkouts are ignored by the main Mercurial repository.) + * "suspicious", which checks the parsed markup for text that looks like + malformed and thus unconverted reST. Without make ------------ -You'll need to install the Sphinx package, either by checking it out via :: +Install the Sphinx package and its dependencies from PyPI. - svn co http://svn.python.org/projects/external/Sphinx-1.0.7/sphinx tools/sphinx +Then, from the ``Docs`` directory, run :: -or by installing it from PyPI. + sphinx-build -b . build/ -Then, you need to install Docutils, either by checking it out via :: - - svn co http://svn.python.org/projects/external/docutils-0.6/docutils tools/docutils - -or by installing it from http://docutils.sf.net/. - -You also need Jinja2, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Jinja-2.3.1/jinja2 tools/jinja2 - -or by installing it from PyPI. - -You can optionally also install Pygments, either as a checkout via :: - - svn co http://svn.python.org/projects/external/Pygments-1.3.1/pygments tools/pygments - -or from PyPI at http://pypi.python.org/pypi/Pygments. - - -Then, make an output directory, e.g. under `build/`, and run :: - - python tools/sphinx-build.py -b . build/ - -where `` is one of html, text, latex, or htmlhelp (for explanations see -the make targets above). +where ```` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). .. _docutils: http://docutils.sourceforge.net/ .. _Jinja: http://jinja.pocoo.org/ -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Mon Mar 10 19:29:50 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 19:29:50 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMDA5?= =?utf-8?q?0=3A_update_Doc/README_and_the_error_message_in_sphinx-build=2E?= =?utf-8?q?py_to?= Message-ID: <3fjSdp5QK9z7Lkh@mail.python.org> http://hg.python.org/cpython/rev/0ae254c11921 changeset: 89558:0ae254c11921 branch: 2.7 parent: 89526:16c5d7c289c6 user: Georg Brandl date: Mon Mar 10 19:26:57 2014 +0100 summary: Closes #20090: update Doc/README and the error message in sphinx-build.py to make it clear that any Python 3.x is not usable with the checked out Sphinx. files: Doc/README.txt | 44 +++++++------------------- Doc/tools/sphinx-build.py | 8 ++-- 2 files changed, 17 insertions(+), 35 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -7,14 +7,13 @@ Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -documentation. There's also a chapter intended to point out differences to -those familiar with the previous docs written in LaTeX. +documentation. Building the docs ================= -You need to have Python 2.4 or higher installed; the toolset used to build the +You need to have Python 2 installed; the toolset used to build the docs is written in Python. It is called *Sphinx*, it is not included in this tree, but maintained separately. Also needed are the docutils, supplying the base markup that Sphinx uses, Jinja, a templating engine, and optionally @@ -33,6 +32,9 @@ HTML output files. To view the generated HTML, point your favorite browser at the top-level index `build/html/index.html` after running "make". +On Windows, we try to emulate the Makefile as closely as possible with a +``make.bat`` file. + Available make targets are: * "html", which builds standalone HTML files for offline viewing. @@ -65,43 +67,23 @@ `tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic and keyword help. + * "suspicious", which checks the parsed markup for text that looks like + malformed and thus unconverted reST. + A "make update" updates the Subversion checkouts in `tools/`. Without make ------------ -You'll need to install the Sphinx package, either by checking it out via :: +Install the Sphinx package and its dependencies from PyPI. - svn co http://svn.python.org/projects/external/Sphinx-0.6.7/sphinx tools/sphinx +Then, from the ``Docs`` directory, run :: -or by installing it from PyPI. + sphinx-build -b . build/ -Then, you need to install Docutils, either by checking it out via :: - - svn co http://svn.python.org/projects/external/docutils-0.6/docutils tools/docutils - -or by installing it from http://docutils.sf.net/. - -You also need Jinja2, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Jinja-2.3.1/jinja2 tools/jinja2 - -or by installing it from PyPI. - -You can optionally also install Pygments, either as a checkout via :: - - svn co http://svn.python.org/projects/external/Pygments-1.3.1/pygments tools/pygments - -or from PyPI at http://pypi.python.org/pypi/Pygments. - - -Then, make an output directory, e.g. under `build/`, and run :: - - python tools/sphinx-build.py -b . build/ - -where `` is one of html, text, latex, or htmlhelp (for explanations see -the make targets above). +where ```` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). Contributing diff --git a/Doc/tools/sphinx-build.py b/Doc/tools/sphinx-build.py --- a/Doc/tools/sphinx-build.py +++ b/Doc/tools/sphinx-build.py @@ -15,13 +15,13 @@ if __name__ == '__main__': - if sys.version_info[:3] < (2, 4, 0): - print >>sys.stderr, """\ -Error: Sphinx needs to be executed with Python 2.4 or newer + if sys.version_info[:3] < (2, 4, 0) or sys.version_info[:3] > (3, 0, 0): + sys.stderr.write("""\ +Error: Sphinx needs to be executed with Python 2.4 or newer (not 3.x though). (If you run this from the Makefile, you can set the PYTHON variable to the path of an alternative interpreter executable, e.g., ``make html PYTHON=python2.5``). -""" +""") sys.exit(1) from sphinx import main -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 19:32:09 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 19:32:09 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_unused_sphinx-build?= =?utf-8?q?=2Epy_entry_script=2E?= Message-ID: <3fjShT0hBzz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/9765cb577bcb changeset: 89559:9765cb577bcb parent: 89557:74961f17cc43 user: Georg Brandl date: Mon Mar 10 19:10:29 2014 +0100 summary: Remove unused sphinx-build.py entry script. files: Doc/tools/sphinx-build.py | 28 --------------------------- 1 files changed, 0 insertions(+), 28 deletions(-) diff --git a/Doc/tools/sphinx-build.py b/Doc/tools/sphinx-build.py deleted file mode 100644 --- a/Doc/tools/sphinx-build.py +++ /dev/null @@ -1,28 +0,0 @@ -# -*- coding: utf-8 -*- -""" - Sphinx - Python documentation toolchain - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - :copyright: 2007-2010 by Georg Brandl. - :license: Python license. -""" - -import sys -import warnings - -# Get rid of UserWarnings reported by pkg_resources. -warnings.filterwarnings('ignore', category=UserWarning, module='jinja2') - -if __name__ == '__main__': - - if sys.version_info[:3] < (2, 4, 0) or sys.version_info[:3] > (3, 0, 0): - sys.stderr.write("""\ -Error: Sphinx needs to be executed with Python 2.4 or newer (not 3.0 though). -(If you run this from the Makefile, you can set the PYTHON variable -to the path of an alternative interpreter executable, e.g., -``make html PYTHON=python2.5``). -""") - sys.exit(1) - - from sphinx import main - sys.exit(main(sys.argv)) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 19:32:10 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 19:32:10 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMyMDA5?= =?utf-8?q?0=3A_update_Doc/README_and_the_error_message_in_sphinx-build=2E?= =?utf-8?q?py_to?= Message-ID: <3fjShV52ycz7Lk4@mail.python.org> http://hg.python.org/cpython/rev/d975f50de5aa changeset: 89560:d975f50de5aa branch: 3.3 parent: 89540:f9cb5a44879c user: Georg Brandl date: Mon Mar 10 19:26:57 2014 +0100 summary: Closes #20090: update Doc/README and the error message in sphinx-build.py to make it clear that any Python 3.x is not usable with the checked out Sphinx. files: Doc/README.txt | 44 +++++++------------------- Doc/tools/sphinx-build.py | 2 +- 2 files changed, 14 insertions(+), 32 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -7,14 +7,13 @@ Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -documentation. There's also a chapter intended to point out differences to -those familiar with the previous docs written in LaTeX. +documentation. Building the docs ================= -You need to have Python 2.4 or higher installed; the toolset used to build the +You need to have Python 2 installed; the toolset used to build the docs is written in Python. It is called *Sphinx*, it is not included in this tree, but maintained separately. Also needed are the docutils, supplying the base markup that Sphinx uses, Jinja, a templating engine, and optionally @@ -33,6 +32,9 @@ HTML output files. To view the generated HTML, point your favorite browser at the top-level index `build/html/index.html` after running "make". +On Windows, we try to emulate the Makefile as closely as possible with a +``make.bat`` file. + To use a Python interpreter that's not called ``python``, use the standard way to set Makefile variables, using e.g. :: @@ -73,43 +75,23 @@ `tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic and keyword help. + * "suspicious", which checks the parsed markup for text that looks like + malformed and thus unconverted reST. + A "make update" updates the Subversion checkouts in `tools/`. Without make ------------ -You'll need to install the Sphinx package, either by checking it out via :: +Install the Sphinx package and its dependencies from PyPI. - svn co http://svn.python.org/projects/external/Sphinx-1.0.7/sphinx tools/sphinx +Then, from the ``Docs`` directory, run :: -or by installing it from PyPI. + sphinx-build -b . build/ -Then, you need to install Docutils, either by checking it out via :: - - svn co http://svn.python.org/projects/external/docutils-0.6/docutils tools/docutils - -or by installing it from http://docutils.sf.net/. - -You also need Jinja2, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Jinja-2.3.1/jinja2 tools/jinja2 - -or by installing it from PyPI. - -You can optionally also install Pygments, either as a checkout via :: - - svn co http://svn.python.org/projects/external/Pygments-1.3.1/pygments tools/pygments - -or from PyPI at http://pypi.python.org/pypi/Pygments. - - -Then, make an output directory, e.g. under `build/`, and run :: - - python tools/sphinx-build.py -b . build/ - -where `` is one of html, text, latex, or htmlhelp (for explanations see -the make targets above). +where ```` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). Contributing diff --git a/Doc/tools/sphinx-build.py b/Doc/tools/sphinx-build.py --- a/Doc/tools/sphinx-build.py +++ b/Doc/tools/sphinx-build.py @@ -17,7 +17,7 @@ if sys.version_info[:3] < (2, 4, 0) or sys.version_info[:3] > (3, 0, 0): sys.stderr.write("""\ -Error: Sphinx needs to be executed with Python 2.4 or newer (not 3.0 though). +Error: Sphinx needs to be executed with Python 2.4 or newer (not 3.x though). (If you run this from the Makefile, you can set the PYTHON variable to the path of an alternative interpreter executable, e.g., ``make html PYTHON=python2.5``). -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 19:32:13 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 10 Mar 2014 19:32:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2320090=3A_update_Doc/README=2Etxt_for_the_chang?= =?utf-8?q?es_in_the_3=2E4_branch=2E?= Message-ID: <3fjShY1TNbz7LlM@mail.python.org> http://hg.python.org/cpython/rev/43820a4fcaba changeset: 89561:43820a4fcaba parent: 89559:9765cb577bcb parent: 89560:d975f50de5aa user: Georg Brandl date: Mon Mar 10 19:31:52 2014 +0100 summary: Closes #20090: update Doc/README.txt for the changes in the 3.4 branch. files: Doc/README.txt | 62 +++++++++++-------------------------- 1 files changed, 19 insertions(+), 43 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -3,36 +3,34 @@ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, prebuilt versions are -available at http://docs.python.org/download/. +available at . Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -developers guide (http://docs.python.org/devguide/documenting.html). -There's also a chapter intended to point out differences to -those familiar with the previous docs written in LaTeX. +developers guide . Building the docs ================= -You need to have Python 2.4 or higher installed; the toolset used to build the -docs is written in Python. It is called *Sphinx*, it is not included in this -tree, but maintained separately. Also needed are the docutils, supplying the -base markup that Sphinx uses, Jinja, a templating engine, and optionally -Pygments, a code highlighter. +You need to have Sphinx installed; it is the toolset +used to build the docs. It is not included in this tree, but maintained +separately and available from PyPI . Using make ---------- -Luckily, a Makefile has been prepared so that on Unix, provided you have -installed Python and Subversion, you can just run :: +A Makefile has been prepared so that on Unix, provided you have installed +Sphinx, you can just run :: make html -to check out the necessary toolset in the `tools/` subdirectory and build the -HTML output files. To view the generated HTML, point your favorite browser at -the top-level index `build/html/index.html` after running "make". +to build the HTML output files. To view the generated HTML, point your favorite +browser at the top-level index `build/html/index.html` after running "make". + +On Windows, we try to emulate the Makefile as closely as possible with a +``make.bat`` file. To use a Python interpreter that's not called ``python``, use the standard way to set Makefile variables, using e.g. :: @@ -74,43 +72,21 @@ `tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic and keyword help. -A "make update" updates the Subversion checkouts in `tools/`. + * "suspicious", which checks the parsed markup for text that looks like + malformed and thus unconverted reST. Without make ------------ -You'll need to install the Sphinx package, either by checking it out via :: +Install the Sphinx package and its dependencies from PyPI. - svn co http://svn.python.org/projects/external/Sphinx-1.0.7/sphinx tools/sphinx +Then, from the ``Docs`` directory, run :: -or by installing it from PyPI. + sphinx-build -b . build/ -Then, you need to install Docutils, either by checking it out via :: - - svn co http://svn.python.org/projects/external/docutils-0.6/docutils tools/docutils - -or by installing it from http://docutils.sf.net/. - -You also need Jinja2, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Jinja-2.3.1/jinja2 tools/jinja2 - -or by installing it from PyPI. - -You can optionally also install Pygments, either as a checkout via :: - - svn co http://svn.python.org/projects/external/Pygments-1.3.1/pygments tools/pygments - -or from PyPI at http://pypi.python.org/pypi/Pygments. - - -Then, make an output directory, e.g. under `build/`, and run :: - - python tools/sphinx-build.py -b . build/ - -where `` is one of html, text, latex, or htmlhelp (for explanations see -the make targets above). +where ```` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). Contributing -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:28 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:28 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_ChainMap=2Enew?= =?utf-8?q?=5Fchild_*m*_argument=2E?= Message-ID: <3fjYYX60qLz7Lk4@mail.python.org> http://hg.python.org/cpython/rev/b11738a62669 changeset: 89562:b11738a62669 user: R David Murray date: Mon Mar 10 09:49:19 2014 -0400 summary: whatsnew: ChainMap.new_child *m* argument. And NEWS entry. files: Doc/whatsnew/3.4.rst | 9 +++++++++ Misc/NEWS | 3 +++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -596,6 +596,15 @@ project, Serhiy Storchaka, and Antoine Pitrou in :issue:`17618`.) +collections +----------- + +The :meth:`.ChainMap.new_child` method now accepts an *m* argument specifying +the child map to add to the chain. This allows an existing mapping and/or a +custom mapping type to be used for the child. (Contributed by Vinay Sajip in +:issue:`16613`.) + + colorsys -------- diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2376,6 +2376,9 @@ - Issue #14850: Now a charmap decoder treats U+FFFE as "undefined mapping" in any mapping, not only in a string. +- Issue #16613: Add *m* argument to ``collections.Chainmap.new_child`` to + allow the new child map to be specified explicitly. + - Issue #16730: importlib.machinery.FileFinder now no longers raises an exception when trying to populate its cache and it finds out the directory is unreadable or has turned into a file. Reported and diagnosed by -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:30 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_Message=2Eset?= =?utf-8?q?=5Fparam_*replace*=2C_Policy=2Econtent=5Fmanager=2E?= Message-ID: <3fjYYZ1c5zz7Lk5@mail.python.org> http://hg.python.org/cpython/rev/4a3f74b3758c changeset: 89563:4a3f74b3758c user: R David Murray date: Mon Mar 10 10:08:05 2014 -0400 summary: whatsnew: Message.set_param *replace*, Policy.content_manager. Also cleaned up the email section and fixed some markup bugs in the email docs. files: Doc/library/email-examples.rst | 6 +- Doc/library/email.contentmanager.rst | 19 ++++-- Doc/library/email.message.rst | 4 +- Doc/library/email.policy.rst | 2 +- Doc/whatsnew/3.4.rst | 43 ++++++++------- 5 files changed, 42 insertions(+), 32 deletions(-) diff --git a/Doc/library/email-examples.rst b/Doc/library/email-examples.rst --- a/Doc/library/email-examples.rst +++ b/Doc/library/email-examples.rst @@ -40,8 +40,10 @@ .. literalinclude:: ../includes/email-alternative.py -Examples using the Provision API -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. _email-contentmanager-api-examples: + +Examples using the Provisional API +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is a reworking of the last example using the provisional API. To make things a bit more interesting, we include a related image in the html part, and we save a copy of what we are going to send to disk, as well as sending it. diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -54,6 +54,7 @@ documented in this module because of the provisional nature of the code, the implementation lives in the :mod:`email.message` module. +.. currentmodule:: email.message .. class:: EmailMessage(policy=default) @@ -235,6 +236,16 @@ all other headers intact and in their original order. +.. class:: MIMEPart(policy=default) + + This class represents a subpart of a MIME message. It is identical to + :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are + added when :meth:`~EmailMessage.set_content` is called, since sub-parts do + not need their own :mailheader:`MIME-Version` headers. + + +.. currentmodule:: email.contentmanager + .. class:: ContentManager() Base class for content managers. Provides the standard registry mechanisms @@ -305,14 +316,6 @@ values of *typekey*, see :meth:`set_content`. -.. class:: MIMEPart(policy=default) - - This class represents a subpart of a MIME message. It is identical to - :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are - added when :meth:`~EmailMessage.set_content` is called, since sub-parts do - not need their own :mailheader:`MIME-Version` headers. - - Content Manager Instances ~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -466,7 +466,7 @@ to ``False``. - .. method:: set_param(param, value, header='Content-Type', requote=True, + .. method:: set_param(param, value, header='Content-Type', requote=True, \ charset=None, language='', replace=False) Set a parameter in the :mailheader:`Content-Type` header. If the @@ -488,7 +488,7 @@ end of the list of headers. If *replace* is ``True``, the header will be updated in place. - .. versionchanged: 3.4 ``replace`` keyword was added. + .. versionchanged:: 3.4 ``replace`` keyword was added. .. method:: del_param(param, header='content-type', requote=True) diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -419,7 +419,7 @@ additional arguments. By default ``content_manager`` is set to :data:`~email.contentmanager.raw_data_manager`. - .. versionadded 3.4 + .. versionadded:: 3.4 The class provides the following concrete implementations of the abstract diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -731,7 +731,8 @@ override the default policy of the message when generating a string representation of it. This means that ``as_string`` can now be used in more circumstances, instead of having to create and use a :mod:`~email.generator` in -order to pass formatting parameters to its ``flatten`` method. +order to pass formatting parameters to its ``flatten`` method. (Contributed by +R. David Murray in :issue:`18600`.) New method :meth:`~email.message.Message.as_bytes` added to produce a bytes representation of the message in a fashion similar to how ``as_string`` @@ -739,26 +740,30 @@ argument, but does accept the *unixfrom* and *policy* arguments. The :class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive -result: a bytes object containing the fully formatted message. - -(Contributed by R. David Murray in :issue:`18600`.) +result: a bytes object containing the fully formatted message. (Contributed +by R. David Murray in :issue:`18600`.) + +The :meth:`.Message.set_param` message now accepts a *replace* keyword argument. +When specified, the associated header will be updated without changing +its location in the list of headers. For backward compatibility, the default +is ``False``. (Contributed by R. David Murray in :issue:`18891`.) + .. _whatsnew_email_contentmanager: -A pair of new subclasses of :class:`~email.message.Message` have been added, -along with a new sub-module, :mod:`~email.contentmanager`. All documentation -is currently in the new module, which is being added as part of the new -:term:`provisional ` email API. These classes provide a -number of new methods that make extracting content from and inserting content -into email messages much easier. See the :mod:`~email.contentmanager` -documentation for details. - -These API additions complete the bulk of the work that was planned as part of -the email6 project. The currently provisional API is scheduled to become final -in Python 3.5 (possibly with a few minor additions in the area of error -handling). - -(Contributed by R. David Murray in :issue:`18891`.) +A pair of new subclasses of :class:`~email.message.Message` have been added +(:class:`.EmailMessage` and :class:`.MIMEPart`), along with a new sub-module, +:mod:`~email.contentmanager` and a new :mod:`~email.policy` attribute +:attr:`~email.policy.EmailPolicy.content_manager`. All documentation is +currently in the new module, which is being added as part of email's new +:term:`provisional API`. These classes provide a number of new methods that +make extracting content from and inserting content into email messages much +easier. For details, see the :mod:`~email.contentmanager` documentation and +the :ref:`email-contentmanager-api-examples`. These API additions complete the +bulk of the work that was planned as part of the email6 project. The currently +provisional API is scheduled to become final in Python 3.5 (possibly with a few +minor additions in the area of error handling). (Contributed by R. David +Murray in :issue:`18891`.) filecmp @@ -1084,7 +1089,7 @@ --- :mod:`pdb` has been enhanced to handle generators, :keyword:`yield`, and -:keyword:`yield from` in a more useful fashion. This is especially helpful when +``yield from`` in a more useful fashion. This is especially helpful when debugging :mod:`asyncio` based programs. (Contributed by Andrew Svetlov and Xavier de Gaye in :issue:`16596`.) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:31 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_filecmp=2EDEFA?= =?utf-8?q?ULT=5FIGNORES_=28=2315442=29?= Message-ID: <3fjYYb3FkLz7LkY@mail.python.org> http://hg.python.org/cpython/rev/2b1d5caf63ca changeset: 89564:2b1d5caf63ca user: R David Murray date: Mon Mar 10 10:47:56 2014 -0400 summary: whatsnew: filecmp.DEFAULT_IGNORES (#15442) files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -776,6 +776,11 @@ than the resolution of a particular filesystem's file modification time field. (Contributed by Mark Levitt in :issue:`18149`.) +New module attribute :data:`~filecmp.DEFAULT_IGNORES` provides the list of +directories that are used as the default value for the *ignore* parameter of +the :func:`~filecmp.dircmp` function. (Contributed by Eli Bendersky in +:issue:`15442`.) + functools --------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:32 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_total=5Forderi?= =?utf-8?q?ng_supports_NotImplemented_=28=2310042=29?= Message-ID: <3fjYYc6Wfmz7LkK@mail.python.org> http://hg.python.org/cpython/rev/1cc413874631 changeset: 89565:1cc413874631 user: R David Murray date: Mon Mar 10 11:02:45 2014 -0400 summary: whatsnew: total_ordering supports NotImplemented (#10042) files: Doc/whatsnew/3.4.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -811,6 +811,10 @@ available for other implementations to use. (Contributed by Brian Thorne in :issue:`12428`.) +:func:`~functools.total_ordering` now supports a return value of +:const:`NotImplemented` from the underlying comparison function. (Contributed +by Katie Miller in :issue:`10042`.) + gc -- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:34 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_logging=2Efile?= =?utf-8?q?Config_accepts_ConfigParser_instances=2E_=28=2316110=29?= Message-ID: <3fjYYf1JrWz7Lk7@mail.python.org> http://hg.python.org/cpython/rev/113341605247 changeset: 89566:113341605247 user: R David Murray date: Mon Mar 10 11:51:19 2014 -0400 summary: whatsnew: logging.fileConfig accepts ConfigParser instances. (#16110) And missing NEWS entry. files: Doc/whatsnew/3.4.rst | 8 ++++++++ Misc/NEWS | 3 +++ 2 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1006,6 +1006,14 @@ setting *port* to ``None``). (Contributed by Vinay Sajip in commit ce46195b56a9.) +:func:`~logging.config.fileConfig` now accepts a +:class:`configparser.RawConfigParser` subclass instance for the *fname* +parameter. This facilitates using a configuration file when logging +configuration is just a part of the overall application configuration, or where +the application modifies the configuration before passing it to +:func:`~logging.config.fileConfig`. (Contributed by Vinay Sajip in +:issue:`16110`.) + .. _whatsnew-marshal-3: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3617,6 +3617,9 @@ - Issue #16169: Fix ctypes.WinError()'s confusion between errno and winerror. +- Issue #16110: logging.fileConfig now accepts a pre-initialised ConfigParser + instance. + - Issue #1492704: shutil.copyfile() raises a distinct SameFileError now if source and destination are the same file. Patch by Atsuo Ishimoto. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:35 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_logging=2Econf?= =?utf-8?q?ig=2Elisten_*verify*_=28=2315452=29=2E?= Message-ID: <3fjYYg3FGSz7LkD@mail.python.org> http://hg.python.org/cpython/rev/fe1804387687 changeset: 89567:fe1804387687 user: R David Murray date: Mon Mar 10 12:03:01 2014 -0400 summary: whatsnew: logging.config.listen *verify* (#15452). files: Doc/whatsnew/3.4.rst | 5 +++++ Misc/NEWS | 4 ++++ 2 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1014,6 +1014,11 @@ :func:`~logging.config.fileConfig`. (Contributed by Vinay Sajip in :issue:`16110`.) +Logging configuration data received from a socket via the +:func:`logging.config.listen` function can now be validated before being +processed by supplying a verification function as the argument to the new +*verify* keyword argument. (Contributed by Vinay Sajip in :issue:`15452`.) + .. _whatsnew-marshal-3: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3638,6 +3638,10 @@ - Issue #9650: List commonly used format codes in time.strftime and time.strptime docsttings. Original patch by Mike Hoy. +- Issue #15452: logging configuration socket listener now has a verify option + that allows an application to apply a verification function to the + received configuration data before it is acted upon. + - Issue #16034: Fix performance regressions in the new `bz2.BZ2File` implementation. Initial patch by Serhiy Storchaka. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:36 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_remove_redunda?= =?utf-8?q?nt_entry=2E?= Message-ID: <3fjYYh5Q7Zz7LkM@mail.python.org> http://hg.python.org/cpython/rev/69e589e5eb1a changeset: 89568:69e589e5eb1a user: R David Murray date: Mon Mar 10 13:21:09 2014 -0400 summary: whatsnew: remove redundant entry. files: Doc/whatsnew/3.4.rst | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1781,12 +1781,6 @@ object allocator have been silenced. (Contributed by Dhiru Kholia in :issue:`18596`.) -* New function :c:func:`Py_SetStandardStreamEncoding` allows an application - that is embedding Python to do the equivalent of setting - :envvar:`PYTHONIOENCODING`. Its arguments override the equivalent - values from :envvar:`PYTHONIOENCODING` if it exists. (Contributed - by Bastien Montagne and Nick Coghlan in :issue:`16129`.) - * The Windows build now uses `Address Space Layout Randomization `_ and `Data Execution Prevention `_. (Contributed by -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:38 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:38 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_multiprocessin?= =?utf-8?q?g_start_methods_and_context_=28=238713_and_=2318999=29?= Message-ID: <3fjYYk44f4z7LkB@mail.python.org> http://hg.python.org/cpython/rev/b941a320601a changeset: 89569:b941a320601a user: R David Murray date: Mon Mar 10 14:27:21 2014 -0400 summary: whatsnew: multiprocessing start methods and context (#8713 and #18999) Also tweaked the docs a bit to use our standard style for versionadded/changed. (I'm guessing there are other places in the multiprocessing docs where similar tweaks should be made.) files: Doc/library/multiprocessing.rst | 35 ++++++++++++-------- Doc/whatsnew/3.4.rst | 24 +++++++++++--- Misc/NEWS | 7 ++++ 3 files changed, 47 insertions(+), 19 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -136,9 +136,11 @@ Available on Unix platforms which support passing file descriptors over Unix pipes. -Before Python 3.4 *fork* was the only option available on Unix. Also, -prior to Python 3.4, child processes would inherit all the parents -inheritable handles on Windows. +.. versionchanged:: 3.4 + *span* added on all unix platforms, and *forkserver* added for + some unix platforms. + Child processes no longer inherit all of the parents inheritable + handles on Windows. On Unix using the *spawn* or *forkserver* start methods will also start a *semaphore tracker* process which tracks the unlinked named @@ -1853,25 +1855,30 @@ callbacks and has a parallel map implementation. *processes* is the number of worker processes to use. If *processes* is - ``None`` then the number returned by :func:`os.cpu_count` is used. If - *initializer* is not ``None`` then each worker process will call + ``None`` then the number returned by :func:`os.cpu_count` is used. + + If *initializer* is not ``None`` then each worker process will call ``initializer(*initargs)`` when it starts. + *maxtasksperchild* is the number of tasks a worker process can complete + before it will exit and be replaced with a fresh worker process, to enable + unused resources to be freed. The default *maxtasksperchild* is None, which + means worker processes will live as long as the pool. + + *context* can be used to specify the context used for starting + the worker processes. Usually a pool is created using the + function :func:`multiprocessing.Pool` or the :meth:`Pool` method + of a context object. In both cases *context* is set + appropriately. + Note that the methods of the pool object should only be called by the process which created the pool. .. versionadded:: 3.2 - *maxtasksperchild* is the number of tasks a worker process can complete - before it will exit and be replaced with a fresh worker process, to enable - unused resources to be freed. The default *maxtasksperchild* is None, which - means worker processes will live as long as the pool. + *maxtasksperchild* .. versionadded:: 3.4 - *context* can be used to specify the context used for starting - the worker processes. Usually a pool is created using the - function :func:`multiprocessing.Pool` or the :meth:`Pool` method - of a context object. In both cases *context* is set - appropriately. + *context* .. note:: diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1063,11 +1063,25 @@ (``spawn`` and ``forkserver``) have been added for starting processes using :mod:`multiprocessing`. These make the mixing of processes with threads more robust, and the ``spawn`` method matches the semantics that multiprocessing has -always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`). - -Also, except when using the old *fork* start method, child processes -will no longer inherit unneeded handles/file descriptors from their parents -(part of :issue:`8713`). +always used on Windows. New function +:func:`~multiprocessing.get_all_start_methods` reports all start methods +available on the platform, :func:`~multiprocessing.get_start_method` reports +the current start method, and :func:`~multiprocessing.set_start_method` sets +the start method. (Contributed by Richard Oudkerk in :issue:`8713`). + +:mod:`multiprocessing` also now has the concept of a ``context``, which +determines how child processes are created. New function +:func:`~multiprocessing.get_context` returns a context that uses a specified +start method. It has the same API as the :mod:`multiprocessing` module itself, +so you can use it to create :class:`~multiprocessing.pool.Pool`\ s and other +objects that will operate within that context. This allows a framework and an +application or different parts of the same application to use multiprocessing +without interfering with each other. (Contributed by Richard Oudkerk in +:issue:`18999`.) + +Except when using the old *fork* start method, child processes no longer +inherit unneeded handles/file descriptors from their parents (part of +:issue:`8713`). :mod:`multiprocessing` now relies on :mod:`runpy` (which implements the ``-m`` switch) to initialise ``__main__`` appropriately in child processes diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1636,6 +1636,9 @@ - Issue #18281: Unused stat constants removed from `tarfile`. +- Issue #18999: Multiprocessing now supports 'contexts' with the same API + as the module, but bound to specified start methods. + - Issue #18468: The re.split, re.findall, and re.sub functions and the group() and groups() methods of match object now always return a string or a bytes object. @@ -2051,6 +2054,10 @@ - Issue #18532: Change the builtin hash algorithms' names to lower case names as promised by hashlib's documentation. +- Issue #8713: add new spwan and forkserver start methods, and new functions + get_all_start_methods, get_start_method, and set_start_method, to + multiprocessing. + - Issue #18405: Improve the entropy of crypt.mksalt(). - Issue #12015: The tempfile module now uses a suffix of 8 random characters -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:39 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:39 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_PEP_424_implem?= =?utf-8?q?entation=2E?= Message-ID: <3fjYYl65mhz7LkK@mail.python.org> http://hg.python.org/cpython/rev/fada398b5d83 changeset: 89570:fada398b5d83 user: R David Murray date: Mon Mar 10 15:00:33 2014 -0400 summary: whatsnew: PEP 424 implementation. files: Doc/c-api/object.rst | 6 +++--- Doc/library/operator.rst | 4 ++-- Doc/whatsnew/3.4.rst | 13 +++++++++++++ Misc/NEWS | 3 +++ 4 files changed, 21 insertions(+), 5 deletions(-) diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -357,9 +357,9 @@ .. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default) - Return an estimated length for the object *o*. First trying to return its - actual length, then an estimate using ``__length_hint__``, and finally - returning the default value. On error ``-1`` is returned. This is the + Return an estimated length for the object *o*. First try to return its + actual length, then an estimate using :meth:`~object.__length_hint__`, and + finally return the default value. On error return ``-1``. This is the equivalent to the Python expression ``operator.length_hint(o, default)``. .. versionadded:: 3.4 diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -240,9 +240,9 @@ .. function:: length_hint(obj, default=0) - Return an estimated length for the object *o*. First trying to return its + Return an estimated length for the object *o*. First try to return its actual length, then an estimate using :meth:`object.__length_hint__`, and - finally returning the default value. + finally return the default value. .. versionadded:: 3.4 diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -393,6 +393,10 @@ several cases as a result of the introduction of Argument Clinic and other changes to the :mod:`inspect` and :mod:`pydoc` modules. +* :meth:`~object.__length_hint__` is now part of the formal language + specification (see :pep:`424`). (Contributed by Armin Ronacher in + :issue:`16148`.) + New Modules =========== @@ -1098,6 +1102,11 @@ reference and for use by alternate implementations of Python. (Contributed by Zachary Ware in :issue:`16694`.) +New function :func:`~operator.length_hint` provides an implementation of the +specification for how the :meth:`~object.__length_hint__` special method should +be used, as part of the :pep:`424` formal specification of this language +feature. (Contributed by Armin Ronacher in :issue:`16148`.) + os -- @@ -1800,6 +1809,10 @@ `_. (Contributed by Christian Heimes in :issue:`16632`.) +* New function :c:func:`PyObject_LengthHint` is the C API equivalent + of :func:`operator.length_hint`. (Contributed by Armin Ronacher in + :issue:`16148`.) + .. _other-improvements-3.4: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2538,6 +2538,9 @@ - Issue #16160: Subclass support now works for types.SimpleNamespace. +- Issue #16148: Implement PEP 424, adding operator.length_hint and + PyObject_LengthHint. + - Upgrade Unicode data (UCD) to version 6.2. - Issue #15379: Fix passing of non-BMP characters as integers for the charmap -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:41 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:41 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_json_dump-with?= =?utf-8?q?-indent_whitespace_change_=28=2316333=29=2E?= Message-ID: <3fjYYn0r83z7Ljv@mail.python.org> http://hg.python.org/cpython/rev/bb43e8e05a7c changeset: 89571:bb43e8e05a7c user: R David Murray date: Mon Mar 10 15:41:28 2014 -0400 summary: whatsnew: json dump-with-indent whitespace change (#16333). files: Doc/whatsnew/3.4.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2340,6 +2340,12 @@ *not* be installed, you must add ``--wihtout-pip`` to your command invocation. +* The default behavior of :func:`json.dump` and :func:`json.dumps` when + an indent is specified has changed: it no longer produces trailing + spaces after the item separating commas at the ends of lines. This + will matter only if you have tests that are doing white-space-sensitive + comparisons of such output (:issue:`16333`). + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:42 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_os=2Eopen_O=5F?= =?utf-8?b?VE1QRklMRSAoIzE4NjczKS4=?= Message-ID: <3fjYYp2jjQz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/0525f7268067 changeset: 89572:0525f7268067 user: R David Murray date: Mon Mar 10 15:41:59 2014 -0400 summary: whatsnew: os.open O_TMPFILE (#18673). files: Doc/whatsnew/3.4.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1129,6 +1129,11 @@ :func:`os.path.ismount` now recognizes volumes mounted below a drive root on Windows. (Contributed by Tim Golden in :issue:`9035`.) +:func:`os.open` supports a new flag, :data:`~os.O_TMPFILE` (unnamed temporary +file) on platforms that provide it (as of 3.4.0 release, that means Linux +systems with a kernel version of 3.11 or newer that have uapi headers). +(Contributed by Christian Heimes in :issue:`18673`.) + pdb --- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:43 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:43 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_os=2Eopen_O=5F?= =?utf-8?q?PATH_and_O=5FTMPFILE=2E?= Message-ID: <3fjYYq4rRBz7LkG@mail.python.org> http://hg.python.org/cpython/rev/6c1af13faa11 changeset: 89573:6c1af13faa11 user: R David Murray date: Mon Mar 10 15:48:22 2014 -0400 summary: whatsnew: os.open O_PATH and O_TMPFILE. files: Doc/library/os.rst | 5 +++-- Doc/whatsnew/3.4.rst | 9 +++++---- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -940,8 +940,9 @@ the C library. .. versionchanged:: 3.4 - Add :data:`O_TMPFILE` constant. It's only available on Linux Kernel 3.11 - or newer. + Add :data:`O_PATH` on systems that support it. + Add :data:`O_TMPFILE`, only available on Linux Kernel 3.11 + or newer. .. function:: openpty() diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1129,10 +1129,11 @@ :func:`os.path.ismount` now recognizes volumes mounted below a drive root on Windows. (Contributed by Tim Golden in :issue:`9035`.) -:func:`os.open` supports a new flag, :data:`~os.O_TMPFILE` (unnamed temporary -file) on platforms that provide it (as of 3.4.0 release, that means Linux -systems with a kernel version of 3.11 or newer that have uapi headers). -(Contributed by Christian Heimes in :issue:`18673`.) +:func:`os.open` supports two new flags on platforms that provide them, +:data:`~os.O_TMPFILE` (unnamed temporary file, as of 3.4.0 release available +only on Linux systems with a kernel version of 3.11 or newer that have uapi +headers), and :data:`~os.O_PATH` (un-opened file descriptor). (Contributed by +Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) pdb -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:44 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:44 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_shutil_copyfil?= =?utf-8?q?e_SameFileError_=28=231492704=29?= Message-ID: <3fjYYr6Vt3z7Ljl@mail.python.org> http://hg.python.org/cpython/rev/276227a93f6f changeset: 89574:276227a93f6f user: R David Murray date: Mon Mar 10 16:12:10 2014 -0400 summary: whatsnew: shutil copyfile SameFileError (#1492704) files: Doc/whatsnew/3.4.rst | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1297,6 +1297,16 @@ (Contributed by Filip Gruszczy?ski in :issue:`13896`.) +shutil +------ + +:func:`~shutil.copyfile` now raises a specific :exc:`~shutil.Error` subclass, +:exc:`~shutil.SameFileError`, when the source and destination are the same +file, which allows an application to take appropriate action on this specific +error. (Contributed by Atsuo Ishimoto and Hynek Schlawack in +:issue:`1492704`.) + + smtpd ----- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:46 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:46 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_urllib_Request?= =?utf-8?q?_objects_are_now_reusable=2E?= Message-ID: <3fjYYt1jqxz7LkS@mail.python.org> http://hg.python.org/cpython/rev/e6d862886e5c changeset: 89575:e6d862886e5c user: R David Murray date: Mon Mar 10 17:06:00 2014 -0400 summary: whatsnew: urllib Request objects are now reusable. #16464, #17485, #17272. files: Doc/whatsnew/3.4.rst | 13 +++++++++++++ Misc/NEWS | 5 ++++- 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1562,6 +1562,19 @@ class attribute on the subclass. (Contributed by Jason R Coombs in :issue:`18978`.) +:class:`~urllib.request.Request` objects are now reusable: if the +:attr:`~urllib.request.Request.full_url` or :attr:`~urllib.request.Request.data` +attributes are modified, all relevant internal properties are updated. This +means, for example, that it is now possible to use the same +:class:`~urllib.request.Request` object in more than one +:meth:`.OpenerDirector.open` call with different *data* arguments, or to +modify a :class:`~urllib.request.Request`\ 's ``url`` rather than recomputing it +from scratch. There is also a new +:meth:`~urllib.request.Request.remove_header` method that can be used to remove +headers from a :class:`~urllib.request.Request`. (Contributed by Alexey +Kachayev in :issue:`16464`, Daniel Wozniak in :issue:`17485`, and Damien Brecht +and Senthil Kumaran in :issue:`17272`.) + unittest -------- diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3053,7 +3053,7 @@ to procedurally generate, in an easy way, small test instances. - Issue #17485: Also delete the Request Content-Length header if the data - attribute is deleted. (Follow on to issue 16464). + attribute is deleted. (Follow on to issue Issue #16464). - Issue #15927: CVS now correctly parses escaped newlines and carriage when parsing with quoting turned off. @@ -3477,6 +3477,9 @@ list() calls aren't added to filter(), map(), and zip() which are directly passed enumerate(). +- Issue #16464: Reset the Content-Length header when a urllib Request is reused + with new data. + - Issue #12848: The pure Python pickle implementation now treats object lengths as unsigned 32-bit integers, like the C implementation does. Patch by Serhiy Storchaka. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:11:47 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:11:47 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_deprecation_of?= =?utf-8?q?_ElementTree_XMLParser_*html*_and_iterparse_*parser*=2E?= Message-ID: <3fjYYv4VWVz7Lkg@mail.python.org> http://hg.python.org/cpython/rev/31e6adf5bfba changeset: 89576:31e6adf5bfba user: R David Murray date: Mon Mar 10 18:09:54 2014 -0400 summary: whatsnew: deprecation of ElementTree XMLParser *html* and iterparse *parser*. #17741 files: Doc/library/xml.etree.elementtree.rst | 3 ++- Doc/whatsnew/3.4.rst | 5 +++++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -952,7 +952,8 @@ specified in the XML file. .. deprecated:: 3.4 - The *html* argument. + The *html* argument. The remaining arguments should be passed via + keywword to prepare for the removal of the *html* argument. .. method:: close() diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2057,6 +2057,11 @@ appropriate uses of :class:`io.TextIOWrapper` (if needed) and its *newline* argument. +* The *parser* argument of :func:`~xml.etree.ElementTree.iterparse` has + been deprecated, as has the *html* argument of + :func:`~xml.etree.ElementTree.XMLParser`. To prepare for the removal of the + latter, all arguments to ``XMLParser`` should be passed by keyword. + Deprecations in the C API ------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 10 23:15:47 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 10 Mar 2014 23:15:47 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_take_credit=2E?= Message-ID: <3fjYfW6tv3z7LjY@mail.python.org> http://hg.python.org/cpython/rev/02fc4fc32eec changeset: 89577:02fc4fc32eec user: R David Murray date: Mon Mar 10 18:15:09 2014 -0400 summary: whatsnew: take credit. They aren't all my words, but most of them are :). files: Doc/whatsnew/3.4.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2,8 +2,7 @@ What's New In Python 3.4 **************************** -.. :Author: Someone - (uncomment if there is a principal author) +:Author: R. David Murray .. Rules for maintenance: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 02:39:10 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 02:39:10 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_remove_XXXs=2E?= Message-ID: <3fjf9B5GY2z7LjS@mail.python.org> http://hg.python.org/cpython/rev/dfedeb76a20f changeset: 89578:dfedeb76a20f user: R David Murray date: Mon Mar 10 21:38:37 2014 -0400 summary: whatsnew: remove XXXs. Can't figure out what the other issue was for the importlib changes. Did find a small typo in the reload doc, though. files: Doc/library/importlib.rst | 2 +- Doc/whatsnew/3.4.rst | 8 +------- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -134,7 +134,7 @@ When :func:`reload` is executed: - * Python modules' code is recompiled and the module-level code re-executed, + * Python module's code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module's dictionary by reusing the :term:`loader` which originally loaded the module. The ``init`` function of extension modules is not called a second diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2062,12 +2062,6 @@ latter, all arguments to ``XMLParser`` should be passed by keyword. -Deprecations in the C API -------------------------- - -XXX: None so far - - Deprecated Features ------------------- @@ -2205,7 +2199,7 @@ * Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. Note that this restores a pre-3.3 behavior in that it means a module is - re-found when re-loaded (:issue:`19413` and XXX). + re-found when re-loaded (:issue:`19413`). * Frozen packages no longer set ``__path__`` to a list containing the package name, they now set it to an empty list. The previous behavior could cause -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 08:13:44 2014 From: python-checkins at python.org (victor.stinner) Date: Tue, 11 Mar 2014 08:13:44 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320888=3A_improve_?= =?utf-8?q?=22Pretty_Top=22_example_of_tracemalloc=2C_use_linecache?= Message-ID: <3fjnbD4gj5z7LjS@mail.python.org> http://hg.python.org/cpython/rev/14c1ff6a8086 changeset: 89579:14c1ff6a8086 user: Victor Stinner date: Tue Mar 11 08:12:48 2014 +0100 summary: Issue #20888: improve "Pretty Top" example of tracemalloc, use linecache files: Doc/library/tracemalloc.rst | 39 ++++++++++++++++-------- 1 files changed, 26 insertions(+), 13 deletions(-) diff --git a/Doc/library/tracemalloc.rst b/Doc/library/tracemalloc.rst --- a/Doc/library/tracemalloc.rst +++ b/Doc/library/tracemalloc.rst @@ -184,6 +184,7 @@ Code to display the 10 lines allocating the most memory with a pretty output, ignoring ```` and ```` files:: + import linecache import os import tracemalloc @@ -201,6 +202,9 @@ filename = os.sep.join(frame.filename.split(os.sep)[-2:]) print("#%s: %s:%s: %.1f KiB" % (index, filename, frame.lineno, stat.size / 1024)) + line = linecache.getline(frame.filename, frame.lineno).strip() + if line: + print(' %s' % line) other = top_stats[limit:] if other: @@ -218,19 +222,28 @@ Example of output of the Python test suite:: - 2013-11-08 14:16:58.149320: Top 10 lines - #1: collections/__init__.py:368: 291.9 KiB - #2: Lib/doctest.py:1291: 200.2 KiB - #3: unittest/case.py:571: 160.3 KiB - #4: Lib/abc.py:133: 99.8 KiB - #5: urllib/parse.py:476: 71.8 KiB - #6: :5: 62.7 KiB - #7: Lib/base64.py:140: 59.8 KiB - #8: Lib/_weakrefset.py:37: 51.8 KiB - #9: collections/__init__.py:362: 50.6 KiB - #10: test/test_site.py:56: 48.0 KiB - 7496 other: 4161.9 KiB - Total allocated size: 5258.8 KiB + Top 10 lines + #1: Lib/base64.py:414: 419.8 KiB + _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars] + #2: Lib/base64.py:306: 419.8 KiB + _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars] + #3: collections/__init__.py:368: 293.6 KiB + exec(class_definition, namespace) + #4: Lib/abc.py:133: 115.2 KiB + cls = super().__new__(mcls, name, bases, namespace) + #5: unittest/case.py:574: 103.1 KiB + testMethod() + #6: Lib/linecache.py:127: 95.4 KiB + lines = fp.readlines() + #7: urllib/parse.py:476: 71.8 KiB + for a in _hexdig for b in _hexdig} + #8: :5: 62.0 KiB + #9: Lib/_weakrefset.py:37: 60.0 KiB + self.data = set() + #10: Lib/base64.py:142: 59.8 KiB + _b32tab2 = [a + b for a in _b32tab for b in _b32tab] + 6220 other: 3602.8 KiB + Total allocated size: 5303.1 KiB See :meth:`Snapshot.statistics` for more options. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Mar 11 10:13:23 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 11 Mar 2014 10:13:23 +0100 Subject: [Python-checkins] Daily reference leaks (dfedeb76a20f): sum=0 Message-ID: results for dfedeb76a20f on branch "default" -------------------------------------------- test_site leaked [0, 2, -2] references, sum=0 test_site leaked [0, 2, -2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogTlLoH8', '-x'] From python-checkins at python.org Tue Mar 11 10:29:13 2014 From: python-checkins at python.org (georg.brandl) Date: Tue, 11 Mar 2014 10:29:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzEzNTMwOiBwb3J0?= =?utf-8?q?_to_2=2E7_branch_=28document_what_os=2Elseek_returns=29=2E?= Message-ID: <3fjrbY57fVz7LkW@mail.python.org> http://hg.python.org/cpython/rev/168e40af4a20 changeset: 89580:168e40af4a20 branch: 2.7 parent: 89558:0ae254c11921 user: Georg Brandl date: Tue Mar 11 10:28:56 2014 +0100 summary: #13530: port to 2.7 branch (document what os.lseek returns). files: Doc/library/os.rst | 2 +- Modules/posixmodule.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -744,7 +744,7 @@ by *how*: :const:`SEEK_SET` or ``0`` to set the position relative to the beginning of the file; :const:`SEEK_CUR` or ``1`` to set it relative to the current position; :const:`SEEK_END` or ``2`` to set it relative to the end of - the file. + the file. Return the new cursor position in bytes, starting from the beginning. Availability: Unix, Windows. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -6667,7 +6667,8 @@ PyDoc_STRVAR(posix_lseek__doc__, "lseek(fd, pos, how) -> newpos\n\n\ -Set the current position of a file descriptor."); +Set the current position of a file descriptor.\n\ +Return the new cursor position in bytes, starting from the beginning."); static PyObject * posix_lseek(PyObject *self, PyObject *args) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 13:17:40 2014 From: python-checkins at python.org (victor.stinner) Date: Tue, 11 Mar 2014 13:17:40 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_What=27s_New_in_Python_3?= =?utf-8?q?=2E4=3A_Security_improvements?= Message-ID: <3fjwKw20ZHz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/16384988a526 changeset: 89581:16384988a526 parent: 89579:14c1ff6a8086 user: Victor Stinner date: Tue Mar 11 13:17:30 2014 +0100 summary: What's New in Python 3.4: Security improvements files: Doc/library/multiprocessing.rst | 2 +- Doc/whatsnew/3.4.rst | 30 ++++++++++++++++++-- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -137,7 +137,7 @@ over Unix pipes. .. versionchanged:: 3.4 - *span* added on all unix platforms, and *forkserver* added for + *spawn* added on all unix platforms, and *forkserver* added for some unix platforms. Child processes no longer inherit all of the parents inheritable handles on Windows. diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -124,8 +124,6 @@ * :ref:`Single-dispatch generic functions ` in :mod:`functools` (:pep:`443`). * New :mod:`pickle` :ref:`protocol 4 ` (:pep:`3154`). -* :ref:`TLSv1.1 and TLSv1.2 support ` for :mod:`ssl` - (:issue:`16692`). * :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork on Unix ` (:issue:`8713`). * :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and @@ -136,6 +134,26 @@ correct introspection of a much wider variety of callable objects * The :mod:`ipaddress` module API has been declared stable +Security improvements: + +* :ref:`Secure and interchangeable hash algorithm ` + (:pep:`456`). +* :ref:`Make newly created file descriptors non-inheritable ` + (:pep:`446`) to avoid leaking file descriptors to child processes. +* A new :func:`hashlib.pbkdf2_hmac` function provides + the `PKCS#5 password-based key derivation function 2 + `_. +* :ref:`TLSv1.1 and TLSv1.2 support ` for :mod:`ssl`. +* :ref:`Retrieving certificates from the Windows system cert store support + ` for :mod:`ssl`. +* :ref:`Server-side SNI (Server Name Indication) support + ` for :mod:`ssl`. +* The :class:`ssl.SSLContext` class got a :ref:`lot of improvements + `. +* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork + on Unix `: *spawn* and *forkserver* avoid + sharing data with child processes; child processes no longer inherit all of + the parents inheritable handles on Windows. CPython implementation improvements: @@ -1364,6 +1382,8 @@ Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orr? and Antoine Pitrou in :issue:`16692`) +.. _whatsnew34-sslcontext: + :class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_verify_locations` accepts a new optional argument *cadata*, which can be used to provide PEM or DER encoded certificates directly via strings or bytes, respectively. @@ -1383,12 +1403,16 @@ list of the loaded ``CA`` certificates. (Contributed by Christian Heimes in and :issue:`18147`.) +.. _whatsnew34-win-cert-store: + Two new windows-only functions, :func:`~ssl.enum_certificates` and :func:`~ssl.enum_crls` provide the ability to retrieve certificates, certificate information, and CRLs from the Windows cert store. (Contributed by Christian Heimes in :issue:`17134`.) -Support for server-side SNI using the new +.. _whatsnew34-sni: + +Support for server-side SNI (Server Name Indication) using the new :meth:`ssl.SSLContext.set_servername_callback` method. (Contributed by Daniel Black in :issue:`8109`.) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 17:57:21 2014 From: python-checkins at python.org (peter.moody) Date: Tue, 11 Mar 2014 17:57:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319157=3A_Include_?= =?utf-8?q?the_broadcast_address_in_the_usuable_hosts_for_IPv6?= Message-ID: <3fk2Xd1p5xz7LjR@mail.python.org> http://hg.python.org/cpython/rev/b6271cbcc762 changeset: 89582:b6271cbcc762 user: Peter Moody date: Tue Mar 11 09:55:46 2014 -0700 summary: Issue #19157: Include the broadcast address in the usuable hosts for IPv6 in ipaddress. files: Lib/ipaddress.py | 12 ++++++++++++ Misc/NEWS | 3 +++ 2 files changed, 15 insertions(+), 0 deletions(-) diff --git a/Lib/ipaddress.py b/Lib/ipaddress.py --- a/Lib/ipaddress.py +++ b/Lib/ipaddress.py @@ -2155,6 +2155,18 @@ if self._prefixlen == (self._max_prefixlen - 1): self.hosts = self.__iter__ + def hosts(self): + """Generate Iterator over usable hosts in a network. + + This is like __iter__ except it doesn't return the + Subnet-Router anycast address. + + """ + network = int(self.network_address) + broadcast = int(self.broadcast_address) + for x in range(network + 1, broadcast + 1): + yield self._address_class(x) + @property def is_site_local(self): """Test if the address is reserved for site-local. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,9 @@ Library ------- +- Issue #19157: Include the broadcast address in the usuable hosts for IPv6 + in ipaddress. + - Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 19:05:22 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 19:05:22 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_summary_sectio?= =?utf-8?q?n_tweaks=2E?= Message-ID: <3fk4365L4Jz7LjS@mail.python.org> http://hg.python.org/cpython/rev/9ae1707d427a changeset: 89583:9ae1707d427a user: R David Murray date: Tue Mar 11 14:03:24 2014 -0400 summary: whatsnew: summary section tweaks. Removed an entry made redundant by the new security section, Moved 'marshal' to the section on features other platforms will need to support, and mention that 'help' is improved by the inspect/pydoc changes (#19561 msg213153). files: Doc/whatsnew/3.4.rst | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -100,6 +100,8 @@ that are not text encodings (multiple issues). * :ref:`A ModuleSpec Type ` for the Import System (:pep:`451`). (Affects importer authors.) +* The :mod:`marshal` format has been made :ref:`more compact and efficient + ` (:issue:`16475`). New library modules: @@ -131,7 +133,8 @@ (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME handling ` (:issue:`18891`). * The :mod:`inspect` and :mod:`pydoc` modules are now capable of - correct introspection of a much wider variety of callable objects + correct introspection of a much wider variety of callable objects, + which improves the output of the Python :func:`help` system. * The :mod:`ipaddress` module API has been declared stable Security improvements: @@ -161,11 +164,7 @@ * Leveraging :pep:`442`, :ref:`module globals are no longer set to None during finalization `, in most cases (:issue:`18214`). * :ref:`Configurable memory allocators ` (:pep:`445`). -* :ref:`Secure and interchangeable hash algorithm ` - (:pep:`456`). * :ref:`Argument Clinic ` (:pep:`436`). -* The :mod:`marshal` format has been made :ref:`more compact and efficient - ` (:issue:`16475`). Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 19:58:00 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 19:58:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_mention_that_u?= =?utf-8?q?pgrading_PIP_means_it_doesn=27t_get_unistalled=2E?= Message-ID: <3fk5Cr3JRhz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/9ec36351f684 changeset: 89584:9ec36351f684 user: R David Murray date: Tue Mar 11 14:57:30 2014 -0400 summary: whatsnew: mention that upgrading PIP means it doesn't get unistalled. #19861 msg213156. files: Doc/whatsnew/3.4.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -450,9 +450,11 @@ :mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first release candidate of the release of CPython with which it ships (this applies to both maintenance releases and feature releases). ``ensurepip`` does not -access the internet. (If the installation has Internet access, it is of course -possible to upgrade ``pip`` to a release more recent than the bundled ``pip`` -by using the bundled ``pip`` command itself once it is installed.) +access the internet. If the installation has Internet access, after +``ensurepip`` is run the bundled ``pip`` can be used to upgrade ``pip`` to a +more recent release than the bundled one. (Note that such an upgraded version +of ``pip`` is considered to be a separately installed package and will not be +removed if Python is uninstalled.) The module is named *ensure*\ pip because if called when ``pip`` is already installed, it does nothing. It also has an ``--upgrade`` option that will -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:00:06 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 11 Mar 2014 20:00:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogZml4IHR5cG8gKGNs?= =?utf-8?q?oses_=2320892=29?= Message-ID: <3fk5GG4txxz7LkM@mail.python.org> http://hg.python.org/cpython/rev/2f149207d3cf changeset: 89585:2f149207d3cf branch: 3.3 parent: 89560:d975f50de5aa user: Benjamin Peterson date: Tue Mar 11 13:59:37 2014 -0500 summary: fix typo (closes #20892) files: Doc/howto/pyporting.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -111,7 +111,7 @@ Projects to Consider -------------------- -The lowest level library for suppoting Python 2 & 3 simultaneously is six_. +The lowest level library for supporting Python 2 & 3 simultaneously is six_. Reading through its documentation will give you an idea of where exactly the Python language changed between versions 2 & 3 and thus what you will want the library to help you continue to support. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:00:08 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 11 Mar 2014 20:00:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E3_=28closes_=2320892=29?= Message-ID: <3fk5GJ19Sfz7Ljb@mail.python.org> http://hg.python.org/cpython/rev/e47646fca428 changeset: 89586:e47646fca428 parent: 89584:9ec36351f684 parent: 89585:2f149207d3cf user: Benjamin Peterson date: Tue Mar 11 13:59:48 2014 -0500 summary: merge 3.3 (closes #20892) files: Doc/howto/pyporting.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -111,7 +111,7 @@ Projects to Consider -------------------- -The lowest level library for suppoting Python 2 & 3 simultaneously is six_. +The lowest level library for supporting Python 2 & 3 simultaneously is six_. Reading through its documentation will give you an idea of where exactly the Python language changed between versions 2 & 3 and thus what you will want the library to help you continue to support. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:13:35 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 20:13:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_doctest_finds_?= =?utf-8?q?tests_in_extension_modules_=28=233158=29?= Message-ID: <3fk5Yq1PmVz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/8520e0ff8e36 changeset: 89587:8520e0ff8e36 user: R David Murray date: Tue Mar 11 15:10:53 2014 -0400 summary: whatsnew: doctest finds tests in extension modules (#3158) #19861 msg213159 files: Doc/whatsnew/3.4.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -745,6 +745,9 @@ shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.) +:mod:`doctest` will now find doctests in extension module ``__doc__`` strings. +(Contributed by Zachary Ware in :issue:`3158`.) + email ----- @@ -2398,6 +2401,11 @@ will matter only if you have tests that are doing white-space-sensitive comparisons of such output (:issue:`16333`). +* :mod:`doctest` now looks for doctests in extension module ``__doc__`` + strings, so if your doctest test discovery includes extension modules that + have things that look like doctests in them you may see test failures you've + never seen before when running your tests (:issue:`3158`). + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:26:32 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 11 Mar 2014 20:26:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_improve_algori?= =?utf-8?q?thms=5F*_documentation?= Message-ID: <3fk5rm1CN2z7LjY@mail.python.org> http://hg.python.org/cpython/rev/abb3d29d1bff changeset: 89588:abb3d29d1bff branch: 3.3 parent: 89585:2f149207d3cf user: Benjamin Peterson date: Tue Mar 11 14:26:12 2014 -0500 summary: improve algorithms_* documentation files: Doc/library/hashlib.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -95,18 +95,18 @@ .. data:: algorithms_guaranteed - Contains the names of the hash algorithms guaranteed to be supported + A set containing the names of the hash algorithms guaranteed to be supported by this module on all platforms. .. versionadded:: 3.2 .. data:: algorithms_available - Contains the names of the hash algorithms that are available - in the running Python interpreter. These names will be recognized - when passed to :func:`new`. :attr:`algorithms_guaranteed` - will always be a subset. Duplicate algorithms with different - name formats may appear in this set (thanks to OpenSSL). + A set containing the names of the hash algorithms that are available in the + running Python interpreter. These names will be recognized when passed to + :func:`new`. :attr:`algorithms_guaranteed` will always be a subset. The + same algorithm may appear multiple times in this set under different names + (thanks to OpenSSL). .. versionadded:: 3.2 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:26:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 11 Mar 2014 20:26:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fk5rn4KfZz7Ljt@mail.python.org> http://hg.python.org/cpython/rev/0d230ad6b51a changeset: 89589:0d230ad6b51a parent: 89587:8520e0ff8e36 parent: 89588:abb3d29d1bff user: Benjamin Peterson date: Tue Mar 11 14:26:21 2014 -0500 summary: merge 3.3 files: Doc/library/hashlib.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -101,18 +101,18 @@ .. data:: algorithms_guaranteed - Contains the names of the hash algorithms guaranteed to be supported + A set containing the names of the hash algorithms guaranteed to be supported by this module on all platforms. .. versionadded:: 3.2 .. data:: algorithms_available - Contains the names of the hash algorithms that are available - in the running Python interpreter. These names will be recognized - when passed to :func:`new`. :attr:`algorithms_guaranteed` - will always be a subset. Duplicate algorithms with different - name formats may appear in this set (thanks to OpenSSL). + A set containing the names of the hash algorithms that are available in the + running Python interpreter. These names will be recognized when passed to + :func:`new`. :attr:`algorithms_guaranteed` will always be a subset. The + same algorithm may appear multiple times in this set under different names + (thanks to OpenSSL). .. versionadded:: 3.2 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 20:28:49 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 11 Mar 2014 20:28:49 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_sync_pyporting?= =?utf-8?q?_howto_from_python_3_docs?= Message-ID: <3fk5vP3s9Tz7LjM@mail.python.org> http://hg.python.org/cpython/rev/3409fb4103e6 changeset: 89590:3409fb4103e6 branch: 2.7 parent: 89580:168e40af4a20 user: Benjamin Peterson date: Tue Mar 11 14:28:37 2014 -0500 summary: sync pyporting howto from python 3 docs files: Doc/howto/pyporting.rst | 614 ++++++++++++--------------- 1 files changed, 268 insertions(+), 346 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -10,238 +10,211 @@ With Python 3 being the future of Python while Python 2 is still in active use, it is good to have your project available for both major releases of - Python. This guide is meant to help you choose which strategy works best - for your project to support both Python 2 & 3 along with how to execute - that strategy. + Python. This guide is meant to help you figure out how best to support both + Python 2 & 3 simultaneously. If you are looking to port an extension module instead of pure Python code, please see :ref:`cporting-howto`. + If you would like to read one core Python developer's take on why Python 3 + came into existence, you can read Nick Coghlan's `Python 3 Q & A`_. -Choosing a Strategy -=================== + If you prefer to read a (free) book on porting a project to Python 3, + consider reading `Porting to Python 3`_ by Lennart Regebro which should cover + much of what is discussed in this HOWTO. -When a project makes the decision that it's time to support both Python 2 & 3, -a decision needs to be made as to how to go about accomplishing that goal. -The chosen strategy will depend on how large the project's existing -codebase is and how much divergence you want from your Python 2 codebase from -your Python 3 one (e.g., starting a new version with Python 3). + For help with porting, you can email the python-porting_ mailing list with + questions. -If your project is brand-new or does not have a large codebase, then you may -want to consider writing/porting :ref:`all of your code for Python 3 -and use 3to2 ` to port your code for Python 2. +The Short Version +================= -If you would prefer to maintain a codebase which is semantically **and** -syntactically compatible with Python 2 & 3 simultaneously, you can write -:ref:`use_same_source`. While this tends to lead to somewhat non-idiomatic -code, it does mean you keep a rapid development process for you, the developer. +* Decide what's the oldest version of Python 2 you want to support (if at all) +* Make sure you have a thorough test suite and use continuous integration + testing to make sure you stay compatible with the versions of Python you care + about +* If you have dependencies, check their Python 3 status using caniusepython3 + (`command-line tool `__, + `web app `__) -Finally, you do have the option of :ref:`using 2to3 ` to translate -Python 2 code into Python 3 code (with some manual help). This can take the -form of branching your code and using 2to3 to start a Python 3 branch. You can -also have users perform the translation at installation time automatically so -that you only have to maintain a Python 2 codebase. +With that done, your options are: -Regardless of which approach you choose, porting is not as hard or -time-consuming as you might initially think. You can also tackle the problem -piece-meal as a good portion of porting is simply updating your code to follow -current best practices in a Python 2/3 compatible way. +* If you are dropping Python 2 support, use 2to3_ to port to Python 3 +* If you are keeping Python 2 support, then start writing Python 2/3-compatible + code starting **TODAY** + + If you have dependencies that have not been ported, reach out to them to port + their project while working to make your code compatible with Python 3 so + you're ready when your dependencies are all ported + + If all your dependencies have been ported (or you have none), go ahead and + port to Python 3 -Universal Bits of Advice ------------------------- +* If you are creating a new project that wants to have 2/3 compatibility, + code in Python 3 and then backport to Python 2 -Regardless of what strategy you pick, there are a few things you should -consider. -One is make sure you have a robust test suite. You need to make sure everything -continues to work, just like when you support a new minor version of Python. -This means making sure your test suite is thorough and is ported properly -between Python 2 & 3. You will also most likely want to use something like tox_ -to automate testing between both a Python 2 and Python 3 VM. +Before You Begin +================ -Two, once your project has Python 3 support, make sure to add the proper -classifier on the Cheeseshop_ (PyPI_). To have your project listed as Python 3 -compatible it must have the -`Python 3 classifier `_ -(from -http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/):: +If your project is on the Cheeseshop_/PyPI_, make sure it has the proper +`trove classifiers`_ to signify what versions of Python it **currently** +supports. At minimum you should specify the major version(s), e.g. +``Programming Language :: Python :: 2`` if your project currently only supports +Python 2. It is preferrable that you be as specific as possible by listing every +major/minor version of Python that you support, e.g. if your project supports +Python 2.6 and 2.7, then you want the classifiers of:: - setup( - name='Your Library', - version='1.0', - classifiers=[ - # make sure to use :: Python *and* :: Python :: 3 so - # that pypi can list the package on the python 3 page - 'Programming Language :: Python', - 'Programming Language :: Python :: 3' - ], - packages=['yourlibrary'], - # make sure to add custom_fixers to the MANIFEST.in - include_package_data=True, - # ... - ) + Programming Language :: Python :: 2 + Programming Language :: Python :: 2.6 + Programming Language :: Python :: 2.7 +Once your project supports Python 3 you will want to go back and add the +appropriate classifiers for Python 3 as well. This is important as setting the +``Programming Language :: Python :: 3`` classifier will lead to your project +being listed under the `Python 3 Packages`_ section of PyPI. -Doing so will cause your project to show up in the -`Python 3 packages list -`_. You will know -you set the classifier properly as visiting your project page on the Cheeseshop -will show a Python 3 logo in the upper-left corner of the page. +Make sure you have a robust test suite. You need to +make sure everything continues to work, just like when you support a new +minor/feature release of Python. This means making sure your test suite is +thorough and is ported properly between Python 2 & 3 (consider using coverage_ +to measure that you have effective test coverage). You will also most likely +want to use something like tox_ to automate testing between all of your +supported versions of Python. You will also want to **port your tests first** so +that you can make sure that you detect breakage during the transition. Tests also +tend to be simpler than the code they are testing so it gives you an idea of how +easy it can be to port code. -Three, the six_ project provides a library which helps iron out differences -between Python 2 & 3. If you find there is a sticky point that is a continual -point of contention in your translation or maintenance of code, consider using -a source-compatible solution relying on six. If you have to create your own -Python 2/3 compatible solution, you can use ``sys.version_info[0] >= 3`` as a -guard. - -Four, read all the approaches. Just because some bit of advice applies to one -approach more than another doesn't mean that some advice doesn't apply to other -strategies. - -Five, drop support for older Python versions if possible. `Python 2.5`_ +Drop support for older Python versions if possible. `Python 2.5`_ introduced a lot of useful syntax and libraries which have become idiomatic in Python 3. `Python 2.6`_ introduced future statements which makes compatibility much easier if you are going from Python 2 to 3. -`Python 2.7`_ continues the trend in the stdlib. So choose the newest version +`Python 2.7`_ continues the trend in the stdlib. Choose the newest version of Python which you believe can be your minimum support version and work from there. +Target the newest version of Python 3 that you can. Beyond just the usual +bugfixes, compatibility has continued to improve between Python 2 and 3 as time +has passed. E.g. Python 3.3 added back the ``u`` prefix for +strings, making source-compatible Python code easier to write. -.. _tox: http://codespeak.net/tox/ -.. _Cheeseshop: -.. _PyPI: http://pypi.python.org/ -.. _six: http://packages.python.org/six -.. _Python 2.7: http://www.python.org/2.7.x -.. _Python 2.6: http://www.python.org/2.6.x -.. _Python 2.5: http://www.python.org/2.5.x -.. _Python 2.4: http://www.python.org/2.4.x -.. _Python 2.3: http://www.python.org/2.3.x -.. _Python 2.2: http://www.python.org/2.2.x +Writing Source-Compatible Python 2/3 Code +========================================= -.. _use_3to2: +Over the years the Python community has discovered that the easiest way to +support both Python 2 and 3 in parallel is to write Python code that works in +either version. While this might sound counter-intuitive at first, it actually +is not difficult and typically only requires following some select +(non-idiomatic) practices and using some key projects to help make bridging +between Python 2 and 3 easier. -Python 3 and 3to2 -================= +Projects to Consider +-------------------- -If you are starting a new project or your codebase is small enough, you may -want to consider writing your code for Python 3 and backporting to Python 2 -using 3to2_. Thanks to Python 3 being more strict about things than Python 2 -(e.g., bytes vs. strings), the source translation can be easier and more -straightforward than from Python 2 to 3. Plus it gives you more direct -experience developing in Python 3 which, since it is the future of Python, is a -good thing long-term. +The lowest level library for supporting Python 2 & 3 simultaneously is six_. +Reading through its documentation will give you an idea of where exactly the +Python language changed between versions 2 & 3 and thus what you will want the +library to help you continue to support. -A drawback of this approach is that 3to2 is a third-party project. This means -that the Python core developers (and thus this guide) can make no promises -about how well 3to2 works at any time. There is nothing to suggest, though, -that 3to2 is not a high-quality project. +To help automate porting your code over to using six, you can use +modernize_. This project will attempt to rewrite your code to be as modern as +possible while using six to smooth out any differences between Python 2 & 3. +If you want to write your compatible code to feel more like Python 3 there is +the future_ project. It tries to provide backports of objects from Python 3 so +that you can use them from Python 2-compatible code, e.g. replacing the +``bytes`` type from Python 2 with the one from Python 3. +It also provides a translation script like modernize (its translation code is +actually partially based on it) to help start working with a pre-existing code +base. It is also unique in that its translation script will also port Python 3 +code backwards as well as Python 2 code forwards. -.. _3to2: https://bitbucket.org/amentajo/lib3to2/overview +Tips & Tricks +------------- -.. _use_2to3: - -Python 2 and 2to3 -================= - -Included with Python since 2.6, the 2to3_ tool (and :mod:`lib2to3` module) -helps with porting Python 2 to Python 3 by performing various source -translations. This is a perfect solution for projects which wish to branch -their Python 3 code from their Python 2 codebase and maintain them as -independent codebases. You can even begin preparing to use this approach -today by writing future-compatible Python code which works cleanly in -Python 2 in conjunction with 2to3; all steps outlined below will work -with Python 2 code up to the point when the actual use of 2to3 occurs. - -Use of 2to3 as an on-demand translation step at install time is also possible, -preventing the need to maintain a separate Python 3 codebase, but this approach -does come with some drawbacks. While users will only have to pay the -translation cost once at installation, you as a developer will need to pay the -cost regularly during development. If your codebase is sufficiently large -enough then the translation step ends up acting like a compilation step, -robbing you of the rapid development process you are used to with Python. -Obviously the time required to translate a project will vary, so do an -experimental translation just to see how long it takes to evaluate whether you -prefer this approach compared to using :ref:`use_same_source` or simply keeping -a separate Python 3 codebase. - -Below are the typical steps taken by a project which uses a 2to3-based approach -to supporting Python 2 & 3. - +To help with writing source-compatible code using one of the projects mentioned +in `Projects to Consider`_, consider following the below suggestions. Some of +them are handled by the suggested projects, so if you do use one of them then +read their documentation first to see which suggestions below will taken care of +for you. Support Python 2.7 ------------------- +////////////////// As a first step, make sure that your project is compatible with `Python 2.7`_. This is just good to do as Python 2.7 is the last release of Python 2 and thus will be used for a rather long time. It also allows for use of the ``-3`` flag -to Python to help discover places in your code which 2to3 cannot handle but are -known to cause issues. +to Python to help discover places in your code where compatibility might be an +issue (the ``-3`` flag is in Python 2.6 but Python 2.7 adds more warnings). Try to Support `Python 2.6`_ and Newer Only -------------------------------------------- +/////////////////////////////////////////// While not possible for all projects, if you can support `Python 2.6`_ and newer **only**, your life will be much easier. Various future statements, stdlib additions, etc. exist only in Python 2.6 and later which greatly assist in -porting to Python 3. But if you project must keep support for `Python 2.5`_ (or -even `Python 2.4`_) then it is still possible to port to Python 3. +supporting Python 3. But if you project must keep support for `Python 2.5`_ then +it is still possible to simultaneously support Python 3. Below are the benefits you gain if you only have to support Python 2.6 and newer. Some of these options are personal choice while others are **strongly** recommended (the ones that are more for personal choice are labeled as such). If you continue to support older versions of Python then you -at least need to watch out for situations that these solutions fix. +at least need to watch out for situations that these solutions fix and handle +them appropriately (which is where library help from e.g. six_ comes in handy). ``from __future__ import print_function`` ''''''''''''''''''''''''''''''''''''''''' -This is a personal choice. 2to3 handles the translation from the print -statement to the print function rather well so this is an optional step. This -future statement does help, though, with getting used to typing -``print('Hello, World')`` instead of ``print 'Hello, World'``. +It will not only get you used to typing ``print()`` as a function instead of a +statement, but it will also give you the various benefits the function has over +the Python 2 statement (six_ provides a function if you support Python 2.5 or +older). ``from __future__ import unicode_literals`` ''''''''''''''''''''''''''''''''''''''''''' -Another personal choice. You can always mark what you want to be a (unicode) -string with a ``u`` prefix to get the same effect. But regardless of whether -you use this future statement or not, you **must** make sure you know exactly -which Python 2 strings you want to be bytes, and which are to be strings. This -means you should, **at minimum** mark all strings that are meant to be text -strings with a ``u`` prefix if you do not use this future statement. +If you choose to use this future statement then all string literals in +Python 2 will be assumed to be Unicode (as is already the case in Python 3). +If you choose not to use this future statement then you should mark all of your +text strings with a ``u`` prefix and only support Python 3.3 or newer. But you +are **strongly** advised to do one or the other (six_ provides a function in +case you don't want to use the future statement **and** you want to support +Python 3.2 or older). -Bytes literals -'''''''''''''' +Bytes/string literals +''''''''''''''''''''' -This is a **very** important one. The ability to prefix Python 2 strings that -are meant to contain bytes with a ``b`` prefix help to very clearly delineate -what is and is not a Python 3 string. When you run 2to3 on code, all Python 2 -strings become Python 3 strings **unless** they are prefixed with ``b``. +This is a **very** important one. Prefix Python 2 strings that +are meant to contain bytes with a ``b`` prefix to very clearly delineate +what is and is not a Python 3 text string (six_ provides a function to use for +Python 2.5 compatibility). + +This point cannot be stressed enough: make sure you know what all of your string +literals in Python 2 are meant to be in Python 3. Any string literal that +should be treated as bytes should have the ``b`` prefix. Any string literal +that should be Unicode/text in Python 2 should either have the ``u`` literal +(supported, but ignored, in Python 3.3 and later) or you should have +``from __future__ import unicode_literals`` at the top of the file. But the key +point is you should know how Python 3 will treat every one one of your string +literals and you should mark them as appropriate. There are some differences between byte literals in Python 2 and those in Python 3 thanks to the bytes type just being an alias to ``str`` in Python 2. -Probably the biggest "gotcha" is that indexing results in different values. In -Python 2, the value of ``b'py'[1]`` is ``'y'``, while in Python 3 it's ``121``. -You can avoid this disparity by always slicing at the size of a single element: -``b'py'[1:2]`` is ``'y'`` in Python 2 and ``b'y'`` in Python 3 (i.e., close -enough). +See the `Handle Common "Gotchas"`_ section for what to watch out for. -You cannot concatenate bytes and strings in Python 3. But since Python -2 has bytes aliased to ``str``, it will succeed: ``b'a' + u'b'`` works in -Python 2, but ``b'a' + 'b'`` in Python 3 is a :exc:`TypeError`. A similar issue -also comes about when doing comparisons between bytes and strings. +``from __future__ import absolute_import`` +'''''''''''''''''''''''''''''''''''''''''' +Discussed in more detail below, but you should use this future statement to +prevent yourself from accidentally using implicit relative imports. Supporting `Python 2.5`_ and Newer Only ---------------------------------------- +/////////////////////////////////////// If you are supporting `Python 2.5`_ and newer there are still some features of Python that you can utilize. @@ -251,7 +224,7 @@ '''''''''''''''''''''''''''''''''''''''''' Implicit relative imports (e.g., importing ``spam.bacon`` from within -``spam.eggs`` with the statement ``import bacon``) does not work in Python 3. +``spam.eggs`` with the statement ``import bacon``) do not work in Python 3. This future statement moves away from that and allows the use of explicit relative imports (e.g., ``from . import bacon``). @@ -261,16 +234,74 @@ the statement, but you still want the __future__ statement to prevent implicit relative imports. In `Python 2.7`_ the __future__ statement is not needed. In other words, unless you are only supporting Python 2.7 or a version earlier -than Python 2.5, use the __future__ statement. +than Python 2.5, use this __future__ statement. +Mark all Unicode strings with a ``u`` prefix +''''''''''''''''''''''''''''''''''''''''''''' + +While Python 2.6 has a ``__future__`` statement to automatically cause Python 2 +to treat all string literals as Unicode, Python 2.5 does not have that shortcut. +This means you should go through and mark all string literals with a ``u`` +prefix to turn them explicitly into text strings where appropriate and only +support Python 3.3 or newer. Otherwise use a project like six_ which provides a +function to pass all text string literals through. + + +Capturing the Currently Raised Exception +'''''''''''''''''''''''''''''''''''''''' + +In Python 2.5 and earlier the syntax to access the current exception is:: + + try: + raise Exception() + except Exception, exc: + # Current exception is 'exc'. + pass + +This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) +to:: + + try: + raise Exception() + except Exception as exc: + # Current exception is 'exc'. + # In Python 3, 'exc' is restricted to the block; in Python 2.6/2.7 it will "leak". + pass + +Because of this syntax change you must change how you capture the current +exception in Python 2.5 and earlier to:: + + try: + raise Exception() + except Exception: + import sys + exc = sys.exc_info()[1] + # Current exception is 'exc'. + pass + +You can get more information about the raised exception from +:func:`sys.exc_info` than simply the current exception instance, but you most +likely don't need it. + +.. note:: + In Python 3, the traceback is attached to the exception instance + through the ``__traceback__`` attribute. If the instance is saved in + a local variable that persists outside of the ``except`` block, the + traceback will create a reference cycle with the current frame and its + dictionary of local variables. This will delay reclaiming dead + resources until the next cyclic :term:`garbage collection` pass. + + In Python 2, this problem only occurs if you save the traceback itself + (e.g. the third element of the tuple returned by :func:`sys.exc_info`) + in a variable. + Handle Common "Gotchas" ------------------------ +/////////////////////// -There are a few things that just consistently come up as sticking points for -people which 2to3 cannot handle automatically or can easily be done in Python 2 -to help modernize your code. +These are things to watch out for no matter what version of Python 2 you are +supporting which are not syntactic considerations. ``from __future__ import division`` @@ -327,9 +358,9 @@ the bytes/string dichotomy. Because Python 2 allowed the ``str`` type to hold textual data, people have over the years been rather loose in their delineation of what ``str`` instances held text compared to bytes. In Python 3 you cannot -be so care-free anymore and need to properly handle the difference. The key +be so care-free anymore and need to properly handle the difference. The key to handling this issue is to make sure that **every** string literal in your -Python 2 code is either syntactically of functionally marked as either bytes or +Python 2 code is either syntactically or functionally marked as either bytes or text data. After this is done you then need to make sure your APIs are designed to either handle a specific type or made to be properly polymorphic. @@ -436,14 +467,7 @@ happen to use the ``unicode(self).encode('utf8')`` idiom as the body of your ``__str__()`` method). -There are two ways to solve this issue. One is to use a custom 2to3 fixer. The -blog post at http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ -specifies how to do this. That will allow 2to3 to change all instances of ``def -__unicode(self): ...`` to ``def __str__(self): ...``. This does require that you -define your ``__str__()`` method in Python 2 before your ``__unicode__()`` -method. - -The other option is to use a mixin class. This allows you to only define a +You can use a mixin class to work around this. This allows you to only define a ``__unicode__()`` method for your class and let the mixin derive ``__str__()`` for you (code from http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/):: @@ -486,6 +510,7 @@ Even better is to use the documented attributes the exception provides. + Don't use ``__getslice__`` & Friends '''''''''''''''''''''''''''''''''''' @@ -497,189 +522,62 @@ Updating doctests ''''''''''''''''' -2to3_ will attempt to generate fixes for doctests that it comes across. It's -not perfect, though. If you wrote a monolithic set of doctests (e.g., a single -docstring containing all of your doctests), you should at least consider -breaking the doctests up into smaller pieces to make it more manageable to fix. -Otherwise it might very well be worth your time and effort to port your tests -to :mod:`unittest`. +Don't forget to make them Python 2/3 compatible as well. If you wrote a +monolithic set of doctests (e.g., a single docstring containing all of your +doctests), you should at least consider breaking the doctests up into smaller +pieces to make it more manageable to fix. Otherwise it might very well be worth +your time and effort to port your tests to :mod:`unittest`. +Update ``map`` for imbalanced input sequences +''''''''''''''''''''''''''''''''''''''''''''' + +With Python 2, when ``map`` was given more than one input sequence it would pad +the shorter sequences with `None` values, returning a sequence as long as the +longest input sequence. + +With Python 3, if the input sequences to ``map`` are of unequal length, ``map`` +will stop at the termination of the shortest of the sequences. For full +compatibility with ``map`` from Python 2.x, wrap the sequence arguments in +:func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes +``list(map(func, itertools.zip_longest(*sequences)))``. + Eliminate ``-3`` Warnings ------------------------- When you run your application's test suite, run it using the ``-3`` flag passed to Python. This will cause various warnings to be raised during execution about -things that 2to3 cannot handle automatically (e.g., modules that have been -removed). Try to eliminate those warnings to make your code even more portable -to Python 3. +things that are semantic changes between Python 2 and 3. Try to eliminate those +warnings to make your code even more portable to Python 3. -Run 2to3 --------- +Alternative Approaches +====================== -Once you have made your Python 2 code future-compatible with Python 3, it's -time to use 2to3_ to actually port your code. +While supporting Python 2 & 3 simultaneously is typically the preferred choice +by people so that they can continue to improve code and have it work for the +most number of users, your life may be easier if you only have to support one +major version of Python going forward. +Supporting Only Python 3 Going Forward From Python 2 Code +--------------------------------------------------------- -Manually -'''''''' +If you have Python 2 code but going forward only want to improve it as Python 3 +code, then you can use 2to3_ to translate your Python 2 code to Python 3 code. +This is only recommended, though, if your current version of your project is +going into maintenance mode and you want all new features to be exclusive to +Python 3. -To manually convert source code using 2to3_, you use the ``2to3`` script that -is installed with Python 2.6 and later.:: - 2to3 +Backporting Python 3 code to Python 2 +------------------------------------- -This will cause 2to3 to write out a diff with all of the fixers applied for the -converted source code. If you would like 2to3 to go ahead and apply the changes -you can pass it the ``-w`` flag:: - - 2to3 -w - -There are other flags available to control exactly which fixers are applied, -etc. - - -During Installation -''''''''''''''''''' - -When a user installs your project for Python 3, you can have either -:mod:`distutils` or Distribute_ run 2to3_ on your behalf. -For distutils, use the following idiom:: - - try: # Python 3 - from distutils.command.build_py import build_py_2to3 as build_py - except ImportError: # Python 2 - from distutils.command.build_py import build_py - - setup(cmdclass = {'build_py': build_py}, - # ... - ) - -For Distribute:: - - setup(use_2to3=True, - # ... - ) - -This will allow you to not have to distribute a separate Python 3 version of -your project. It does require, though, that when you perform development that -you at least build your project and use the built Python 3 source for testing. - - -Verify & Test -------------- - -At this point you should (hopefully) have your project converted in such a way -that it works in Python 3. Verify it by running your unit tests and making sure -nothing has gone awry. If you miss something then figure out how to fix it in -Python 3, backport to your Python 2 code, and run your code through 2to3 again -to verify the fix transforms properly. - - -.. _2to3: http://docs.python.org/py3k/library/2to3.html -.. _Distribute: http://packages.python.org/distribute/ - - -.. _use_same_source: - -Python 2/3 Compatible Source -============================ - -While it may seem counter-intuitive, you can write Python code which is -source-compatible between Python 2 & 3. It does lead to code that is not -entirely idiomatic Python (e.g., having to extract the currently raised -exception from ``sys.exc_info()[1]``), but it can be run under Python 2 -**and** Python 3 without using 2to3_ as a translation step (although the tool -should be used to help find potential portability problems). This allows you to -continue to have a rapid development process regardless of whether you are -developing under Python 2 or Python 3. Whether this approach or using -:ref:`use_2to3` works best for you will be a per-project decision. - -To get a complete idea of what issues you will need to deal with, see the -`What's New in Python 3.0`_. Others have reorganized the data in other formats -such as http://docs.pythonsprints.com/python3_porting/py-porting.html\ . - -The following are some steps to take to try to support both Python 2 & 3 from -the same source code. - - -.. _What's New in Python 3.0: http://docs.python.org/release/3.0/whatsnew/3.0.html - - -Follow The Steps for Using 2to3_ --------------------------------- - -All of the steps outlined in how to -:ref:`port Python 2 code with 2to3 ` apply -to creating a Python 2/3 codebase. This includes trying only support Python 2.6 -or newer (the :mod:`__future__` statements work in Python 3 without issue), -eliminating warnings that are triggered by ``-3``, etc. - -You should even consider running 2to3_ over your code (without committing the -changes). This will let you know where potential pain points are within your -code so that you can fix them properly before they become an issue. - - -Use six_ --------- - -The six_ project contains many things to help you write portable Python code. -You should make sure to read its documentation from beginning to end and use -any and all features it provides. That way you will minimize any mistakes you -might make in writing cross-version code. - - -Capturing the Currently Raised Exception ----------------------------------------- - -One change between Python 2 and 3 that will require changing how you code (if -you support `Python 2.5`_ and earlier) is -accessing the currently raised exception. In Python 2.5 and earlier the syntax -to access the current exception is:: - - try: - raise Exception() - except Exception, exc: - # Current exception is 'exc' - pass - -This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) -to:: - - try: - raise Exception() - except Exception as exc: - # Current exception is 'exc' - # In Python 3, 'exc' is restricted to the block; Python 2.6 will "leak" - pass - -Because of this syntax change you must change to capturing the current -exception to:: - - try: - raise Exception() - except Exception: - import sys - exc = sys.exc_info()[1] - # Current exception is 'exc' - pass - -You can get more information about the raised exception from -:func:`sys.exc_info` than simply the current exception instance, but you most -likely don't need it. - -.. note:: - In Python 3, the traceback is attached to the exception instance - through the ``__traceback__`` attribute. If the instance is saved in - a local variable that persists outside of the ``except`` block, the - traceback will create a reference cycle with the current frame and its - dictionary of local variables. This will delay reclaiming dead - resources until the next cyclic :term:`garbage collection` pass. - - In Python 2, this problem only occurs if you save the traceback itself - (e.g. the third element of the tuple returned by :func:`sys.exc_info`) - in a variable. +If you have Python 3 code and have little interest in supporting Python 2 you +can use 3to2_ to translate from Python 3 code to Python 2 code. This is only +recommended if you don't plan to heavily support Python 2 users. Otherwise +write your code for Python 3 and then backport as far back as you want. This +is typically easier than going from Python 2 to 3 as you will have worked out +any difficulties with e.g. bytes/strings, etc. Other Resources @@ -687,17 +585,41 @@ The authors of the following blog posts, wiki pages, and books deserve special thanks for making public their tips for porting Python 2 code to Python 3 (and -thus helping provide information for this document): +thus helping provide information for this document and its various revisions +over the years): +* http://wiki.python.org/moin/PortingPythonToPy3k * http://python3porting.com/ * http://docs.pythonsprints.com/python3_porting/py-porting.html * http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/ * http://dabeaz.blogspot.com/2011/01/porting-py65-and-my-superboard-to.html * http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ * http://lucumr.pocoo.org/2010/2/11/porting-to-python-3-a-guide/ -* http://wiki.python.org/moin/PortingPythonToPy3k +* https://wiki.ubuntu.com/Python/3 If you feel there is something missing from this document that should be added, please email the python-porting_ mailing list. + + +.. _2to3: http://docs.python.org/2/library/2to3.html +.. _3to2: https://pypi.python.org/pypi/3to2 +.. _Cheeseshop: PyPI_ +.. _coverage: https://pypi.python.org/pypi/coverage +.. _future: http://python-future.org/ +.. _modernize: https://github.com/mitsuhiko/python-modernize +.. _Porting to Python 3: http://python3porting.com/ +.. _PyPI: http://pypi.python.org/ +.. _Python 2.2: http://www.python.org/2.2.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 2.6: http://www.python.org/2.6.x +.. _Python 2.7: http://www.python.org/2.7.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 3.3: http://www.python.org/3.3.x +.. _Python 3 Packages: https://pypi.python.org/pypi?:action=browse&c=533&show=all +.. _Python 3 Q & A: http://ncoghlan-devs-python-notes.readthedocs.org/en/latest/python3/questions_and_answers.html .. _python-porting: http://mail.python.org/mailman/listinfo/python-porting +.. _six: https://pypi.python.org/pypi/six +.. _tox: https://pypi.python.org/pypi/tox +.. _trove classifiers: https://pypi.python.org/pypi?%3Aaction=list_classifiers + -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 23:47:12 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 23:47:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzIwMDMwOiBkb2Mg?= =?utf-8?q?that_TestLoader=2Ediscover_returns_a_TestSuite=2E?= Message-ID: <3fkBJJ4szGz7Lk0@mail.python.org> http://hg.python.org/cpython/rev/cc74393062f7 changeset: 89591:cc74393062f7 branch: 2.7 user: R David Murray date: Tue Mar 11 18:12:25 2014 -0400 summary: #20030: doc that TestLoader.discover returns a TestSuite. Patch by Lita Cho. files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1510,11 +1510,11 @@ .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) - Find and return all test modules from the specified start directory, - recursing into subdirectories to find them. Only test files that match - *pattern* will be loaded. (Using shell style pattern matching.) Only - module names that are importable (i.e. are valid Python identifiers) will - be loaded. + Find all the test modules by recursing into subdirectories from the + specified start directory, and return a TestSuite object containing them. + Only test files that match *pattern* will be loaded. (Using shell style + pattern matching.) Only module names that are importable (i.e. are valid + Python identifiers) will be loaded. All test modules must be importable from the top level of the project. If the start directory is not the top level directory then the top level -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 23:47:14 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 23:47:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzIwMDMwOiBkb2Mg?= =?utf-8?q?that_TestLoader=2Ediscover_returns_a_TestSuite=2E?= Message-ID: <3fkBJL1TyCz7LkM@mail.python.org> http://hg.python.org/cpython/rev/71df53af61ec changeset: 89592:71df53af61ec branch: 3.3 parent: 89588:abb3d29d1bff user: R David Murray date: Tue Mar 11 18:46:00 2014 -0400 summary: #20030: doc that TestLoader.discover returns a TestSuite. Patch by Lita Cho. files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1489,11 +1489,11 @@ .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) - Find and return all test modules from the specified start directory, - recursing into subdirectories to find them. Only test files that match - *pattern* will be loaded. (Using shell style pattern matching.) Only - module names that are importable (i.e. are valid Python identifiers) will - be loaded. + Find all the test modules by recursing into subdirectories from the + specified start directory, and return a TestSuite object containing them. + Only test files that match *pattern* will be loaded. (Using shell style + pattern matching.) Only module names that are importable (i.e. are valid + Python identifiers) will be loaded. All test modules must be importable from the top level of the project. If the start directory is not the top level directory then the top level -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Mar 11 23:47:15 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 11 Mar 2014 23:47:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2320030=3A_doc_that_TestLoader=2Ediscover_returns?= =?utf-8?q?_a_TestSuite=2E?= Message-ID: <3fkBJM5Cc9z7Lk2@mail.python.org> http://hg.python.org/cpython/rev/ecc26a050384 changeset: 89593:ecc26a050384 parent: 89589:0d230ad6b51a parent: 89592:71df53af61ec user: R David Murray date: Tue Mar 11 18:46:37 2014 -0400 summary: Merge #20030: doc that TestLoader.discover returns a TestSuite. files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1624,11 +1624,11 @@ .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) - Find and return all test modules from the specified start directory, - recursing into subdirectories to find them. Only test files that match - *pattern* will be loaded. (Using shell style pattern matching.) Only - module names that are importable (i.e. are valid Python identifiers) will - be loaded. + Find all the test modules by recursing into subdirectories from the + specified start directory, and return a TestSuite object containing them. + Only test files that match *pattern* will be loaded. (Using shell style + pattern matching.) Only module names that are importable (i.e. are valid + Python identifiers) will be loaded. All test modules must be importable from the top level of the project. If the start directory is not the top level directory then the top level -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 02:59:15 2014 From: python-checkins at python.org (r.david.murray) Date: Wed, 12 Mar 2014 02:59:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_spelling_error?= =?utf-8?q?s=2E?= Message-ID: <3fkGYv0nRkzSqJ@mail.python.org> http://hg.python.org/cpython/rev/996652f3c136 changeset: 89594:996652f3c136 user: R David Murray date: Tue Mar 11 21:58:54 2014 -0400 summary: whatsnew: spelling errors. Corrections from Jim Jewett in #19861. files: Doc/whatsnew/3.4.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1680,7 +1680,7 @@ :meth:`wave.open` now supports the context manager protocol. (Contributed by Claudiu Popa in :issue:`17616`.) -:mod:`wave` can now :ref:`write output to unssekable files +:mod:`wave` can now :ref:`write output to unseekable files `. (Contributed by David Jones, Guilherme Polo, and Serhiy Storchaka in :issue:`5202`.) @@ -2392,7 +2392,7 @@ (:issue:`15204`). * If you use :ref:`pyvenv ` in a script and desire that pip - *not* be installed, you must add ``--wihtout-pip`` to your command + *not* be installed, you must add ``--without-pip`` to your command invocation. * The default behavior of :func:`json.dump` and :func:`json.dumps` when -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 05:31:42 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 05:31:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Link_to_OpenHatch_setup_g?= =?utf-8?b?dWlkZSAoIzIwODgyKS4=?= Message-ID: <3fkKxp3spsz7LjX@mail.python.org> http://hg.python.org/devguide/rev/3ffbbbac0a4d changeset: 673:3ffbbbac0a4d user: ?ric Araujo date: Wed Mar 12 00:31:33 2014 -0400 summary: Link to OpenHatch setup guide (#20882). This page contains information useful for total newcomers to Python or free software development (e.g. what is an IRC client and how to install one). files: setup.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/setup.rst b/setup.rst --- a/setup.rst +++ b/setup.rst @@ -7,6 +7,12 @@ available from http://www.python.org/). It also gives an overview of the directory structure of the CPython source code. +OpenHatch also has a great `setup guide`_ for Python for people who are +completely new to contributing to open source. + +.. _setup guide: https://openhatch.org/wiki/Contributing_to_Python + + .. _setup: Getting Set Up -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Wed Mar 12 07:16:53 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 07:16:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Define_what_?= =?utf-8?b?4oCcdXBkYXRpbmfigJ0gbWVhbnMgaW4gZG9jcyBmb3Igb3BlbiAoIzE5NjI3?= =?utf-8?b?KS4=?= Message-ID: <3fkNH9070kz7Ljb@mail.python.org> http://hg.python.org/cpython/rev/1f823c1559bd changeset: 89595:1f823c1559bd branch: 2.7 parent: 89591:cc74393062f7 user: ?ric Araujo date: Wed Mar 12 02:16:37 2014 -0400 summary: Define what ?updating? means in docs for open (#19627). Original patch by Lina Clark. files: Doc/library/functions.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -880,8 +880,8 @@ to use the system default, which is usually line buffered for tty devices and fully buffered for other files. If omitted, the system default is used. [#]_ - Modes ``'r+'``, ``'w+'`` and ``'a+'`` open the file for updating (note that - ``'w+'`` truncates the file). Append ``'b'`` to the mode to open the file in + Modes ``'r+'``, ``'w+'`` and ``'a+'`` open the file for updating (reading and writing); + note that ``'w+'`` truncates the file. Append ``'b'`` to the mode to open the file in binary mode, on systems that differentiate between binary and text files; on systems that don't have this distinction, adding the ``'b'`` has no effect. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 08:15:05 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 08:15:05 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQXZvaWQg4oCcZXJy?= =?utf-8?q?or=3A_None=E2=80=9D_messages_from_distutils_=28=234931=29=2E?= Message-ID: <3fkPZK2sS0z7Ljl@mail.python.org> http://hg.python.org/cpython/rev/a865f6fb82b4 changeset: 89596:a865f6fb82b4 branch: 2.7 user: ?ric Araujo date: Wed Mar 12 03:14:48 2014 -0400 summary: Avoid ?error: None? messages from distutils (#4931). Thanks to Amaury Forgeot d?Arc and Philip J. Eby. files: Doc/distutils/apiref.rst | 9 ------- Lib/distutils/core.py | 7 +---- Lib/distutils/dir_util.py | 4 +-- Lib/distutils/tests/test_util.py | 12 ++++++++- Lib/distutils/util.py | 23 +++---------------- Misc/NEWS | 3 ++ 6 files changed, 20 insertions(+), 38 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1167,15 +1167,6 @@ underscore. No { } or ( ) style quoting is available. -.. function:: grok_environment_error(exc[, prefix='error: ']) - - Generate a useful error message from an :exc:`EnvironmentError` (:exc:`IOError` - or :exc:`OSError`) exception object. Handles Python 1.5.1 and later styles, - and does what it can to deal with exception objects that don't have a filename - (which happens when the error is due to a two-file operation, such as - :func:`~os.rename` or :func:`~os.link`). Returns the error message as a - string prefixed with *prefix*. - .. function:: split_quoted(s) diff --git a/Lib/distutils/core.py b/Lib/distutils/core.py --- a/Lib/distutils/core.py +++ b/Lib/distutils/core.py @@ -14,7 +14,6 @@ from distutils.debug import DEBUG from distutils.errors import (DistutilsSetupError, DistutilsArgError, DistutilsError, CCompilerError) -from distutils.util import grok_environment_error # Mainly import these so setup scripts can "from distutils.core import" them. from distutils.dist import Distribution @@ -153,13 +152,11 @@ except KeyboardInterrupt: raise SystemExit, "interrupted" except (IOError, os.error), exc: - error = grok_environment_error(exc) - if DEBUG: - sys.stderr.write(error + "\n") + sys.stderr.write("error: %s\n" % (exc,)) raise else: - raise SystemExit, error + raise SystemExit, exc except (DistutilsError, CCompilerError), msg: diff --git a/Lib/distutils/dir_util.py b/Lib/distutils/dir_util.py --- a/Lib/distutils/dir_util.py +++ b/Lib/distutils/dir_util.py @@ -185,7 +185,6 @@ Any errors are ignored (apart from being reported to stdout if 'verbose' is true). """ - from distutils.util import grok_environment_error global _path_created if verbose >= 1: @@ -202,8 +201,7 @@ if abspath in _path_created: del _path_created[abspath] except (IOError, OSError), exc: - log.warn(grok_environment_error( - exc, "error removing %s: " % directory)) + log.warn("error removing %s: %s", directory, exc) def ensure_relative(path): """Take the full path 'path', and make it a relative path. diff --git a/Lib/distutils/tests/test_util.py b/Lib/distutils/tests/test_util.py --- a/Lib/distutils/tests/test_util.py +++ b/Lib/distutils/tests/test_util.py @@ -3,8 +3,9 @@ import unittest from test.test_support import run_unittest -from distutils.errors import DistutilsPlatformError, DistutilsByteCompileError -from distutils.util import byte_compile +from distutils.errors import DistutilsByteCompileError +from distutils.util import byte_compile, grok_environment_error + class UtilTestCase(unittest.TestCase): @@ -18,6 +19,13 @@ finally: sys.dont_write_bytecode = old_dont_write_bytecode + def test_grok_environment_error(self): + # test obsolete function to ensure backward compat (#4931) + exc = IOError("Unable to find batch file") + msg = grok_environment_error(exc) + self.assertEqual(msg, "error: Unable to find batch file") + + def test_suite(): return unittest.makeSuite(UtilTestCase) diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py --- a/Lib/distutils/util.py +++ b/Lib/distutils/util.py @@ -213,25 +213,10 @@ def grok_environment_error (exc, prefix="error: "): - """Generate a useful error message from an EnvironmentError (IOError or - OSError) exception object. Handles Python 1.5.1 and 1.5.2 styles, and - does what it can to deal with exception objects that don't have a - filename (which happens when the error is due to a two-file operation, - such as 'rename()' or 'link()'. Returns the error message as a string - prefixed with 'prefix'. - """ - # check for Python 1.5.2-style {IO,OS}Error exception objects - if hasattr(exc, 'filename') and hasattr(exc, 'strerror'): - if exc.filename: - error = prefix + "%s: %s" % (exc.filename, exc.strerror) - else: - # two-argument functions in posix module don't - # include the filename in the exception object! - error = prefix + "%s" % exc.strerror - else: - error = prefix + str(exc[-1]) - - return error + # Function kept for backward compatibility. + # Used to try clever things with EnvironmentErrors, + # but nowadays str(exception) produces good messages. + return prefix + str(exc) # Needed by 'split_quoted()' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,9 @@ as documented. The pattern and source keyword parameters are left as deprecated aliases. +- Issue #4931: distutils should not produce unhelpful "error: None" messages + anymore. distutils.util.grok_environment_error is kept but doc-deprecated. + - Improve the random module's default seeding to use 256 bits of entropy from os.urandom(). This was already done for Python 3, mildly improving security with a bigger seed space. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 08:19:46 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 08:19:46 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Restore_missin?= =?utf-8?q?g_part_of_error_message_=28=234931=29?= Message-ID: <3fkPgk0W9xz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/0f1237b61f58 changeset: 89597:0f1237b61f58 branch: 2.7 user: ?ric Araujo date: Wed Mar 12 03:19:33 2014 -0400 summary: Restore missing part of error message (#4931) files: Lib/distutils/core.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/distutils/core.py b/Lib/distutils/core.py --- a/Lib/distutils/core.py +++ b/Lib/distutils/core.py @@ -156,7 +156,7 @@ sys.stderr.write("error: %s\n" % (exc,)) raise else: - raise SystemExit, exc + raise SystemExit, "error: %s" % (exc,) except (DistutilsError, CCompilerError), msg: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 09:11:10 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 09:11:10 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQXZvaWQg4oCcZXJy?= =?utf-8?q?or=3A_None=E2=80=9D_messages_from_distutils_=28=234931=29=2E?= Message-ID: <3fkQq264Kvz7LjR@mail.python.org> http://hg.python.org/cpython/rev/504eb00998f2 changeset: 89598:504eb00998f2 branch: 3.3 parent: 89592:71df53af61ec user: ?ric Araujo date: Wed Mar 12 03:34:02 2014 -0400 summary: Avoid ?error: None? messages from distutils (#4931). Thanks to Amaury Forgeot d?Arc and Philip J. Eby. files: Doc/distutils/apiref.rst | 9 ------- Lib/distutils/core.py | 7 +---- Lib/distutils/dir_util.py | 6 +--- Lib/distutils/tests/test_util.py | 10 +++++++- Lib/distutils/util.py | 23 +++---------------- Misc/NEWS | 3 ++ 6 files changed, 20 insertions(+), 38 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1171,15 +1171,6 @@ underscore. No { } or ( ) style quoting is available. -.. function:: grok_environment_error(exc[, prefix='error: ']) - - Generate a useful error message from an :exc:`OSError` exception object. - Handles Python 1.5.1 and later styles, and does what it can to deal with - exception objects that don't have a filename (which happens when the error - is due to a two-file operation, such as :func:`~os.rename` or :func:`~os.link`). - Returns the error message as a string prefixed with *prefix*. - - .. function:: split_quoted(s) Split a string up according to Unix shell-like rules for quotes and backslashes. diff --git a/Lib/distutils/core.py b/Lib/distutils/core.py --- a/Lib/distutils/core.py +++ b/Lib/distutils/core.py @@ -11,7 +11,6 @@ from distutils.debug import DEBUG from distutils.errors import * -from distutils.util import grok_environment_error # Mainly import these so setup scripts can "from distutils.core import" them. from distutils.dist import Distribution @@ -149,13 +148,11 @@ except KeyboardInterrupt: raise SystemExit("interrupted") except (IOError, os.error) as exc: - error = grok_environment_error(exc) - if DEBUG: - sys.stderr.write(error + "\n") + sys.stderr.write("error: %s\n" % (exc,)) raise else: - raise SystemExit(error) + raise SystemExit("error: %s" % (exc,)) except (DistutilsError, CCompilerError) as msg: diff --git a/Lib/distutils/dir_util.py b/Lib/distutils/dir_util.py --- a/Lib/distutils/dir_util.py +++ b/Lib/distutils/dir_util.py @@ -2,7 +2,7 @@ Utility functions for manipulating directories and directory trees.""" -import os, sys +import os import errno from distutils.errors import DistutilsFileError, DistutilsInternalError from distutils import log @@ -182,7 +182,6 @@ Any errors are ignored (apart from being reported to stdout if 'verbose' is true). """ - from distutils.util import grok_environment_error global _path_created if verbose >= 1: @@ -199,8 +198,7 @@ if abspath in _path_created: del _path_created[abspath] except (IOError, OSError) as exc: - log.warn(grok_environment_error( - exc, "error removing %s: " % directory)) + log.warn("error removing %s: %s", directory, exc) def ensure_relative(path): """Take the full path 'path', and make it a relative path. diff --git a/Lib/distutils/tests/test_util.py b/Lib/distutils/tests/test_util.py --- a/Lib/distutils/tests/test_util.py +++ b/Lib/distutils/tests/test_util.py @@ -8,7 +8,8 @@ from distutils.errors import DistutilsPlatformError, DistutilsByteCompileError from distutils.util import (get_platform, convert_path, change_root, check_environ, split_quoted, strtobool, - rfc822_escape, byte_compile) + rfc822_escape, byte_compile, + grok_environment_error) from distutils import util # used to patch _environ_checked from distutils.sysconfig import get_config_vars from distutils import sysconfig @@ -285,6 +286,13 @@ finally: sys.dont_write_bytecode = old_dont_write_bytecode + def test_grok_environment_error(self): + # test obsolete function to ensure backward compat (#4931) + exc = IOError("Unable to find batch file") + msg = grok_environment_error(exc) + self.assertEqual(msg, "error: Unable to find batch file") + + def test_suite(): return unittest.makeSuite(UtilTestCase) diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py --- a/Lib/distutils/util.py +++ b/Lib/distutils/util.py @@ -213,25 +213,10 @@ def grok_environment_error (exc, prefix="error: "): - """Generate a useful error message from an EnvironmentError (IOError or - OSError) exception object. Handles Python 1.5.1 and 1.5.2 styles, and - does what it can to deal with exception objects that don't have a - filename (which happens when the error is due to a two-file operation, - such as 'rename()' or 'link()'. Returns the error message as a string - prefixed with 'prefix'. - """ - # check for Python 1.5.2-style {IO,OS}Error exception objects - if hasattr(exc, 'filename') and hasattr(exc, 'strerror'): - if exc.filename: - error = prefix + "%s: %s" % (exc.filename, exc.strerror) - else: - # two-argument functions in posix module don't - # include the filename in the exception object! - error = prefix + "%s" % exc.strerror - else: - error = prefix + str(exc.args[-1]) - - return error + # Function kept for backward compatibility. + # Used to try clever things with EnvironmentErrors, + # but nowadays str(exception) produces good messages. + return prefix + str(exc) # Needed by 'split_quoted()' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -16,6 +16,9 @@ - Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. +- Issue #4931: distutils should not produce unhelpful "error: None" messages + anymore. distutils.util.grok_environment_error is kept but doc-deprecated. + - Issue #20283: RE pattern methods now accept the string keyword parameters as documented. The pattern and source keyword parameters are left as deprecated aliases. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 09:11:12 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 09:11:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy4zICgjNDkzMSk=?= Message-ID: <3fkQq41R92z7Ljl@mail.python.org> http://hg.python.org/cpython/rev/c7bd0f953687 changeset: 89599:c7bd0f953687 parent: 89594:996652f3c136 parent: 89598:504eb00998f2 user: ?ric Araujo date: Wed Mar 12 04:10:51 2014 -0400 summary: Merge 3.3 (#4931) files: Doc/distutils/apiref.rst | 9 ------- Lib/distutils/core.py | 7 +---- Lib/distutils/dir_util.py | 6 +--- Lib/distutils/tests/test_util.py | 10 +++++++- Lib/distutils/util.py | 23 +++---------------- Misc/NEWS | 3 ++ 6 files changed, 20 insertions(+), 38 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1160,15 +1160,6 @@ underscore. No { } or ( ) style quoting is available. -.. function:: grok_environment_error(exc[, prefix='error: ']) - - Generate a useful error message from an :exc:`OSError` exception object. - Handles Python 1.5.1 and later styles, and does what it can to deal with - exception objects that don't have a filename (which happens when the error - is due to a two-file operation, such as :func:`~os.rename` or :func:`~os.link`). - Returns the error message as a string prefixed with *prefix*. - - .. function:: split_quoted(s) Split a string up according to Unix shell-like rules for quotes and backslashes. diff --git a/Lib/distutils/core.py b/Lib/distutils/core.py --- a/Lib/distutils/core.py +++ b/Lib/distutils/core.py @@ -11,7 +11,6 @@ from distutils.debug import DEBUG from distutils.errors import * -from distutils.util import grok_environment_error # Mainly import these so setup scripts can "from distutils.core import" them. from distutils.dist import Distribution @@ -150,13 +149,11 @@ except KeyboardInterrupt: raise SystemExit("interrupted") except OSError as exc: - error = grok_environment_error(exc) - if DEBUG: - sys.stderr.write(error + "\n") + sys.stderr.write("error: %s\n" % (exc,)) raise else: - raise SystemExit(error) + raise SystemExit("error: %s" % (exc,)) except (DistutilsError, CCompilerError) as msg: diff --git a/Lib/distutils/dir_util.py b/Lib/distutils/dir_util.py --- a/Lib/distutils/dir_util.py +++ b/Lib/distutils/dir_util.py @@ -2,7 +2,7 @@ Utility functions for manipulating directories and directory trees.""" -import os, sys +import os import errno from distutils.errors import DistutilsFileError, DistutilsInternalError from distutils import log @@ -182,7 +182,6 @@ Any errors are ignored (apart from being reported to stdout if 'verbose' is true). """ - from distutils.util import grok_environment_error global _path_created if verbose >= 1: @@ -199,8 +198,7 @@ if abspath in _path_created: del _path_created[abspath] except OSError as exc: - log.warn(grok_environment_error( - exc, "error removing %s: " % directory)) + log.warn("error removing %s: %s", directory, exc) def ensure_relative(path): """Take the full path 'path', and make it a relative path. diff --git a/Lib/distutils/tests/test_util.py b/Lib/distutils/tests/test_util.py --- a/Lib/distutils/tests/test_util.py +++ b/Lib/distutils/tests/test_util.py @@ -8,7 +8,8 @@ from distutils.errors import DistutilsPlatformError, DistutilsByteCompileError from distutils.util import (get_platform, convert_path, change_root, check_environ, split_quoted, strtobool, - rfc822_escape, byte_compile) + rfc822_escape, byte_compile, + grok_environment_error) from distutils import util # used to patch _environ_checked from distutils.sysconfig import get_config_vars from distutils import sysconfig @@ -285,6 +286,13 @@ finally: sys.dont_write_bytecode = old_dont_write_bytecode + def test_grok_environment_error(self): + # test obsolete function to ensure backward compat (#4931) + exc = IOError("Unable to find batch file") + msg = grok_environment_error(exc) + self.assertEqual(msg, "error: Unable to find batch file") + + def test_suite(): return unittest.makeSuite(UtilTestCase) diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py --- a/Lib/distutils/util.py +++ b/Lib/distutils/util.py @@ -207,25 +207,10 @@ def grok_environment_error (exc, prefix="error: "): - """Generate a useful error message from an OSError - exception object. Handles Python 1.5.1 and 1.5.2 styles, and - does what it can to deal with exception objects that don't have a - filename (which happens when the error is due to a two-file operation, - such as 'rename()' or 'link()'. Returns the error message as a string - prefixed with 'prefix'. - """ - # check for Python 1.5.2-style {IO,OS}Error exception objects - if hasattr(exc, 'filename') and hasattr(exc, 'strerror'): - if exc.filename: - error = prefix + "%s: %s" % (exc.filename, exc.strerror) - else: - # two-argument functions in posix module don't - # include the filename in the exception object! - error = prefix + "%s" % exc.strerror - else: - error = prefix + str(exc.args[-1]) - - return error + # Function kept for backward compatibility. + # Used to try clever things with EnvironmentErrors, + # but nowadays str(exception) produces good messages. + return prefix + str(exc) # Needed by 'split_quoted()' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -23,6 +23,9 @@ - Issue #19157: Include the broadcast address in the usuable hosts for IPv6 in ipaddress. +- Issue #4931: distutils should not produce unhelpful "error: None" messages + anymore. distutils.util.grok_environment_error is kept but doc-deprecated. + - Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Wed Mar 12 09:55:20 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 12 Mar 2014 09:55:20 +0100 Subject: [Python-checkins] Daily reference leaks (996652f3c136): sum=4 Message-ID: results for 996652f3c136 on branch "default" -------------------------------------------- test_site leaked [2, 0, 0] references, sum=2 test_site leaked [2, 0, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogZe_tyR', '-x'] From python-checkins at python.org Wed Mar 12 10:04:07 2014 From: python-checkins at python.org (eric.araujo) Date: Wed, 12 Mar 2014 10:04:07 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_missing_im?= =?utf-8?q?port_in_bdist=5Frpm_=28=2318045=29?= Message-ID: <3fkS075gmWz7Ltt@mail.python.org> http://hg.python.org/cpython/rev/677327810121 changeset: 89600:677327810121 branch: 2.7 parent: 89597:0f1237b61f58 user: ?ric Araujo date: Wed Mar 12 05:01:25 2014 -0400 summary: Fix missing import in bdist_rpm (#18045) files: Lib/distutils/command/bdist_rpm.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/command/bdist_rpm.py b/Lib/distutils/command/bdist_rpm.py --- a/Lib/distutils/command/bdist_rpm.py +++ b/Lib/distutils/command/bdist_rpm.py @@ -12,6 +12,7 @@ from distutils.core import Command from distutils.debug import DEBUG from distutils.file_util import write_file +from distutils.sysconfig import get_python_version from distutils.errors import (DistutilsOptionError, DistutilsPlatformError, DistutilsFileError, DistutilsExecError) from distutils import log -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 12:41:55 2014 From: python-checkins at python.org (victor.stinner) Date: Wed, 12 Mar 2014 12:41:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320896=3A_Workarou?= =?utf-8?q?nd_the_bug_temporarely_to_fix_buildbots?= Message-ID: <3fkWVC20L1z7Ljb@mail.python.org> http://hg.python.org/cpython/rev/c13398566409 changeset: 89601:c13398566409 parent: 89599:c7bd0f953687 user: Victor Stinner date: Wed Mar 12 12:41:44 2014 +0100 summary: Issue #20896: Workaround the bug temporarely to fix buildbots files: Lib/test/test_ssl.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1351,12 +1351,13 @@ def test_get_server_certificate(self): def _test_get_server_certificate(host, port, cert=None): with support.transient_internet(host): - pem = ssl.get_server_certificate((host, port)) + # FIXME: force PROTOCOL_SSLv23 for workaround bug #20896 + pem = ssl.get_server_certificate((host, port), ssl_version=ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) try: - pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE) + pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE, ssl_version=ssl.PROTOCOL_SSLv23) except ssl.SSLError as x: #should fail if support.verbose: @@ -1364,7 +1365,7 @@ else: self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) - pem = ssl.get_server_certificate((host, port), ca_certs=cert) + pem = ssl.get_server_certificate((host, port), ca_certs=cert, ssl_version=ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 19:14:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 19:14:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_c1339?= =?utf-8?q?8566409?= Message-ID: <3fkhCF0sPPz7LjX@mail.python.org> http://hg.python.org/cpython/rev/efe527e0f802 changeset: 89602:efe527e0f802 user: Benjamin Peterson date: Wed Mar 12 13:14:19 2014 -0500 summary: Backed out changeset c13398566409 files: Lib/test/test_ssl.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1351,13 +1351,12 @@ def test_get_server_certificate(self): def _test_get_server_certificate(host, port, cert=None): with support.transient_internet(host): - # FIXME: force PROTOCOL_SSLv23 for workaround bug #20896 - pem = ssl.get_server_certificate((host, port), ssl_version=ssl.PROTOCOL_SSLv23) + pem = ssl.get_server_certificate((host, port)) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) try: - pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE, ssl_version=ssl.PROTOCOL_SSLv23) + pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE) except ssl.SSLError as x: #should fail if support.verbose: @@ -1365,7 +1364,7 @@ else: self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) - pem = ssl.get_server_certificate((host, port), ca_certs=cert, ssl_version=ssl.PROTOCOL_SSLv23) + pem = ssl.get_server_certificate((host, port), ca_certs=cert) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 21:07:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 21:07:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogdXNlIHN1cHBvcnQu?= =?utf-8?q?rmtree_instead_of_shutil_=28closes_=2319614=29?= Message-ID: <3fkkjh6zn0z7LjM@mail.python.org> http://hg.python.org/cpython/rev/a5f767bf9d1c changeset: 89603:a5f767bf9d1c branch: 3.3 parent: 89598:504eb00998f2 user: Benjamin Peterson date: Wed Mar 12 15:07:01 2014 -0500 summary: use support.rmtree instead of shutil (closes #19614) Patch by Sean Rodman. files: Lib/test/test_support.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -102,7 +102,7 @@ self.assertTrue(os.path.isdir(path)) self.assertFalse(os.path.isdir(path)) finally: - shutil.rmtree(parent_dir) + support.rmtree(parent_dir) def test_temp_dir__path_none(self): """Test passing no path.""" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 21:07:38 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 21:07:38 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMTk2MTQp?= Message-ID: <3fkkjk1jhrz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/8a77e22aff6c changeset: 89604:8a77e22aff6c parent: 89602:efe527e0f802 parent: 89603:a5f767bf9d1c user: Benjamin Peterson date: Wed Mar 12 15:07:22 2014 -0500 summary: merge 3.3 (#19614) files: Lib/test/test_support.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -102,7 +102,7 @@ self.assertTrue(os.path.isdir(path)) self.assertFalse(os.path.isdir(path)) finally: - shutil.rmtree(parent_dir) + support.rmtree(parent_dir) def test_temp_dir__path_none(self): """Test passing no path.""" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 21:09:14 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 21:09:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_add_Sean_Rodma?= =?utf-8?q?n?= Message-ID: <3fkklZ4KdDz7LjN@mail.python.org> http://hg.python.org/cpython/rev/e682f87a6822 changeset: 89605:e682f87a6822 branch: 3.3 parent: 89603:a5f767bf9d1c user: Benjamin Peterson date: Wed Mar 12 15:08:43 2014 -0500 summary: add Sean Rodman files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1048,6 +1048,7 @@ Jim Robinson Mark Roddy Kevin Rodgers +Sean Rodman Giampaolo Rodola Elson Rodriguez Adi Roiban -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 21:09:15 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 21:09:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_add_Sean_Rodma?= =?utf-8?q?n?= Message-ID: <3fkklb61zkz7LjN@mail.python.org> http://hg.python.org/cpython/rev/9fe5894ef95e changeset: 89606:9fe5894ef95e branch: 2.7 parent: 89600:677327810121 user: Benjamin Peterson date: Wed Mar 12 15:08:43 2014 -0500 summary: add Sean Rodman files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -876,6 +876,7 @@ Jim Robinson Andy Robinson Kevin Rodgers +Sean Rodman Giampaolo Rodola Adi Roiban Mike Romberg -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Mar 12 21:09:17 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 12 Mar 2014 21:09:17 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fkkld0Fyjz7LjM@mail.python.org> http://hg.python.org/cpython/rev/fbdaca8bf3f5 changeset: 89607:fbdaca8bf3f5 parent: 89604:8a77e22aff6c parent: 89605:e682f87a6822 user: Benjamin Peterson date: Wed Mar 12 15:09:00 2014 -0500 summary: merge 3.3 files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1098,6 +1098,7 @@ Jim Robinson Mark Roddy Kevin Rodgers +Sean Rodman Giampaolo Rodola Elson Rodriguez Adi Roiban -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:11:15 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 00:11:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4xKTogdXNlIHNzbC5QUk9U?= =?utf-8?q?OCOL=5FSSLv23_for_maximum_compatibility_=28closes_=2320896=29?= Message-ID: <3fkpnb15WRz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/23add5382fb3 changeset: 89608:23add5382fb3 branch: 3.1 parent: 89430:c25e1442529f user: Benjamin Peterson date: Wed Mar 12 18:05:53 2014 -0500 summary: use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896) files: Lib/test/test_ssl.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -195,12 +195,15 @@ def test_get_server_certificate(self): with support.transient_internet("svn.python.org"): - pem = ssl.get_server_certificate(("svn.python.org", 443)) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on svn.python.org:443!") try: - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=CERTFILE) except ssl.SSLError as x: #should fail if support.verbose: @@ -208,7 +211,9 @@ else: self.fail("Got server certificate %s for svn.python.org!" % pem) - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=SVN_PYTHON_ORG_ROOT_CERT) if not pem: self.fail("No server certificate on svn.python.org:443!") if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:11:16 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 00:11:16 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogdXNlIHNzbC5QUk9U?= =?utf-8?q?OCOL=5FSSLv23_for_maximum_compatibility_=28closes_=2320896=29?= Message-ID: <3fkpnc381WzMbR@mail.python.org> http://hg.python.org/cpython/rev/789ca594960f changeset: 89609:789ca594960f branch: 3.2 parent: 89431:e82dcd700e8c user: Benjamin Peterson date: Wed Mar 12 18:05:53 2014 -0500 summary: use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896) files: Lib/test/test_ssl.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -690,12 +690,15 @@ def test_get_server_certificate(self): with support.transient_internet("svn.python.org"): - pem = ssl.get_server_certificate(("svn.python.org", 443)) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on svn.python.org:443!") try: - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=CERTFILE) except ssl.SSLError as x: #should fail if support.verbose: @@ -703,7 +706,9 @@ else: self.fail("Got server certificate %s for svn.python.org!" % pem) - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=SVN_PYTHON_ORG_ROOT_CERT) if not pem: self.fail("No server certificate on svn.python.org:443!") if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:11:17 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 00:11:17 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogdXNlIHNzbC5QUk9U?= =?utf-8?q?OCOL=5FSSLv23_for_maximum_compatibility_=28closes_=2320896=29?= Message-ID: <3fkpnd546PzMbR@mail.python.org> http://hg.python.org/cpython/rev/de97d0334314 changeset: 89610:de97d0334314 branch: 2.7 parent: 89606:9fe5894ef95e user: Benjamin Peterson date: Wed Mar 12 18:05:53 2014 -0500 summary: use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896) files: Lib/test/test_ssl.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -390,19 +390,24 @@ def test_get_server_certificate(self): with test_support.transient_internet("svn.python.org"): - pem = ssl.get_server_certificate(("svn.python.org", 443)) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on svn.python.org:443!") try: - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=CERTFILE) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=CERTFILE) except ssl.SSLError: #should fail pass else: self.fail("Got server certificate %s for svn.python.org!" % pem) - pem = ssl.get_server_certificate(("svn.python.org", 443), ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + pem = ssl.get_server_certificate(("svn.python.org", 443), + ssl.PROTOCOL_SSLv23, + ca_certs=SVN_PYTHON_ORG_ROOT_CERT) if not pem: self.fail("No server certificate on svn.python.org:443!") if test_support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:11:18 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 00:11:18 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2_=28=2320896=29?= Message-ID: <3fkpnf73zFz7LkC@mail.python.org> http://hg.python.org/cpython/rev/12df02358137 changeset: 89611:12df02358137 branch: 3.3 parent: 89605:e682f87a6822 parent: 89609:789ca594960f user: Benjamin Peterson date: Wed Mar 12 18:10:47 2014 -0500 summary: merge 3.2 (#20896) files: Lib/test/test_ssl.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -964,12 +964,15 @@ def test_get_server_certificate(self): def _test_get_server_certificate(host, port, cert=None): with support.transient_internet(host): - pem = ssl.get_server_certificate((host, port)) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) try: - pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23, + ca_certs=CERTFILE) except ssl.SSLError as x: #should fail if support.verbose: @@ -977,7 +980,9 @@ else: self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) - pem = ssl.get_server_certificate((host, port), ca_certs=cert) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23, + ca_certs=cert) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:11:20 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 00:11:20 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMjA4OTYp?= Message-ID: <3fkpnh1tbbz7LjS@mail.python.org> http://hg.python.org/cpython/rev/0cba79667c7d changeset: 89612:0cba79667c7d parent: 89607:fbdaca8bf3f5 parent: 89611:12df02358137 user: Benjamin Peterson date: Wed Mar 12 18:10:57 2014 -0500 summary: merge 3.3 (#20896) files: Lib/test/test_ssl.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1351,12 +1351,15 @@ def test_get_server_certificate(self): def _test_get_server_certificate(host, port, cert=None): with support.transient_internet(host): - pem = ssl.get_server_certificate((host, port)) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) try: - pem = ssl.get_server_certificate((host, port), ca_certs=CERTFILE) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23, + ca_certs=CERTFILE) except ssl.SSLError as x: #should fail if support.verbose: @@ -1364,7 +1367,9 @@ else: self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) - pem = ssl.get_server_certificate((host, port), ca_certs=cert) + pem = ssl.get_server_certificate((host, port), + ssl.PROTOCOL_SSLv23, + ca_certs=cert) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) if support.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:36:18 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 00:36:18 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_note_marku?= =?utf-8?b?cCAoIzE2ODA1KS4=?= Message-ID: <3fkqLV4vK0zQvG@mail.python.org> http://hg.python.org/cpython/rev/ca43fa662b0d changeset: 89613:ca43fa662b0d branch: 2.7 parent: 89610:de97d0334314 user: ?ric Araujo date: Wed Mar 12 19:35:54 2014 -0400 summary: Fix note markup (#16805). Patch by Tshepang Lekhonkhobe, reviewed by Georg Brandl. files: Doc/c-api/init.rst | 1 + Doc/library/getopt.rst | 1 + Doc/library/io.rst | 1 + Doc/library/logging.config.rst | 8 ++++++-- Doc/library/os.rst | 4 +++- Doc/library/socket.rst | 1 + 6 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -534,6 +534,7 @@ .. index:: module: thread .. note:: + When only the main thread exists, no GIL operations are needed. This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit. Therefore, the lock is not diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -10,6 +10,7 @@ -------------- .. note:: + The :mod:`getopt` module is a parser for command line options whose API is designed to be familiar to users of the C :c:func:`getopt` function. Users who are unfamiliar with the C :c:func:`getopt` function or who would like to write diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -644,6 +644,7 @@ :exc:`UnsupportedOperation`. .. warning:: + :class:`BufferedRWPair` does not attempt to synchronize accesses to its underlying raw streams. You should not pass it the same object as reader and writer; use :class:`BufferedRandom` instead. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -118,7 +118,9 @@ send it to the socket as a string of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. - .. note:: Because portions of the configuration are passed through + .. note:: + + Because portions of the configuration are passed through :func:`eval`, use of this function may open its users to a security risk. While the function only binds to a socket on ``localhost``, and so does not accept connections from remote machines, there are scenarios where @@ -721,7 +723,9 @@ :class:`~logging.Formatter` can present exception tracebacks in an expanded or condensed format. -.. note:: Due to the use of :func:`eval` as described above, there are +.. note:: + + Due to the use of :func:`eval` as described above, there are potential security risks which result from using the :func:`listen` to send and receive configurations via sockets. The risks are limited to where multiple users with no mutual trust run code on the same machine; see the diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -157,7 +157,9 @@ Availability: Unix. - .. note:: On Mac OS X, :func:`getgroups` behavior differs somewhat from + .. note:: + + On Mac OS X, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a deployment target of :const:`10.5` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -562,6 +562,7 @@ automatically closed when they are garbage-collected. .. note:: + :meth:`close()` releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call :meth:`shutdown()` -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:52:05 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 00:52:05 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_note_marku?= =?utf-8?b?cCAoIzE2ODA1KS4=?= Message-ID: <3fkqhj0q5cz7LjR@mail.python.org> http://hg.python.org/cpython/rev/9dc653f48357 changeset: 89614:9dc653f48357 branch: 3.3 parent: 89611:12df02358137 user: ?ric Araujo date: Wed Mar 12 19:51:00 2014 -0400 summary: Fix note markup (#16805). Patch by Tshepang Lekhonkhobe, reviewed by Georg Brandl. files: Doc/c-api/arg.rst | 1 + Doc/c-api/init.rst | 1 + Doc/faq/library.rst | 1 + Doc/library/fractions.rst | 8 ++++++-- Doc/library/getopt.rst | 1 + Doc/library/io.rst | 1 + Doc/library/logging.config.rst | 8 ++++++-- Doc/library/os.rst | 4 +++- Doc/library/pkgutil.rst | 2 ++ Doc/library/socket.rst | 1 + Doc/library/ssl.rst | 1 + Doc/library/sys.rst | 4 +++- Doc/library/test.rst | 1 + 13 files changed, 28 insertions(+), 6 deletions(-) diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -45,6 +45,7 @@ Unless otherwise stated, buffers are not NUL-terminated. .. note:: + For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of the length argument (int or :c:type:`Py_ssize_t`) is controlled by defining the macro :c:macro:`PY_SSIZE_T_CLEAN` before including diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -548,6 +548,7 @@ .. index:: module: _thread .. note:: + When only the main thread exists, no GIL operations are needed. This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit. Therefore, the lock is not diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -509,6 +509,7 @@ you can also use the :mod:`array` module. .. note:: + To read and write binary data, it is mandatory to open the file in binary mode (here, passing ``"rb"`` to :func:`open`). If you use ``"r"`` instead (the default), the file will be open in text mode diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -99,7 +99,9 @@ value of *flt*, which must be a :class:`float`. Beware that ``Fraction.from_float(0.3)`` is not the same value as ``Fraction(3, 10)`` - .. note:: From Python 3.2 onwards, you can also construct a + .. note:: + + From Python 3.2 onwards, you can also construct a :class:`Fraction` instance directly from a :class:`float`. @@ -108,7 +110,9 @@ This class method constructs a :class:`Fraction` representing the exact value of *dec*, which must be a :class:`decimal.Decimal` instance. - .. note:: From Python 3.2 onwards, you can also construct a + .. note:: + + From Python 3.2 onwards, you can also construct a :class:`Fraction` instance directly from a :class:`decimal.Decimal` instance. diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -10,6 +10,7 @@ -------------- .. note:: + The :mod:`getopt` module is a parser for command line options whose API is designed to be familiar to users of the C :c:func:`getopt` function. Users who are unfamiliar with the C :c:func:`getopt` function or who would like to write diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -681,6 +681,7 @@ :exc:`UnsupportedOperation`. .. warning:: + :class:`BufferedRWPair` does not attempt to synchronize accesses to its underlying raw streams. You should not pass it the same object as reader and writer; use :class:`BufferedRandom` instead. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -115,7 +115,9 @@ send it to the socket as a string of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. - .. note:: Because portions of the configuration are passed through + .. note:: + + Because portions of the configuration are passed through :func:`eval`, use of this function may open its users to a security risk. While the function only binds to a socket on ``localhost``, and so does not accept connections from remote machines, there are scenarios where @@ -714,7 +716,9 @@ :class:`~logging.Formatter` can present exception tracebacks in an expanded or condensed format. -.. note:: Due to the use of :func:`eval` as described above, there are +.. note:: + + Due to the use of :func:`eval` as described above, there are potential security risks which result from using the :func:`listen` to send and receive configurations via sockets. The risks are limited to where multiple users with no mutual trust run code on the same machine; see the diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -260,7 +260,9 @@ Availability: Unix. - .. note:: On Mac OS X, :func:`getgroups` behavior differs somewhat from + .. note:: + + On Mac OS X, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a deployment target of :const:`10.5` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -146,6 +146,7 @@ *prefix* is a string to output on the front of every module name on output. .. note:: + Only works for a :term:`finder` which defines an ``iter_modules()`` method. This interface is non-standard, so the module also provides implementations for :class:`importlib.machinery.FileFinder` and @@ -184,6 +185,7 @@ walk_packages(ctypes.__path__, ctypes.__name__ + '.') .. note:: + Only works for a :term:`finder` which defines an ``iter_modules()`` method. This interface is non-standard, so the module also provides implementations for :class:`importlib.machinery.FileFinder` and diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -755,6 +755,7 @@ :keyword:`with` statement around them. .. note:: + :meth:`close()` releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call :meth:`shutdown()` diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -623,6 +623,7 @@ 'version': 3} .. note:: + To validate a certificate for a particular service, you can use the :func:`match_hostname` function. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -227,7 +227,9 @@ installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y* is the version number of Python, for example ``3.2``. - .. note:: If a :ref:`virtual environment ` is in effect, this + .. note:: + + If a :ref:`virtual environment ` is in effect, this value will be changed in ``site.py`` to point to the virtual environment. The value for the Python installation will still be available, via :data:`base_exec_prefix`. diff --git a/Doc/library/test.rst b/Doc/library/test.rst --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -199,6 +199,7 @@ test suite. .. note:: + :mod:`test.support` is not a public module. It is documented here to help Python developers write tests. The API of this module is subject to change without backwards compatibility concerns between releases. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 00:52:06 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 00:52:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy4zICgjMTY4MDUp?= Message-ID: <3fkqhk49Tsz7Lk6@mail.python.org> http://hg.python.org/cpython/rev/44e815a33ae3 changeset: 89615:44e815a33ae3 parent: 89612:0cba79667c7d parent: 89614:9dc653f48357 user: ?ric Araujo date: Wed Mar 12 19:51:50 2014 -0400 summary: Merge 3.3 (#16805) files: Doc/c-api/arg.rst | 1 + Doc/c-api/init.rst | 1 + Doc/faq/library.rst | 1 + Doc/library/fractions.rst | 8 ++++++-- Doc/library/getopt.rst | 1 + Doc/library/io.rst | 1 + Doc/library/logging.config.rst | 8 ++++++-- Doc/library/os.rst | 4 +++- Doc/library/pkgutil.rst | 2 ++ Doc/library/socket.rst | 1 + Doc/library/ssl.rst | 1 + Doc/library/sys.rst | 4 +++- Doc/library/test.rst | 1 + 13 files changed, 28 insertions(+), 6 deletions(-) diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -45,6 +45,7 @@ Unless otherwise stated, buffers are not NUL-terminated. .. note:: + For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of the length argument (int or :c:type:`Py_ssize_t`) is controlled by defining the macro :c:macro:`PY_SSIZE_T_CLEAN` before including diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -582,6 +582,7 @@ .. index:: module: _thread .. note:: + When only the main thread exists, no GIL operations are needed. This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit. Therefore, the lock is not diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -513,6 +513,7 @@ you can also use the :mod:`array` module. .. note:: + To read and write binary data, it is mandatory to open the file in binary mode (here, passing ``"rb"`` to :func:`open`). If you use ``"r"`` instead (the default), the file will be open in text mode diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -99,7 +99,9 @@ value of *flt*, which must be a :class:`float`. Beware that ``Fraction.from_float(0.3)`` is not the same value as ``Fraction(3, 10)`` - .. note:: From Python 3.2 onwards, you can also construct a + .. note:: + + From Python 3.2 onwards, you can also construct a :class:`Fraction` instance directly from a :class:`float`. @@ -108,7 +110,9 @@ This class method constructs a :class:`Fraction` representing the exact value of *dec*, which must be a :class:`decimal.Decimal` instance. - .. note:: From Python 3.2 onwards, you can also construct a + .. note:: + + From Python 3.2 onwards, you can also construct a :class:`Fraction` instance directly from a :class:`decimal.Decimal` instance. diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -10,6 +10,7 @@ -------------- .. note:: + The :mod:`getopt` module is a parser for command line options whose API is designed to be familiar to users of the C :c:func:`getopt` function. Users who are unfamiliar with the C :c:func:`getopt` function or who would like to write diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -686,6 +686,7 @@ :exc:`UnsupportedOperation`. .. warning:: + :class:`BufferedRWPair` does not attempt to synchronize accesses to its underlying raw streams. You should not pass it the same object as reader and writer; use :class:`BufferedRandom` instead. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -148,7 +148,9 @@ send it to the socket as a string of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. - .. note:: Because portions of the configuration are passed through + .. note:: + + Because portions of the configuration are passed through :func:`eval`, use of this function may open its users to a security risk. While the function only binds to a socket on ``localhost``, and so does not accept connections from remote machines, there are scenarios where @@ -752,7 +754,9 @@ :class:`~logging.Formatter` can present exception tracebacks in an expanded or condensed format. -.. note:: Due to the use of :func:`eval` as described above, there are +.. note:: + + Due to the use of :func:`eval` as described above, there are potential security risks which result from using the :func:`listen` to send and receive configurations via sockets. The risks are limited to where multiple users with no mutual trust run code on the same machine; see the diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -260,7 +260,9 @@ Availability: Unix. - .. note:: On Mac OS X, :func:`getgroups` behavior differs somewhat from + .. note:: + + On Mac OS X, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a deployment target of :const:`10.5` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -147,6 +147,7 @@ *prefix* is a string to output on the front of every module name on output. .. note:: + Only works for a :term:`finder` which defines an ``iter_modules()`` method. This interface is non-standard, so the module also provides implementations for :class:`importlib.machinery.FileFinder` and @@ -185,6 +186,7 @@ walk_packages(ctypes.__path__, ctypes.__name__ + '.') .. note:: + Only works for a :term:`finder` which defines an ``iter_modules()`` method. This interface is non-standard, so the module also provides implementations for :class:`importlib.machinery.FileFinder` and diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -799,6 +799,7 @@ :keyword:`with` statement around them. .. note:: + :meth:`close()` releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call :meth:`shutdown()` diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -818,6 +818,7 @@ 'version': 3} .. note:: + To validate a certificate for a particular service, you can use the :func:`match_hostname` function. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -227,7 +227,9 @@ installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y* is the version number of Python, for example ``3.2``. - .. note:: If a :ref:`virtual environment ` is in effect, this + .. note:: + + If a :ref:`virtual environment ` is in effect, this value will be changed in ``site.py`` to point to the virtual environment. The value for the Python installation will still be available, via :data:`base_exec_prefix`. diff --git a/Doc/library/test.rst b/Doc/library/test.rst --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -199,6 +199,7 @@ test suite. .. note:: + :mod:`test.support` is not a public module. It is documented here to help Python developers write tests. The API of this module is subject to change without backwards compatibility concerns between releases. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:20:02 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 03:20:02 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Make_distutils?= =?utf-8?q?_error_messages_more_helpful_=28=2311599=29=2E?= Message-ID: <3fktzQ3wJWz7LjN@mail.python.org> http://hg.python.org/cpython/rev/37bca30c3e19 changeset: 89616:37bca30c3e19 branch: 2.7 parent: 89613:ca43fa662b0d user: ?ric Araujo date: Wed Mar 12 22:19:39 2014 -0400 summary: Make distutils error messages more helpful (#11599). When running external programs such as a C compiler and getting an error code, distutils only prints the program name. With this change, one can get the full command line by setting the DISTUTILS_DEBUG environment variable. This should have no compatibility issues, unless there are tools that depend on the exact format of distutils debug messages. files: Doc/distutils/setupscript.rst | 9 ++- Doc/install/index.rst | 4 +- Lib/distutils/spawn.py | 63 ++++++++++++++++------ Misc/NEWS | 4 + 4 files changed, 58 insertions(+), 22 deletions(-) diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -684,6 +684,8 @@ DistributionMetadata.download_url = None +.. _debug-setup-script: + Debugging the setup script ========================== @@ -699,7 +701,8 @@ and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the -failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed -information what it is doing, and prints the full traceback in case an exception -occurs. +information about what it is doing, dump the full traceback when an exception +occurs, and print the whole command line when an external program (like a C +compiler) fails. diff --git a/Doc/install/index.rst b/Doc/install/index.rst --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -58,7 +58,9 @@ document; there will be some brief forays into using Python's interactive mode to explore your installation, but that's it. If you're looking for information on how to distribute your own Python modules so that others may use them, see -the :ref:`distutils-index` manual. +the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of +interest. + .. _inst-trivial-install: diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -12,6 +12,7 @@ import os from distutils.errors import DistutilsPlatformError, DistutilsExecError +from distutils.debug import DEBUG from distutils import log def spawn(cmd, search_path=1, verbose=0, dry_run=0): @@ -30,6 +31,9 @@ Raise DistutilsExecError if running the program fails in any way; just return on success. """ + # cmd is documented as a list, but just in case some code passes a tuple + # in, protect our %-formatting code against horrible death + cmd = list(cmd) if os.name == 'posix': _spawn_posix(cmd, search_path, dry_run=dry_run) elif os.name == 'nt': @@ -69,12 +73,16 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError, exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' failed: %s" % (cmd[0], exc[-1]) + "command %r failed: %s" % (cmd, exc[-1]) if rc != 0: # and this reflects the command running but failing + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' failed with exit status %d" % (cmd[0], rc) + "command %r failed with exit status %d" % (cmd, rc) def _spawn_os2(cmd, search_path=1, verbose=0, dry_run=0): executable = cmd[0] @@ -88,13 +96,17 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError, exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' failed: %s" % (cmd[0], exc[-1]) + "command %r failed: %s" % (cmd, exc[-1]) if rc != 0: # and this reflects the command running but failing - log.debug("command '%s' failed with exit status %d" % (cmd[0], rc)) + if not DEBUG: + cmd = executable + log.debug("command %r failed with exit status %d" % (cmd, rc)) raise DistutilsExecError, \ - "command '%s' failed with exit status %d" % (cmd[0], rc) + "command %r failed with exit status %d" % (cmd, rc) if sys.platform == 'darwin': from distutils import sysconfig @@ -105,8 +117,9 @@ log.info(' '.join(cmd)) if dry_run: return + executable = cmd[0] exec_fn = search_path and os.execvp or os.execv - exec_args = [cmd[0], cmd] + env = None if sys.platform == 'darwin': global _cfg_target, _cfg_target_split if _cfg_target is None: @@ -127,18 +140,24 @@ env = dict(os.environ, MACOSX_DEPLOYMENT_TARGET=cur_target) exec_fn = search_path and os.execvpe or os.execve - exec_args.append(env) pid = os.fork() if pid == 0: # in the child try: - exec_fn(*exec_args) + if env is None: + exec_fn(executable, cmd) + else: + exec_fn(executable, cmd, env) except OSError, e: - sys.stderr.write("unable to execute %s: %s\n" % - (cmd[0], e.strerror)) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r: %s\n" % + (cmd, e.strerror)) os._exit(1) - sys.stderr.write("unable to execute %s for unknown reasons" % cmd[0]) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r for unknown reasons" % cmd) os._exit(1) else: # in the parent # Loop until the child either exits or is terminated by a signal @@ -150,29 +169,37 @@ import errno if exc.errno == errno.EINTR: continue + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' failed: %s" % (cmd[0], exc[-1]) + "command %r failed: %s" % (cmd, exc[-1]) if os.WIFSIGNALED(status): + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' terminated by signal %d" % \ - (cmd[0], os.WTERMSIG(status)) + "command %r terminated by signal %d" % \ + (cmd, os.WTERMSIG(status)) elif os.WIFEXITED(status): exit_status = os.WEXITSTATUS(status) if exit_status == 0: return # hey, it succeeded! else: + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "command '%s' failed with exit status %d" % \ - (cmd[0], exit_status) + "command %r failed with exit status %d" % \ + (cmd, exit_status) elif os.WIFSTOPPED(status): continue else: + if not DEBUG: + cmd = executable raise DistutilsExecError, \ - "unknown error executing '%s': termination status %d" % \ - (cmd[0], status) + "unknown error executing %r: termination status %d" % \ + (cmd, status) def find_executable(executable, path=None): """Tries to find 'executable' in the directories listed in 'path'. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,10 @@ as documented. The pattern and source keyword parameters are left as deprecated aliases. +- Issue #11599: When an external command (e.g. compiler) fails, distutils now + prints out the whole command line (instead of just the command name) if the + environment variable DISTUTILS_DEBUG is set. + - Issue #4931: distutils should not produce unhelpful "error: None" messages anymore. distutils.util.grok_environment_error is kept but doc-deprecated. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:42:21 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:42:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_remove_unneces?= =?utf-8?q?sary_word_=28closes_=2319060=29?= Message-ID: <3fkvT91vpBz7LjT@mail.python.org> http://hg.python.org/cpython/rev/1d31060f8a5c changeset: 89617:1d31060f8a5c branch: 2.7 user: Benjamin Peterson date: Wed Mar 12 21:41:35 2014 -0500 summary: remove unnecessary word (closes #19060) Patch by Anastasia Filatova. files: Doc/library/subprocess.rst | 2 +- Lib/subprocess.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -12,7 +12,7 @@ The :mod:`subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to -replace several other, older modules and functions, such as:: +replace several older modules and functions:: os.system os.spawn* diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -11,7 +11,7 @@ This module allows you to spawn processes, connect to their input/output/error pipes, and obtain their return codes. This module -intends to replace several other, older modules and functions, like: +intends to replace several older modules and functions: os.system os.spawn* -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:42:22 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:42:22 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_remove_unneces?= =?utf-8?q?sary_word_=28closes_=2319060=29?= Message-ID: <3fkvTB3kZnz7LjY@mail.python.org> http://hg.python.org/cpython/rev/4d8a9d12edfa changeset: 89618:4d8a9d12edfa branch: 3.3 parent: 89614:9dc653f48357 user: Benjamin Peterson date: Wed Mar 12 21:41:35 2014 -0500 summary: remove unnecessary word (closes #19060) Patch by Anastasia Filatova. files: Doc/library/subprocess.rst | 2 +- Lib/subprocess.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -9,7 +9,7 @@ The :mod:`subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to -replace several other, older modules and functions, such as:: +replace several older modules and functions:: os.system os.spawn* diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -11,7 +11,7 @@ This module allows you to spawn processes, connect to their input/output/error pipes, and obtain their return codes. This module -intends to replace several other, older modules and functions, like: +intends to replace several older modules and functions: os.system os.spawn* -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:42:23 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:42:23 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMTkwNjAp?= Message-ID: <3fkvTC5gBFz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/6f93ab911d5d changeset: 89619:6f93ab911d5d parent: 89615:44e815a33ae3 parent: 89618:4d8a9d12edfa user: Benjamin Peterson date: Wed Mar 12 21:42:04 2014 -0500 summary: merge 3.3 (#19060) files: Doc/library/subprocess.rst | 2 +- Lib/subprocess.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -9,7 +9,7 @@ The :mod:`subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to -replace several other, older modules and functions, such as:: +replace several older modules and functions:: os.system os.spawn* diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -11,7 +11,7 @@ This module allows you to spawn processes, connect to their input/output/error pipes, and obtain their return codes. This module -intends to replace several other, older modules and functions, like: +intends to replace several older modules and functions: os.system os.spawn* -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:52:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:52:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_weaken_callbac?= =?utf-8?q?k_count_inequality_=28closes_=2320901=29?= Message-ID: <3fkvhx0d6kz7LjT@mail.python.org> http://hg.python.org/cpython/rev/1763e27a182d changeset: 89620:1763e27a182d branch: 2.7 parent: 89617:1d31060f8a5c user: Benjamin Peterson date: Wed Mar 12 21:51:52 2014 -0500 summary: weaken callback count inequality (closes #20901) files: Lib/sqlite3/test/hooks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/sqlite3/test/hooks.py b/Lib/sqlite3/test/hooks.py --- a/Lib/sqlite3/test/hooks.py +++ b/Lib/sqlite3/test/hooks.py @@ -162,7 +162,7 @@ create table bar (a, b) """) second_count = len(progress_calls) - self.assertGreater(first_count, second_count) + self.assertGreaterEqual(first_count, second_count) def CheckCancelOperation(self): """ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:52:34 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:52:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_weaken_callbac?= =?utf-8?q?k_count_inequality_=28closes_=2320901=29?= Message-ID: <3fkvhy236gz7LjT@mail.python.org> http://hg.python.org/cpython/rev/dbc9e3ed5e9f changeset: 89621:dbc9e3ed5e9f branch: 3.3 parent: 89618:4d8a9d12edfa user: Benjamin Peterson date: Wed Mar 12 21:51:52 2014 -0500 summary: weaken callback count inequality (closes #20901) files: Lib/sqlite3/test/hooks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/sqlite3/test/hooks.py b/Lib/sqlite3/test/hooks.py --- a/Lib/sqlite3/test/hooks.py +++ b/Lib/sqlite3/test/hooks.py @@ -162,7 +162,7 @@ create table bar (a, b) """) second_count = len(progress_calls) - self.assertGreater(first_count, second_count) + self.assertGreaterEqual(first_count, second_count) def CheckCancelOperation(self): """ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 03:52:35 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 13 Mar 2014 03:52:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMjA5MDEp?= Message-ID: <3fkvhz3SLjz7Ljb@mail.python.org> http://hg.python.org/cpython/rev/4d626a9df062 changeset: 89622:4d626a9df062 parent: 89619:6f93ab911d5d parent: 89621:dbc9e3ed5e9f user: Benjamin Peterson date: Wed Mar 12 21:52:19 2014 -0500 summary: merge 3.3 (#20901) files: Lib/sqlite3/test/hooks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/sqlite3/test/hooks.py b/Lib/sqlite3/test/hooks.py --- a/Lib/sqlite3/test/hooks.py +++ b/Lib/sqlite3/test/hooks.py @@ -162,7 +162,7 @@ create table bar (a, b) """) second_count = len(progress_calls) - self.assertGreater(first_count, second_count) + self.assertGreaterEqual(first_count, second_count) def CheckCancelOperation(self): """ -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Thu Mar 13 09:53:17 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 13 Mar 2014 09:53:17 +0100 Subject: [Python-checkins] Daily reference leaks (44e815a33ae3): sum=4 Message-ID: results for 44e815a33ae3 on branch "default" -------------------------------------------- test_site leaked [2, -2, 2] references, sum=2 test_site leaked [2, -2, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog3DbgH5', '-x'] From python-checkins at python.org Thu Mar 13 10:51:32 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 13 Mar 2014 10:51:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2320787=3A_asyncio_?= =?utf-8?q?doc=3A_fix_typo=2E_Patch_written_by_akira=2E?= Message-ID: <3fl50N2L6Kz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/7a42fb12a736 changeset: 89623:7a42fb12a736 user: Victor Stinner date: Thu Mar 13 10:50:01 2014 +0100 summary: Close #20787: asyncio doc: fix typo. Patch written by akira. files: Doc/library/asyncio-eventloop.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -488,7 +488,7 @@ * *stdout*: Either a file-like object representing the pipe to be connected to the subprocess's standard output stream using - :meth:`~BaseEventLoop.connect_write_pipe`, or the constant + :meth:`~BaseEventLoop.connect_read_pipe`, or the constant :const:`subprocess.PIPE` (the default). By default a new pipe will be created and connected. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 10:54:32 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 13 Mar 2014 10:54:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2320889=3A_asyncio_?= =?utf-8?q?doc=3A_Document_acquire=28=29=2C_locked=28=29_and_release=28=29?= =?utf-8?q?_method_of?= Message-ID: <3fl53r5KQ0z7LjV@mail.python.org> http://hg.python.org/cpython/rev/f22e1d89486d changeset: 89624:f22e1d89486d user: Victor Stinner date: Thu Mar 13 10:54:18 2014 +0100 summary: Close #20889: asyncio doc: Document acquire(), locked() and release() method of Condition files: Doc/library/asyncio-sync.rst | 27 +++++++++++++++++++++++- 1 files changed, 26 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -64,7 +64,7 @@ .. method:: locked() - Return ``True`` if lock is acquired. + Return ``True`` if the lock is acquired. .. method:: acquire() @@ -141,6 +141,15 @@ A new :class:`Lock` object is created and used as the underlying lock. + .. method:: acquire() + + Acquire the underlying lock. + + This method blocks until the lock is unlocked, then sets it to locked and + returns ``True``. + + This method is a :ref:`coroutine `. + .. method:: notify(n=1) By default, wake up one coroutine waiting on this condition, if any. @@ -156,6 +165,10 @@ call until it can reacquire the lock. Since :meth:`notify` does not release the lock, its caller should. + .. method:: locked() + + Return ``True`` if the underlying lock is acquired. + .. method:: notify_all() Wake up all threads waiting on this condition. This method acts like @@ -163,6 +176,18 @@ calling thread has not acquired the lock when this method is called, a :exc:`RuntimeError` is raised. + .. method:: release() + + Release the underlying lock. + + When the lock is locked, reset it to unlocked, and return. If any other + coroutines are blocked waiting for the lock to become unlocked, allow + exactly one of them to proceed. + + When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. + + There is no return value. + .. method:: wait() Wait until notified. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 10:58:12 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 13 Mar 2014 10:58:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio_doc=3A_functions_a?= =?utf-8?q?re_coroutine=2C_they_don=27t_return_a_coroutine?= Message-ID: <3fl5840pt2z7LjZ@mail.python.org> http://hg.python.org/cpython/rev/077dce5c4196 changeset: 89625:077dce5c4196 user: Victor Stinner date: Thu Mar 13 10:58:03 2014 +0100 summary: asyncio doc: functions are coroutine, they don't return a coroutine It's not exact, but easier to understand. files: Doc/library/asyncio-protocol.rst | 2 +- Doc/library/asyncio-subprocess.rst | 4 ++-- Doc/library/asyncio-sync.rst | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -497,6 +497,6 @@ :meth:`Transport.close` can be called immediately after :meth:`WriteTransport.write` even if data are not sent yet on the socket: both methods are asynchronous. ``yield from`` is not needed because these transport -methods don't return coroutines. +methods are not coroutines. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -11,13 +11,13 @@ Run the shell command *cmd* given as a string. Return a :class:`~asyncio.subprocess.Process` instance. - This function returns a :ref:`coroutine object `. + This function is a :ref:`coroutine `. .. function:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) Create a subprocess. Return a :class:`~asyncio.subprocess.Process` instance. - This function returns a :ref:`coroutine object `. + This function is a :ref:`coroutine `. Use the :meth:`BaseEventLoop.connect_read_pipe` and :meth:`BaseEventLoop.connect_write_pipe` methods to connect pipes. diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -73,7 +73,7 @@ This method blocks until the lock is unlocked, then sets it to locked and returns ``True``. - This method returns a :ref:`coroutine object `. + This method is a :ref:`coroutine `. .. method:: release() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 11:02:29 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 11:02:29 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Make_distutils?= =?utf-8?q?_error_messages_more_helpful_=28=2311599=29=2E?= Message-ID: <3fl5F139lZz7LjV@mail.python.org> http://hg.python.org/cpython/rev/43b06352221b changeset: 89626:43b06352221b branch: 3.3 parent: 89621:dbc9e3ed5e9f user: ?ric Araujo date: Thu Mar 13 04:55:35 2014 -0400 summary: Make distutils error messages more helpful (#11599). When running external programs such as a C compiler and getting an error code, distutils only prints the program name. With this change, one can get the full command line by setting the DISTUTILS_DEBUG environment variable. This should have no compatibility issues, unless there are tools that depend on the exact format of distutils debug messages. files: Doc/distutils/setupscript.rst | 9 ++- Doc/install/index.rst | 3 +- Lib/distutils/spawn.py | 63 ++++++++++++++++------ Misc/NEWS | 4 + 4 files changed, 57 insertions(+), 22 deletions(-) diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -685,6 +685,8 @@ DistributionMetadata.download_url = None +.. _debug-setup-script: + Debugging the setup script ========================== @@ -700,7 +702,8 @@ and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the -failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed -information what it is doing, and prints the full traceback in case an exception -occurs. +information about what it is doing, dump the full traceback when an exception +occurs, and print the whole command line when an external program (like a C +compiler) fails. diff --git a/Doc/install/index.rst b/Doc/install/index.rst --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -58,7 +58,8 @@ document; there will be some brief forays into using Python's interactive mode to explore your installation, but that's it. If you're looking for information on how to distribute your own Python modules so that others may use them, see -the :ref:`distutils-index` manual. +the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of +interest. .. _inst-trivial-install: diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -10,6 +10,7 @@ import os from distutils.errors import DistutilsPlatformError, DistutilsExecError +from distutils.debug import DEBUG from distutils import log def spawn(cmd, search_path=1, verbose=0, dry_run=0): @@ -28,6 +29,9 @@ Raise DistutilsExecError if running the program fails in any way; just return on success. """ + # cmd is documented as a list, but just in case some code passes a tuple + # in, protect our %-formatting code against horrible death + cmd = list(cmd) if os.name == 'posix': _spawn_posix(cmd, search_path, dry_run=dry_run) elif os.name == 'nt': @@ -67,12 +71,16 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError as exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if rc != 0: # and this reflects the command running but failing + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" % (cmd[0], rc)) + "command %r failed with exit status %d" % (cmd, rc)) def _spawn_os2(cmd, search_path=1, verbose=0, dry_run=0): executable = cmd[0] @@ -86,13 +94,17 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError as exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if rc != 0: # and this reflects the command running but failing - log.debug("command '%s' failed with exit status %d" % (cmd[0], rc)) + if not DEBUG: + cmd = executable + log.debug("command %r failed with exit status %d" % (cmd, rc)) raise DistutilsExecError( - "command '%s' failed with exit status %d" % (cmd[0], rc)) + "command %r failed with exit status %d" % (cmd, rc)) if sys.platform == 'darwin': from distutils import sysconfig @@ -103,8 +115,9 @@ log.info(' '.join(cmd)) if dry_run: return + executable = cmd[0] exec_fn = search_path and os.execvp or os.execv - exec_args = [cmd[0], cmd] + env = None if sys.platform == 'darwin': global _cfg_target, _cfg_target_split if _cfg_target is None: @@ -125,17 +138,23 @@ env = dict(os.environ, MACOSX_DEPLOYMENT_TARGET=cur_target) exec_fn = search_path and os.execvpe or os.execve - exec_args.append(env) pid = os.fork() if pid == 0: # in the child try: - exec_fn(*exec_args) + if env is None: + exec_fn(executable, cmd) + else: + exec_fn(executable, cmd, env) except OSError as e: - sys.stderr.write("unable to execute %s: %s\n" - % (cmd[0], e.strerror)) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r: %s\n" + % (cmd, e.strerror)) os._exit(1) - sys.stderr.write("unable to execute %s for unknown reasons" % cmd[0]) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r for unknown reasons" % cmd) os._exit(1) else: # in the parent # Loop until the child either exits or is terminated by a signal @@ -147,26 +166,34 @@ import errno if exc.errno == errno.EINTR: continue + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if os.WIFSIGNALED(status): + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' terminated by signal %d" - % (cmd[0], os.WTERMSIG(status))) + "command %r terminated by signal %d" + % (cmd, os.WTERMSIG(status))) elif os.WIFEXITED(status): exit_status = os.WEXITSTATUS(status) if exit_status == 0: return # hey, it succeeded! else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" - % (cmd[0], exit_status)) + "command %r failed with exit status %d" + % (cmd, exit_status)) elif os.WIFSTOPPED(status): continue else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "unknown error executing '%s': termination status %d" - % (cmd[0], status)) + "unknown error executing %r: termination status %d" + % (cmd, status)) def find_executable(executable, path=None): """Tries to find 'executable' in the directories listed in 'path'. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -16,6 +16,10 @@ - Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. +- Issue #11599: When an external command (e.g. compiler) fails, distutils now + prints out the whole command line (instead of just the command name) if the + environment variable DISTUTILS_DEBUG is set. + - Issue #4931: distutils should not produce unhelpful "error: None" messages anymore. distutils.util.grok_environment_error is kept but doc-deprecated. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 11:02:30 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 11:02:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy4zICgjMTE1OTkp?= Message-ID: <3fl5F26FWWz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/c04260b59e8c changeset: 89627:c04260b59e8c parent: 89624:f22e1d89486d parent: 89626:43b06352221b user: ?ric Araujo date: Thu Mar 13 05:59:31 2014 -0400 summary: Merge 3.3 (#11599) files: Doc/distutils/setupscript.rst | 9 ++- Doc/install/index.rst | 3 +- Lib/distutils/spawn.py | 55 ++++++++++++++++------ Misc/NEWS | 4 + 4 files changed, 52 insertions(+), 19 deletions(-) diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -685,6 +685,8 @@ DistributionMetadata.download_url = None +.. _debug-setup-script: + Debugging the setup script ========================== @@ -700,7 +702,8 @@ and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the -failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed -information what it is doing, and prints the full traceback in case an exception -occurs. +information about what it is doing, dump the full traceback when an exception +occurs, and print the whole command line when an external program (like a C +compiler) fails. diff --git a/Doc/install/index.rst b/Doc/install/index.rst --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -58,7 +58,8 @@ document; there will be some brief forays into using Python's interactive mode to explore your installation, but that's it. If you're looking for information on how to distribute your own Python modules so that others may use them, see -the :ref:`distutils-index` manual. +the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of +interest. .. _inst-trivial-install: diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -10,6 +10,7 @@ import os from distutils.errors import DistutilsPlatformError, DistutilsExecError +from distutils.debug import DEBUG from distutils import log def spawn(cmd, search_path=1, verbose=0, dry_run=0): @@ -28,10 +29,15 @@ Raise DistutilsExecError if running the program fails in any way; just return on success. """ + # cmd is documented as a list, but just in case some code passes a tuple + # in, protect our %-formatting code against horrible death + cmd = list(cmd) if os.name == 'posix': _spawn_posix(cmd, search_path, dry_run=dry_run) elif os.name == 'nt': _spawn_nt(cmd, search_path, dry_run=dry_run) + elif os.name == 'os2': + _spawn_os2(cmd, search_path, dry_run=dry_run) else: raise DistutilsPlatformError( "don't know how to spawn programs on platform '%s'" % os.name) @@ -65,12 +71,16 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError as exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if rc != 0: # and this reflects the command running but failing + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" % (cmd[0], rc)) + "command %r failed with exit status %d" % (cmd, rc)) if sys.platform == 'darwin': from distutils import sysconfig @@ -81,8 +91,9 @@ log.info(' '.join(cmd)) if dry_run: return + executable = cmd[0] exec_fn = search_path and os.execvp or os.execv - exec_args = [cmd[0], cmd] + env = None if sys.platform == 'darwin': global _cfg_target, _cfg_target_split if _cfg_target is None: @@ -103,17 +114,23 @@ env = dict(os.environ, MACOSX_DEPLOYMENT_TARGET=cur_target) exec_fn = search_path and os.execvpe or os.execve - exec_args.append(env) pid = os.fork() if pid == 0: # in the child try: - exec_fn(*exec_args) + if env is None: + exec_fn(executable, cmd) + else: + exec_fn(executable, cmd, env) except OSError as e: - sys.stderr.write("unable to execute %s: %s\n" - % (cmd[0], e.strerror)) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r: %s\n" + % (cmd, e.strerror)) os._exit(1) - sys.stderr.write("unable to execute %s for unknown reasons" % cmd[0]) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r for unknown reasons" % cmd) os._exit(1) else: # in the parent # Loop until the child either exits or is terminated by a signal @@ -125,26 +142,34 @@ import errno if exc.errno == errno.EINTR: continue + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if os.WIFSIGNALED(status): + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' terminated by signal %d" - % (cmd[0], os.WTERMSIG(status))) + "command %r terminated by signal %d" + % (cmd, os.WTERMSIG(status))) elif os.WIFEXITED(status): exit_status = os.WEXITSTATUS(status) if exit_status == 0: return # hey, it succeeded! else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" - % (cmd[0], exit_status)) + "command %r failed with exit status %d" + % (cmd, exit_status)) elif os.WIFSTOPPED(status): continue else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "unknown error executing '%s': termination status %d" - % (cmd[0], status)) + "unknown error executing %r: termination status %d" + % (cmd, status)) def find_executable(executable, path=None): """Tries to find 'executable' in the directories listed in 'path'. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -23,6 +23,10 @@ - Issue #19157: Include the broadcast address in the usuable hosts for IPv6 in ipaddress. +- Issue #11599: When an external command (e.g. compiler) fails, distutils now + prints out the whole command line (instead of just the command name) if the + environment variable DISTUTILS_DEBUG is set. + - Issue #4931: distutils should not produce unhelpful "error: None" messages anymore. distutils.util.grok_environment_error is kept but doc-deprecated. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 11:02:32 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 11:02:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_heads?= Message-ID: <3fl5F42hGyz7Lk5@mail.python.org> http://hg.python.org/cpython/rev/69d503b12432 changeset: 89628:69d503b12432 parent: 89625:077dce5c4196 parent: 89627:c04260b59e8c user: ?ric Araujo date: Thu Mar 13 06:02:15 2014 -0400 summary: Merge heads files: Doc/distutils/setupscript.rst | 9 ++- Doc/install/index.rst | 3 +- Lib/distutils/spawn.py | 55 ++++++++++++++++------ Misc/NEWS | 4 + 4 files changed, 52 insertions(+), 19 deletions(-) diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -685,6 +685,8 @@ DistributionMetadata.download_url = None +.. _debug-setup-script: + Debugging the setup script ========================== @@ -700,7 +702,8 @@ and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the -failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed -information what it is doing, and prints the full traceback in case an exception -occurs. +information about what it is doing, dump the full traceback when an exception +occurs, and print the whole command line when an external program (like a C +compiler) fails. diff --git a/Doc/install/index.rst b/Doc/install/index.rst --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -58,7 +58,8 @@ document; there will be some brief forays into using Python's interactive mode to explore your installation, but that's it. If you're looking for information on how to distribute your own Python modules so that others may use them, see -the :ref:`distutils-index` manual. +the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of +interest. .. _inst-trivial-install: diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -10,6 +10,7 @@ import os from distutils.errors import DistutilsPlatformError, DistutilsExecError +from distutils.debug import DEBUG from distutils import log def spawn(cmd, search_path=1, verbose=0, dry_run=0): @@ -28,10 +29,15 @@ Raise DistutilsExecError if running the program fails in any way; just return on success. """ + # cmd is documented as a list, but just in case some code passes a tuple + # in, protect our %-formatting code against horrible death + cmd = list(cmd) if os.name == 'posix': _spawn_posix(cmd, search_path, dry_run=dry_run) elif os.name == 'nt': _spawn_nt(cmd, search_path, dry_run=dry_run) + elif os.name == 'os2': + _spawn_os2(cmd, search_path, dry_run=dry_run) else: raise DistutilsPlatformError( "don't know how to spawn programs on platform '%s'" % os.name) @@ -65,12 +71,16 @@ rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError as exc: # this seems to happen when the command isn't found + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if rc != 0: # and this reflects the command running but failing + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" % (cmd[0], rc)) + "command %r failed with exit status %d" % (cmd, rc)) if sys.platform == 'darwin': from distutils import sysconfig @@ -81,8 +91,9 @@ log.info(' '.join(cmd)) if dry_run: return + executable = cmd[0] exec_fn = search_path and os.execvp or os.execv - exec_args = [cmd[0], cmd] + env = None if sys.platform == 'darwin': global _cfg_target, _cfg_target_split if _cfg_target is None: @@ -103,17 +114,23 @@ env = dict(os.environ, MACOSX_DEPLOYMENT_TARGET=cur_target) exec_fn = search_path and os.execvpe or os.execve - exec_args.append(env) pid = os.fork() if pid == 0: # in the child try: - exec_fn(*exec_args) + if env is None: + exec_fn(executable, cmd) + else: + exec_fn(executable, cmd, env) except OSError as e: - sys.stderr.write("unable to execute %s: %s\n" - % (cmd[0], e.strerror)) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r: %s\n" + % (cmd, e.strerror)) os._exit(1) - sys.stderr.write("unable to execute %s for unknown reasons" % cmd[0]) + if not DEBUG: + cmd = executable + sys.stderr.write("unable to execute %r for unknown reasons" % cmd) os._exit(1) else: # in the parent # Loop until the child either exits or is terminated by a signal @@ -125,26 +142,34 @@ import errno if exc.errno == errno.EINTR: continue + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed: %s" % (cmd[0], exc.args[-1])) + "command %r failed: %s" % (cmd, exc.args[-1])) if os.WIFSIGNALED(status): + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' terminated by signal %d" - % (cmd[0], os.WTERMSIG(status))) + "command %r terminated by signal %d" + % (cmd, os.WTERMSIG(status))) elif os.WIFEXITED(status): exit_status = os.WEXITSTATUS(status) if exit_status == 0: return # hey, it succeeded! else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "command '%s' failed with exit status %d" - % (cmd[0], exit_status)) + "command %r failed with exit status %d" + % (cmd, exit_status)) elif os.WIFSTOPPED(status): continue else: + if not DEBUG: + cmd = executable raise DistutilsExecError( - "unknown error executing '%s': termination status %d" - % (cmd[0], status)) + "unknown error executing %r: termination status %d" + % (cmd, status)) def find_executable(executable, path=None): """Tries to find 'executable' in the directories listed in 'path'. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -23,6 +23,10 @@ - Issue #19157: Include the broadcast address in the usuable hosts for IPv6 in ipaddress. +- Issue #11599: When an external command (e.g. compiler) fails, distutils now + prints out the whole command line (instead of just the command name) if the + environment variable DISTUTILS_DEBUG is set. + - Issue #4931: distutils should not produce unhelpful "error: None" messages anymore. distutils.util.grok_environment_error is kept but doc-deprecated. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 13:14:06 2014 From: python-checkins at python.org (nick.coghlan) Date: Thu, 13 Mar 2014 13:14:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2319407=3A_New_inst?= =?utf-8?q?allation_=26_distribution_guides?= Message-ID: <3fl88t0kQHz7LjY@mail.python.org> http://hg.python.org/cpython/rev/d22ef969cb82 changeset: 89629:d22ef969cb82 user: Nick Coghlan date: Thu Mar 13 22:13:45 2014 +1000 summary: Close #19407: New installation & distribution guides - based on pip and other PyPA tools - includes references to the new Python Packaging User Guide where appropriate (and the relevant section is at least partially filled in) - started new FAQ sections - both guides aim to introduce users to basic open source concepts if they aren't aware of them - existing guides have been relocated (now linked from the distutils docs) rather then removed, since there is some needed material that has yet to be relocated to the distutils docs as a reference for the legacy formats files: Doc/conf.py | 4 +- Doc/contents.rst | 12 +- Doc/distributing/index.rst | 143 +++++++++ Doc/distutils/index.rst | 6 +- Doc/extending/index.rst | 42 ++- Doc/install/index.rst | 6 +- Doc/installing/index.rst | 211 ++++++++++++++ Doc/library/distutils.rst | 19 +- Doc/library/ensurepip.rst | 2 +- Doc/tools/sphinxext/indexcontent.html | 8 +- Doc/tutorial/whatnow.rst | 4 +- Doc/using/venv-create.inc | 5 + Doc/using/windows.rst | 4 + Doc/whatsnew/3.4.rst | 19 + Misc/NEWS | 6 + 15 files changed, 459 insertions(+), 32 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -120,11 +120,11 @@ latex_documents = [ ('c-api/index', 'c-api.tex', 'The Python/C API', _stdauthor, 'manual'), - ('distutils/index', 'distutils.tex', + ('distributing/index', 'distributing.tex', 'Distributing Python Modules', _stdauthor, 'manual'), ('extending/index', 'extending.tex', 'Extending and Embedding Python', _stdauthor, 'manual'), - ('install/index', 'install.tex', + ('installing/index', 'installing.tex', 'Installing Python Modules', _stdauthor, 'manual'), ('library/index', 'library.tex', 'The Python Library Reference', _stdauthor, 'manual'), diff --git a/Doc/contents.rst b/Doc/contents.rst --- a/Doc/contents.rst +++ b/Doc/contents.rst @@ -11,8 +11,8 @@ library/index.rst extending/index.rst c-api/index.rst - distutils/index.rst - install/index.rst + distributing/index.rst + installing/index.rst howto/index.rst faq/index.rst glossary.rst @@ -21,3 +21,11 @@ bugs.rst copyright.rst license.rst + +.. include legacy packaging docs in build + +.. toctree:: + :hidden: + + distutils/index.rst + install/index.rst diff --git a/Doc/distributing/index.rst b/Doc/distributing/index.rst new file mode 100644 --- /dev/null +++ b/Doc/distributing/index.rst @@ -0,0 +1,143 @@ +.. _distributing-index: + +############################### + Distributing Python Modules +############################### + +:Email: distutils-sig at python.org + + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their software +available for other Python developers to use under open source license terms. + +This allows Python users to share and collaborate effectively, benefiting +from the solutions others have already created to common (and sometimes +even rare!) problems, as well as potentially contributing their own +solutions to the common pool. + +This guide covers the distribution part of the process. For a guide to +installing other Python projects, refer to the +:ref:`installation guide `. + +.. note:: + + For corporate and other institutional users, be aware that many + organisations have their own policies around using and contributing to + open source software. Please take such policies into account when making + use of the distribution and installation tools provided with Python. + + +Key terms +========= + +* the `Python Package Index `__ is a public + repository of open source licensed packages made available for use by + other Python users +* the `Python Packaging Authority + `__ are the group of + developers and documentation authors responsible for the maintenance and + evolution of the standard packaging tools and the associated metadata and + file format standards. They maintain a variety of tools, documentation + and issue trackers on both `GitHub ` and + `BitBucket `__. +* ``distutils`` is the original build and distribution system first added to + the Python standard library in 1998. While direct use of ``distutils`` is + being phased out, it still laid the foundation for the current packaging + and distribution infrastructure, and it not only remains part of the + standard library, but its name lives on in other ways (such as the name + of the mailing list used to coordinate Python packaging standards + development). + + +Open source licensing and collaboration +======================================= + +In most parts of the world, software is automatically covered by copyright. +This means that other developers require explicit permission to copy, use, +modify and redistribute the software. + +Open source licensing is a way of explicitly granting such permission in a +relatively consistent way, allowing developers to share and collaborate +efficiently by making common solutions to various problems freely available. +This leaves many developers free to spend more time focusing on the problems +that are relatively unique to their specific situation. + +The distribution tools provided with Python are designed to make it +reasonably straightforward for developers to make their own contributions +back to that common pool of software if they choose to do so. + +The same distribution tools can also be used to distribute software within +an organisation, regardless of whether that software is published as open +source software or not. + + +Installing the tools +==================== + +The standard library does not include build tools that support modern +Python packaging standards, as the core development team has found that it +is important to have standard tools that work consistently, even on older +versions of Python. + +The currently recommended build and distribution tools can be installed +using ``pip``:: + + pip install setuptools wheel twine + + +Reading the guide +================= + +The Python Packaging User Guide covers the various key steps and elements +involved in creating a project + +* `Project structure`_ +* `Building and packaging the project`_ +* `Uploading the project to the Python Package Index`_ + +.. _Project structure: \ + http://packaging.python.org/en/latest/tutorial.html#creating-your-own-project +.. _Building and packaging the project: \ + http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project +.. _Uploading the project to the Python Package Index: \ + http://packaging.python.org/en/latest/tutorial.html#building-uploading-your-project-to-pypi + + +How do I...? +============ + +These are quick answers or links for some common tasks. + +... choose a name for my project? +--------------------------------- + +This isn't an easy topic, but here are a few tips: + +* check the Python Package Index to see if the name is already in use +* check popular hosting sites like GitHub, BitBucket, etc to see if there + is already a project with that name +* check what comes up in a web search for the name you're considering +* avoid particularly common words, especially ones with multiple meanings, + as they can make it difficult for users to find your software when + searching for it + + +... create and distribute binary extensions? +-------------------------------------------- + +This is actually quite a complex topic, with a variety of alternatives +available depending on exactly what you're aiming to achieve. See the +Python Packaging User Guide for more information and recommendations. + +.. seealso:: + + `Python Packaging User Guide: Binary Extensions + `__ + +.. other topics: + + Once the Development & Deployment part of PPUG is fleshed out, some of + those sections should be linked from new questions here (most notably, + we should have a question about avoiding depending on PyPI that links to + http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches) diff --git a/Doc/distutils/index.rst b/Doc/distutils/index.rst --- a/Doc/distutils/index.rst +++ b/Doc/distutils/index.rst @@ -1,8 +1,8 @@ .. _distutils-index: -############################### - Distributing Python Modules -############################### +############################################## + Distributing Python Modules (Legacy version) +############################################## :Authors: Greg Ward, Anthony Baxter :Email: distutils-sig at python.org diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -21,14 +21,31 @@ For a detailed description of the whole Python/C API, see the separate :ref:`c-api-index`. -.. note:: - This guide only covers the basic tools for creating extensions provided - as part of this version of CPython. Third party tools may offer simpler - alternatives. Refer to the `binary extensions section - `__ - in the Python Packaging User Guide for more information. +Recommended third party tools +============================= +This guide only covers the basic tools for creating extensions provided +as part of this version of CPython. Third party tools like Cython, +``cffi``, SWIG and Numba offer both simpler and more sophisticated +approaches to creating C and C++ extensions for Python. + +.. seealso:: + + `Python Packaging User Guide: Binary Extensions `_ + The Python Packaging User Guide not only covers several available + tools that simplify the creation of binary extensions, but also + discusses the various reasons why creating an extension module may be + desirable in the first place. + + +Creating extensions without third party tools +============================================= + +This section of the guide covers creating C and C++ extensions without +assistance from third party tools. It is intended primarily for creators +of those tools, rather than being a recommended way to create your own +C extensions. .. toctree:: :maxdepth: 2 @@ -38,4 +55,17 @@ newtypes.rst building.rst windows.rst + +Embedding the CPython runtime in a larger application +===================================================== + +Sometimes, rather than creating an extension that runs inside the Python +interpreter as the main application, it is desirable to instead embed +the CPython runtime inside a larger application. This section covers +some of the details involved in doing that successfully. + +.. toctree:: + :maxdepth: 2 + :numbered: + embedding.rst diff --git a/Doc/install/index.rst b/Doc/install/index.rst --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -2,9 +2,9 @@ .. _install-index: -***************************** - Installing Python Modules -***************************** +******************************************** + Installing Python Modules (Legacy version) +******************************************** :Author: Greg Ward diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst new file mode 100644 --- /dev/null +++ b/Doc/installing/index.rst @@ -0,0 +1,211 @@ +.. highlightlang:: none + +.. _installing-index: + +***************************** + Installing Python Modules +***************************** + +:Email: distutils-sig at python.org + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their software +available for other Python developers to use under open source license terms. + +This allows Python users to share and collaborate effectively, benefiting +from the solutions others have already created to common (and sometimes +even rare!) problems, as well as potentially contributing their own +solutions to the common pool. + +This guide covers the installation part of the process. For a guide to +creating and sharing your own Python projects, refer to the +:ref:`distribution guide `. + +.. note:: + + For corporate and other institutional users, be aware that many + organisations have their own policies around using and contributing to + open source software. Please take such policies into account when making + use of the distribution and installation tools provided with Python. + + +Key terms +========= + +* ``pip`` is the preferred installer program. Starting with Python 3.4, it + is included by default with the Python binary installers. + installed into virtual environments +* a virtual environment is a semi-isolated Python environment that allows + packages to be installed for use by a particular application, rather than + being installed system wide +* ``pyvenv`` is the standard tool for creating virtual environments, and has + been part of Python since Python 3.3. Starting with Python 3.4, it + defaults to installing ``pip`` into all created virtual environments +* the `Python Package Index `__ is a public + repository of open source licensed packages made available for use by + other Python users +* the `Python Packaging Authority + `__ are the group of + developers and documentation authors responsible for the maintenance and + evolution of the standard packaging tools and the associated metadata and + file format standards. They maintain a variety of tools, documentation + and issue trackers on both `GitHub ` and + `BitBucket `__. +* ``distutils`` is the original build and distribution system first added to + the Python standard library in 1998. While direct use of ``distutils`` is + being phased out, it still laid the foundation for the current packaging + and distribution infrastructure, and it not only remains part of the + standard library, but its name lives on in other ways (such as the name + of the mailing list used to coordinate Python packaging standards + development). + + +Basic usage +=========== + +The standard packaging tools are all designed to be used from the command +line. For Windows users, the examples below assume that the option to +adjust the system PATH environment variable was selected when installing +Python. For Linux users, the command to install into the system version of +Python 3 is likely to be ``pip3`` rather than ``pip``. + +The following command will install the latest version of a module and its +dependencies from the Python Package Index:: + + pip install SomePackage + +It's also possible to specify an exact or minimum version directly on the +command line:: + + pip install SomePackage==1.0.4 # specific version + pip install 'SomePackage>=1.0.4' # minimum version + +Normally, if a suitable module is already installed, attempting to install +it again will have no effect. Upgrading existing modules must be requested +explicitly:: + + pip install --upgrade SomePackage + +More information and resources regarding ``pip`` and its capabilities can be +found in the `Python Packaging User Guide `__. + +``pyvenv`` has its own documentation at :ref:`scripts-pyvenv`. Installing +into an active virtual environment uses the commands shown above. + +.. seealso:: + + `Python Packaging User Guide: Installing Python packages + `__ + + +How do I ...? +============= + +These are quick answers or links for some common tasks. + +... install ``pip`` in versions of Python prior to Python 3.4? +-------------------------------------------------------------- + +Python only started bundling ``pip`` with Python 3.4. For earlier versions, +``pip`` needs to be "bootstrapped" as described in the Python Packaging +User Guide. + +.. seealso:: + + `Python Packaging User Guide: Installing the Tools + `__ + + +.. installing-per-user-installation: + +... install packages just for the current user? +----------------------------------------------- + +Passing the ``--user`` option to ``pip install`` will install a package +just for the current user, rather than for all users of the system. + + +... install scientific Python packages? +--------------------------------------- + +A number of scientific Python packages have complex binary dependencies, and +aren't currently easy to install using ``pip`` directly. At this point in +time, it will often be easier for users to install these packages by +`other means +`__ +rather than attempting to install them with ``pip``. + +.. seealso:: + + `Python Packaging User Guide: Installing Scientific Packages + `__ + + +... work with multiple versions of Python installed in parallel? +---------------------------------------------------------------- + +On Linux, Mac OS X and other POSIX systems, use the versioned Python commands +in combination with the ``-m`` switch to run the appropriate copy of +``pip``:: + + python2 -m pip install SomePackage # default Python 2 + python2.7 -m pip install SomePackage # specifically Python 2.7 + python3 -m pip install SomePackage # default Python 3 + python3.4 -m pip install SomePackage # specifically Python 3.4 + +(appropriately versioned ``pip`` commands may also be available) + +On Windows, use the ``py`` Python launcher in combination with the ``-m`` +switch:: + + py -2 -m pip install SomePackage # default Python 2 + py -2.7 -m pip install SomePackage # specifically Python 2.7 + py -3 -m pip install SomePackage # default Python 3 + py -3.4 -m pip install SomePackage # specifically Python 3.4 + +.. other questions: + + Once the Development & Deployment part of PPUG is fleshed out, some of + those sections should be linked from new questions here (most notably, + we should have a question about avoiding depending on PyPI that links to + http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches) + + +Common installation issues +========================== + +Installing into the system Python on Linux +------------------------------------------ + +On Linux systems, a Python installation will typically be included as part +of the distribution. Installing into this Python installation requires +root access to the system, and may interfere with the operation of the +system package manager and other components of the system if a component +is unexpectedly upgraded using ``pip``. + +On such systems, it is often better to use a virtual environment or a +per-user installation when installing packages with ``pip``. + + +Installing binary extensions +---------------------------- + +Python has typically relied heavily on source based distribution, with end +users being expected to compile extension modules from source as part of +the installation process. + +With the introduction of support for the binary ``wheel`` format, and the +ability to publish wheels for at least Windows and Mac OS X through the +Python Package Index, this problem is expected to diminish over time, +as users are more regularly able to install pre-built extensions rather +than needing to build them themselves. + +Some of the solutions for installing `scientific software +`__ +that is not yet available as pre-built ``wheel`` files may also help with +obtaining other binary extensions without needing to build them locally. + +.. seealso:: + + `Python Packaging User Guide: Binary Extensions + `__ diff --git a/Doc/library/distutils.rst b/Doc/library/distutils.rst --- a/Doc/library/distutils.rst +++ b/Doc/library/distutils.rst @@ -12,14 +12,15 @@ 100%-pure Python, or may be extension modules written in C, or may be collections of Python packages which include modules coded in both Python and C. +Most Python users will *not* want to use this module directly, but instead +use the cross-version tools maintained by the Python Packaging Authority. +Refer to the `Python Packaging User Guide `_ +for more information. -User documentation and API reference are provided in another document: +For the benefits of packaging tool authors and users seeking a deeper +understanding of the details of the current packaging and distribution +system, the legacy :mod:`distutils` based user documentation and API +reference remain available: -.. seealso:: - - :ref:`distutils-index` - The manual for developers and packagers of Python modules. This describes - how to prepare :mod:`distutils`\ -based packages so that they may be - easily installed into an existing Python installation. It also contains - instructions for end-users wanting to install a distutils-based package, - :ref:`install-index`. +* :ref:`install-index` +* :ref:`distutils-index` diff --git a/Doc/library/ensurepip.rst b/Doc/library/ensurepip.rst --- a/Doc/library/ensurepip.rst +++ b/Doc/library/ensurepip.rst @@ -28,7 +28,7 @@ .. seealso:: - :ref:`install-index` + :ref:`installing-index` The end user guide for installing Python packages :pep:`453`: Explicit bootstrapping of pip in Python installations diff --git a/Doc/tools/sphinxext/indexcontent.html b/Doc/tools/sphinxext/indexcontent.html --- a/Doc/tools/sphinxext/indexcontent.html +++ b/Doc/tools/sphinxext/indexcontent.html @@ -16,14 +16,14 @@ + + - - diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -21,8 +21,8 @@ and many other tasks. Skimming through the Library Reference will give you an idea of what's available. -* :ref:`install-index` explains how to install external modules written by other - Python users. +* :ref:`installing-index` explains how to install additional modules written + by other Python users. * :ref:`reference-index`: A detailed explanation of Python's syntax and semantics. It's heavy reading, but is useful as a complete guide to the diff --git a/Doc/using/venv-create.inc b/Doc/using/venv-create.inc --- a/Doc/using/venv-create.inc +++ b/Doc/using/venv-create.inc @@ -11,6 +11,11 @@ Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages`` subdirectory (on Windows, this is ``Lib\site-packages``). +.. seealso:: + + `Python Packaging User Guide: Creating and using virtual environments + `__ + .. highlight:: none On Windows, you may have to invoke the ``pyvenv`` script as follows, if you diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -11,6 +11,10 @@ This document aims to give an overview of Windows-specific behaviour you should know about when using Python on Microsoft Windows. +.. XXX (ncoghlan) + + This looks rather stale to me... + Installing Python ================= diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -180,6 +180,9 @@ PEP 453: Explicit Bootstrapping of PIP in Python Installations -------------------------------------------------------------- +Bootstrapping pip by default +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard cross-platform mechanism to bootstrap the pip installer into Python installations and virtual environments. @@ -207,6 +210,22 @@ __ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors + +Documentation changes +~~~~~~~~~~~~~~~~~~~~~ + +As part of this change, the :ref:`installing-index` and +:ref:`distributing-index` sections of the documentation have been +completely redesigned as short getting started and FAQ documents. Most +packaging documentation has now been moved out to the Python Packaging +Authority maintained `Python Packaging User Guide +`__ and the documentation of the individual +projects. + +However, as this migration is currently still incomplete, the legacy +versions of those guides remaining available as :ref:`install-index` +and :ref:`distutils-index`. + .. note:: To avoid conflicts between parallel Python 2 and Python 3 installations, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -59,6 +59,12 @@ - Issue #20765: Add missing documentation for PurePath.with_name() and PurePath.with_suffix(). +- Issue #19407: New package installation and distribution guides based on + the Python Packaging Authority tools. Existing guides have been retained + as legacy links from the distutils docs, as they still contain some + required reference material for tool developers that isn't recorded + anywhere else. + Tests ----- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 13:33:53 2014 From: python-checkins at python.org (nick.coghlan) Date: Thu, 13 Mar 2014 13:33:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319407=3A_fix_link?= Message-ID: <3fl8bj05V0z7Lk7@mail.python.org> http://hg.python.org/cpython/rev/f5be4ea5b43e changeset: 89630:f5be4ea5b43e user: Nick Coghlan date: Thu Mar 13 22:33:33 2014 +1000 summary: Issue #19407: fix link files: Doc/distributing/index.rst | 2 +- Doc/installing/index.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/distributing/index.rst b/Doc/distributing/index.rst --- a/Doc/distributing/index.rst +++ b/Doc/distributing/index.rst @@ -39,7 +39,7 @@ developers and documentation authors responsible for the maintenance and evolution of the standard packaging tools and the associated metadata and file format standards. They maintain a variety of tools, documentation - and issue trackers on both `GitHub ` and + and issue trackers on both `GitHub `__ and `BitBucket `__. * ``distutils`` is the original build and distribution system first added to the Python standard library in 1998. While direct use of ``distutils`` is diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst --- a/Doc/installing/index.rst +++ b/Doc/installing/index.rst @@ -49,7 +49,7 @@ developers and documentation authors responsible for the maintenance and evolution of the standard packaging tools and the associated metadata and file format standards. They maintain a variety of tools, documentation - and issue trackers on both `GitHub ` and + and issue trackers on both `GitHub `__ and `BitBucket `__. * ``distutils`` is the original build and distribution system first added to the Python standard library in 1998. While direct use of ``distutils`` is -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 14:54:54 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 13 Mar 2014 14:54:54 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_What=27s_New_in_Python_3?= =?utf-8?q?=2E4=3A_more_security_changes?= Message-ID: <3flBPB2vDKzMfc@mail.python.org> http://hg.python.org/cpython/rev/78186a0affd3 changeset: 89631:78186a0affd3 user: Victor Stinner date: Thu Mar 13 14:54:46 2014 +0100 summary: What's New in Python 3.4: more security changes files: Doc/whatsnew/3.4.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -143,6 +143,12 @@ (:pep:`456`). * :ref:`Make newly created file descriptors non-inheritable ` (:pep:`446`) to avoid leaking file descriptors to child processes. +* New command line option for :ref:`isolated mode `, + (:issue:`16499`). +* All modules of the standard library now support server certificate + verification including hostname matching (:func:`ssl.match_hostname`) and CRL + (Certificate Revocation list, see + :func:`ssl.SSLContext.load_verify_locations`). * A new :func:`hashlib.pbkdf2_hmac` function provides the `PKCS#5 password-based key derivation function 2 `_. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 16:04:42 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 13 Mar 2014 16:04:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_fix_unittest_s?= =?utf-8?q?ubtest_example=2E?= Message-ID: <3flCxk0ZHSz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/9c641c97db36 changeset: 89632:9c641c97db36 user: R David Murray date: Wed Mar 12 09:31:50 2014 -0400 summary: whatsnew: fix unittest subtest example. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1642,7 +1642,7 @@ class NumbersTest(unittest.TestCase): def test_even(self): for i in range(6): - with self.subTest(i=1): + with self.subTest(i=i): self.assertEqual(i % 2, 0) will result in six subtests, each identified in the unittest verbose output -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 16:11:08 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 13 Mar 2014 16:11:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_make_it_clear_?= =?utf-8?q?I=27m_the_editor=2C_not_author_of_all_text=2E?= Message-ID: <3flD5823Pjz7Llc@mail.python.org> http://hg.python.org/cpython/rev/8ca08f43eb1e changeset: 89633:8ca08f43eb1e user: R David Murray date: Thu Mar 13 11:10:01 2014 -0400 summary: whatsnew: make it clear I'm the editor, not author of all text. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2,7 +2,7 @@ What's New In Python 3.4 **************************** -:Author: R. David Murray +:Author: R. David Murray (Editor) .. Rules for maintenance: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 17:01:11 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 13 Mar 2014 17:01:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_HTTPError=2Ehe?= =?utf-8?b?YWRlcnMgKCMxNTcwMSku?= Message-ID: <3flFBv4Fz6z7Lk7@mail.python.org> http://hg.python.org/cpython/rev/361c10d06b9c changeset: 89634:361c10d06b9c user: R David Murray date: Thu Mar 13 11:33:29 2014 -0400 summary: whatsnew: HTTPError.headers (#15701). files: Doc/library/urllib.error.rst | 2 +- Doc/whatsnew/3.4.rst | 5 +++++ 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/urllib.error.rst b/Doc/library/urllib.error.rst --- a/Doc/library/urllib.error.rst +++ b/Doc/library/urllib.error.rst @@ -48,7 +48,7 @@ .. attribute:: headers - The HTTP response headers for the HTTP request that cause the + The HTTP response headers for the HTTP request that caused the :exc:`HTTPError`. .. versionadded:: 3.4 diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1627,6 +1627,11 @@ Kachayev in :issue:`16464`, Daniel Wozniak in :issue:`17485`, and Damien Brecht and Senthil Kumaran in :issue:`17272`.) +:class:`~urllib.error.HTTPError` objects now have a +:attr:`~urllib.error.HTTPError.headers` attribute that provides access to the +HTTP response headers associated with the error. (Contributed by +Berker Peksag in :issue:`15701`.) + unittest -------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 17:01:12 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 13 Mar 2014 17:01:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_collections_no?= =?utf-8?q?_longer_implicitly_imports_=27abc=27_=28=2320784=29=2E?= Message-ID: <3flFBw63s5z7Lkp@mail.python.org> http://hg.python.org/cpython/rev/d575398d1916 changeset: 89635:d575398d1916 user: R David Murray date: Thu Mar 13 12:00:17 2014 -0400 summary: whatsnew: collections no longer implicitly imports 'abc' (#20784). files: Doc/whatsnew/3.4.rst | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2436,6 +2436,13 @@ have things that look like doctests in them you may see test failures you've never seen before when running your tests (:issue:`3158`). +* The :mod:`collections.abc` module has been slightly refactored as + part of the Python startup improvements. As a consequence of this, it is no + longer the case that importing :mod:`collections` automatically imports + :mod:`collections.abc`. If your program depended on the (undocumented) + implicit import, you will need to add an explicit ``import collections.abc`` + (:issue:`20784`). + Changes in the C API -------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 17:38:40 2014 From: python-checkins at python.org (jesus.cea) Date: Thu, 13 Mar 2014 17:38:40 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMDkw?= =?utf-8?q?8=3A_Memory_leak_in_Reg2Py=28=29?= Message-ID: <3flG2826wdz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/b44a3f6676b8 changeset: 89636:b44a3f6676b8 branch: 2.7 parent: 89620:1763e27a182d user: Jesus Cea date: Thu Mar 13 17:33:43 2014 +0100 summary: Closes #20908: Memory leak in Reg2Py() files: PC/_winreg.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/PC/_winreg.c b/PC/_winreg.c --- a/PC/_winreg.c +++ b/PC/_winreg.c @@ -948,8 +948,10 @@ fixupMultiSZ(str, retDataBuf, retDataSize); obData = PyList_New(s); - if (obData == NULL) + if (obData == NULL) { + free(str); return NULL; + } for (index = 0; index < s; index++) { size_t len = _mbstrlen(str[index]); @@ -957,6 +959,7 @@ PyErr_SetString(PyExc_OverflowError, "registry string is too long for a Python string"); Py_DECREF(obData); + free(str); return NULL; } PyList_SetItem(obData, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 17:38:41 2014 From: python-checkins at python.org (jesus.cea) Date: Thu, 13 Mar 2014 17:38:41 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMyMDkw?= =?utf-8?q?8=3A_Memory_leak_in_Reg2Py=28=29?= Message-ID: <3flG295GTZz7Lk7@mail.python.org> http://hg.python.org/cpython/rev/07968254be96 changeset: 89637:07968254be96 branch: 3.3 parent: 89626:43b06352221b user: Jesus Cea date: Thu Mar 13 17:35:32 2014 +0100 summary: Closes #20908: Memory leak in Reg2Py() files: PC/winreg.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/PC/winreg.c b/PC/winreg.c --- a/PC/winreg.c +++ b/PC/winreg.c @@ -943,8 +943,10 @@ fixupMultiSZ(str, data, len); obData = PyList_New(s); - if (obData == NULL) + if (obData == NULL) { + free(str); return NULL; + } for (index = 0; index < s; index++) { size_t len = wcslen(str[index]); @@ -952,6 +954,7 @@ PyErr_SetString(PyExc_OverflowError, "registry string is too long for a Python string"); Py_DECREF(obData); + free(str); return NULL; } PyList_SetItem(obData, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 17:38:43 2014 From: python-checkins at python.org (jesus.cea) Date: Thu, 13 Mar 2014 17:38:43 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_MERGE=3A_Closes_=2320908=3A_Memory_leak_in_Reg2Py=28=29?= Message-ID: <3flG2C19gvz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/227405961789 changeset: 89638:227405961789 parent: 89635:d575398d1916 parent: 89637:07968254be96 user: Jesus Cea date: Thu Mar 13 17:38:20 2014 +0100 summary: MERGE: Closes #20908: Memory leak in Reg2Py() files: PC/winreg.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/PC/winreg.c b/PC/winreg.c --- a/PC/winreg.c +++ b/PC/winreg.c @@ -943,8 +943,10 @@ fixupMultiSZ(str, data, len); obData = PyList_New(s); - if (obData == NULL) + if (obData == NULL) { + free(str); return NULL; + } for (index = 0; index < s; index++) { size_t len = wcslen(str[index]); @@ -952,6 +954,7 @@ PyErr_SetString(PyExc_OverflowError, "registry string is too long for a Python string"); Py_DECREF(obData); + free(str); return NULL; } PyList_SetItem(obData, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 19:09:11 2014 From: python-checkins at python.org (victor.stinner) Date: Thu, 13 Mar 2014 19:09:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320908=3A_PyMem=5F?= =?utf-8?q?Malloc=28=29_must_be_used_with_PyMem=5FFree=28=29=2C_not_with_f?= =?utf-8?b?cmVlKCk=?= Message-ID: <3flJ2b2bgmz7LjT@mail.python.org> http://hg.python.org/cpython/rev/6e3e91662216 changeset: 89639:6e3e91662216 user: Victor Stinner date: Thu Mar 13 19:08:10 2014 +0100 summary: Issue #20908: PyMem_Malloc() must be used with PyMem_Free(), not with free() files: PC/winreg.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/PC/winreg.c b/PC/winreg.c --- a/PC/winreg.c +++ b/PC/winreg.c @@ -944,7 +944,7 @@ fixupMultiSZ(str, data, len); obData = PyList_New(s); if (obData == NULL) { - free(str); + PyMem_Free(str); return NULL; } for (index = 0; index < s; index++) @@ -954,7 +954,7 @@ PyErr_SetString(PyExc_OverflowError, "registry string is too long for a Python string"); Py_DECREF(obData); - free(str); + PyMem_Free(str); return NULL; } PyList_SetItem(obData, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 19:37:06 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 13 Mar 2014 19:37:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_remove_=27draf?= =?utf-8?q?t=27_note=2C_tidy_up_summary=2C_collect_SSLContext_stuff=2E?= Message-ID: <3flJfp5SyhzRdL@mail.python.org> http://hg.python.org/cpython/rev/5e27593aa215 changeset: 89640:5e27593aa215 user: R David Murray date: Thu Mar 13 14:36:09 2014 -0400 summary: whatsnew: remove 'draft' note, tidy up summary, collect SSLContext stuff. Since Victor linked to it in a block, it seems to make sense to have all the SSLContext changes next to each other. I also sorted all the SSL security enhancements next to each other in the security enhancements summary. files: Doc/whatsnew/3.4.rst | 70 +++++++++++++++---------------- 1 files changed, 33 insertions(+), 37 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -67,11 +67,6 @@ For full details, see the `changelog `_. -.. note:: Prerelease users should be aware that this document is currently in - draft form. While it should be close to complete for the Python 3.4 - release candidates, adjustments and additions to the document may be made - up until the final release. - .. seealso:: @@ -92,9 +87,9 @@ New expected features for Python implementations: * :ref:`pip should always be "available" ` (:pep:`453`). -* :ref:`Make newly created file descriptors non-inheritable ` +* :ref:`Newly created file descriptors are non-inheritable ` (:pep:`446`). -* command line option for :ref:`isolated mode `, +* command line option for :ref:`isolated mode ` (:issue:`16499`). * :ref:`improvements in the handling of codecs ` that are not text encodings (multiple issues). @@ -145,10 +140,11 @@ (:pep:`446`) to avoid leaking file descriptors to child processes. * New command line option for :ref:`isolated mode `, (:issue:`16499`). -* All modules of the standard library now support server certificate - verification including hostname matching (:func:`ssl.match_hostname`) and CRL - (Certificate Revocation list, see - :func:`ssl.SSLContext.load_verify_locations`). +* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork + on Unix `. *spawn* and *forkserver* are + more secure because they avoid sharing data with child processes. +* :mod:`multiprocessing` child processes on Windows no longer inherit + all of the parent's inheritable handles, only the necessary ones. * A new :func:`hashlib.pbkdf2_hmac` function provides the `PKCS#5 password-based key derivation function 2 `_. @@ -157,18 +153,18 @@ ` for :mod:`ssl`. * :ref:`Server-side SNI (Server Name Indication) support ` for :mod:`ssl`. -* The :class:`ssl.SSLContext` class got a :ref:`lot of improvements +* The :class:`ssl.SSLContext` class has a :ref:`lot of improvements `. -* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork - on Unix `: *spawn* and *forkserver* avoid - sharing data with child processes; child processes no longer inherit all of - the parents inheritable handles on Windows. +* All modules in the standard library that support SSL now support server + certificate verification, including hostname matching + (:func:`ssl.match_hostname`) and CRLs (Certificate Revocation lists, see + :func:`ssl.SSLContext.load_verify_locations`). CPython implementation improvements: * :ref:`Safe object finalization ` (:pep:`442`). -* Leveraging :pep:`442`, :ref:`module globals are no longer set to None - during finalization `, in most cases (:issue:`18214`). +* Leveraging :pep:`442`, in most cases :ref:`module globals are no longer set + to None during finalization ` (:issue:`18214`). * :ref:`Configurable memory allocators ` (:pep:`445`). * :ref:`Argument Clinic ` (:pep:`436`). @@ -251,8 +247,8 @@ .. _whatsnew-pep-446: -PEP 446: Make Newly Created File Descriptors Non-Inheritable ------------------------------------------------------------- +PEP 446: Newly Created File Descriptors Are Non-Inheritable +----------------------------------------------------------- :pep:`446` makes newly created file descriptors :ref:`non-inheritable `. New functions and methods: @@ -1432,23 +1428,6 @@ list of the loaded ``CA`` certificates. (Contributed by Christian Heimes in and :issue:`18147`.) -.. _whatsnew34-win-cert-store: - -Two new windows-only functions, :func:`~ssl.enum_certificates` and -:func:`~ssl.enum_crls` provide the ability to retrieve certificates, -certificate information, and CRLs from the Windows cert store. (Contributed -by Christian Heimes in :issue:`17134`.) - -.. _whatsnew34-sni: - -Support for server-side SNI (Server Name Indication) using the new -:meth:`ssl.SSLContext.set_servername_callback` method. -(Contributed by Daniel Black in :issue:`8109`.) - -The dictionary returned by :meth:`.SSLSocket.getpeercert` contains additional -``X509v3`` extension items: ``crlDistributionPoints``, ``calIssuers``, and -``OCSP`` URIs. (Contributed by Christian Heimes in :issue:`18379`.) - If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has an new attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the certificate verification process by setting it to some combination of the new @@ -1474,6 +1453,23 @@ best-practice security requirements change. (Contributed by Christian Heimes in :issue:`19689`.) +.. _whatsnew34-win-cert-store: + +Two new windows-only functions, :func:`~ssl.enum_certificates` and +:func:`~ssl.enum_crls` provide the ability to retrieve certificates, +certificate information, and CRLs from the Windows cert store. (Contributed +by Christian Heimes in :issue:`17134`.) + +.. _whatsnew34-sni: + +Support for server-side SNI (Server Name Indication) using the new +:meth:`ssl.SSLContext.set_servername_callback` method. +(Contributed by Daniel Black in :issue:`8109`.) + +The dictionary returned by :meth:`.SSLSocket.getpeercert` contains additional +``X509v3`` extension items: ``crlDistributionPoints``, ``calIssuers``, and +``OCSP`` URIs. (Contributed by Christian Heimes in :issue:`18379`.) + stat ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 20:43:19 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 13 Mar 2014 20:43:19 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319407=3A_Fix_typo?= =?utf-8?q?s_in_new_distribution_=26_installation_guides=2E?= Message-ID: <3flL7C0Prtz7LjM@mail.python.org> http://hg.python.org/cpython/rev/25dc02a2acae changeset: 89641:25dc02a2acae user: Ned Deily date: Thu Mar 13 12:42:36 2014 -0700 summary: Issue #19407: Fix typos in new distribution & installation guides. Also applies to cherry pick Issue #20909. files: Doc/distributing/index.rst | 2 +- Doc/installing/index.rst | 1 - 2 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/distributing/index.rst b/Doc/distributing/index.rst --- a/Doc/distributing/index.rst +++ b/Doc/distributing/index.rst @@ -101,7 +101,7 @@ .. _Building and packaging the project: \ http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project .. _Uploading the project to the Python Package Index: \ - http://packaging.python.org/en/latest/tutorial.html#building-uploading-your-project-to-pypi + http://packaging.python.org/en/latest/tutorial.html#uploading-your-project-to-pypi How do I...? diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst --- a/Doc/installing/index.rst +++ b/Doc/installing/index.rst @@ -34,7 +34,6 @@ * ``pip`` is the preferred installer program. Starting with Python 3.4, it is included by default with the Python binary installers. - installed into virtual environments * a virtual environment is a semi-isolated Python environment that allows packages to be installed for use by a particular application, rather than being installed system wide -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 21:37:53 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 21:37:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Clarify_distut?= =?utf-8?q?ils=E2=80=99_clean_command_=28ref_=236142=29?= Message-ID: <3flML91dCxz7LjM@mail.python.org> http://hg.python.org/cpython/rev/67ada6ab7fe2 changeset: 89642:67ada6ab7fe2 branch: 2.7 parent: 89636:b44a3f6676b8 user: ?ric Araujo date: Thu Mar 13 16:17:11 2014 -0400 summary: Clarify distutils? clean command (ref #6142) files: Doc/distutils/apiref.rst | 8 ++++++-- Doc/distutils/configfile.rst | 2 ++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1907,8 +1907,12 @@ .. module:: distutils.command.clean :synopsis: Clean a package build area - -.. % todo +This command removes the temporary files created by :command:`build` +and its subcommands, like intermediary compiled object files. With +the ``--all`` option, the complete build directory will be removed. + +Extension modules built :ref:`in place ` +will not be cleaned, as they are not in the build directory. :mod:`distutils.command.config` --- Perform package configuration diff --git a/Doc/distutils/configfile.rst b/Doc/distutils/configfile.rst --- a/Doc/distutils/configfile.rst +++ b/Doc/distutils/configfile.rst @@ -69,6 +69,8 @@ Note that an option spelled :option:`--foo-bar` on the command-line is spelled :option:`foo_bar` in configuration files. +.. _distutils-build-ext-inplace: + For example, say you want your extensions to be built "in-place"---that is, you have an extension :mod:`pkg.ext`, and you want the compiled extension file (:file:`ext.so` on Unix, say) to be put in the same source directory as your -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 23:32:44 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 23:32:44 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Clarify_distut?= =?utf-8?q?ils=E2=80=99_clean_command_=28ref_=236142=29?= Message-ID: <3flPth3jLSz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/066b896b6132 changeset: 89643:066b896b6132 branch: 3.3 parent: 89637:07968254be96 user: ?ric Araujo date: Thu Mar 13 16:17:11 2014 -0400 summary: Clarify distutils? clean command (ref #6142) files: Doc/distutils/apiref.rst | 8 ++++++-- Doc/distutils/configfile.rst | 2 ++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1934,8 +1934,12 @@ .. module:: distutils.command.clean :synopsis: Clean a package build area - -.. % todo +This command removes the temporary files created by :command:`build` +and its subcommands, like intermediary compiled object files. With +the ``--all`` option, the complete build directory will be removed. + +Extension modules built :ref:`in place ` +will not be cleaned, as they are not in the build directory. :mod:`distutils.command.config` --- Perform package configuration diff --git a/Doc/distutils/configfile.rst b/Doc/distutils/configfile.rst --- a/Doc/distutils/configfile.rst +++ b/Doc/distutils/configfile.rst @@ -69,6 +69,8 @@ Note that an option spelled :option:`--foo-bar` on the command-line is spelled :option:`foo_bar` in configuration files. +.. _distutils-build-ext-inplace: + For example, say you want your extensions to be built "in-place"---that is, you have an extension :mod:`pkg.ext`, and you want the compiled extension file (:file:`ext.so` on Unix, say) to be put in the same source directory as your -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Mar 13 23:32:45 2014 From: python-checkins at python.org (eric.araujo) Date: Thu, 13 Mar 2014 23:32:45 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Clarify_distutils=E2=80=99_clean_command_=28ref_=236142?= =?utf-8?q?=29?= Message-ID: <3flPtj5b7Nz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/50ee8fe8e841 changeset: 89644:50ee8fe8e841 parent: 89641:25dc02a2acae parent: 89643:066b896b6132 user: ?ric Araujo date: Thu Mar 13 18:32:18 2014 -0400 summary: Clarify distutils? clean command (ref #6142) files: Doc/distutils/apiref.rst | 8 ++++++-- Doc/distutils/configfile.rst | 2 ++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -1923,8 +1923,12 @@ .. module:: distutils.command.clean :synopsis: Clean a package build area - -.. % todo +This command removes the temporary files created by :command:`build` +and its subcommands, like intermediary compiled object files. With +the ``--all`` option, the complete build directory will be removed. + +Extension modules built :ref:`in place ` +will not be cleaned, as they are not in the build directory. :mod:`distutils.command.config` --- Perform package configuration diff --git a/Doc/distutils/configfile.rst b/Doc/distutils/configfile.rst --- a/Doc/distutils/configfile.rst +++ b/Doc/distutils/configfile.rst @@ -69,6 +69,8 @@ Note that an option spelled :option:`--foo-bar` on the command-line is spelled :option:`foo_bar` in configuration files. +.. _distutils-build-ext-inplace: + For example, say you want your extensions to be built "in-place"---that is, you have an extension :mod:`pkg.ext`, and you want the compiled extension file (:file:`ext.so` on Unix, say) to be put in the same source directory as your -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 00:31:11 2014 From: python-checkins at python.org (eric.araujo) Date: Fri, 14 Mar 2014 00:31:11 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_references_to_make_?= =?utf-8?q?targets_deleted_in_eef7899ea7ab?= Message-ID: <3flRB76CyBz7LjX@mail.python.org> http://hg.python.org/cpython/rev/74a6b3c9c6cd changeset: 89645:74a6b3c9c6cd user: ?ric Araujo date: Thu Mar 13 19:30:43 2014 -0400 summary: Remove references to make targets deleted in eef7899ea7ab files: Doc/Makefile | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/Makefile b/Doc/Makefile --- a/Doc/Makefile +++ b/Doc/Makefile @@ -14,7 +14,7 @@ ALLSPHINXOPTS = -b $(BUILDER) -d build/doctrees -D latex_paper_size=$(PAPER) \ $(SPHINXOPTS) . build/$(BUILDER) $(SOURCES) -.PHONY: help checkout update build html htmlhelp latex text changes linkcheck \ +.PHONY: help build html htmlhelp latex text changes linkcheck \ suspicious coverage doctest pydoc-topics htmlview clean dist check serve \ autobuild-dev autobuild-stable @@ -36,7 +36,7 @@ @echo " check to run a check for frequent markup errors" @echo " serve to serve the documentation on the localhost (8000)" -build: checkout +build: $(SPHINXBUILD) $(ALLSPHINXOPTS) @echo @@ -157,7 +157,6 @@ # for development releases: always build autobuild-dev: - make update make dist SPHINXOPTS='-A daily=1 -A versionswitcher=1' -make suspicious -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 00:31:55 2014 From: python-checkins at python.org (eric.araujo) Date: Fri, 14 Mar 2014 00:31:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Fix_typo?= Message-ID: <3flRBz089Tz7LjX@mail.python.org> http://hg.python.org/devguide/rev/1a01870ccfa7 changeset: 674:1a01870ccfa7 user: ?ric Araujo date: Thu Mar 13 19:31:20 2014 -0400 summary: Fix typo files: coverage.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/coverage.rst b/coverage.rst --- a/coverage.rst +++ b/coverage.rst @@ -133,7 +133,7 @@ But one of the strengths of coverage.py is its HTML-based reports which let you visually see what lines of code were not tested:: - ./python COVERAGDIR html -i --include=`pwd`/Lib/* --omit="Lib/test/*,Lib/*/tests/*" + ./python COVERAGEDIR html -i --include=`pwd`/Lib/* --omit="Lib/test/*,Lib/*/tests/*" This will generate an HTML report in a directory named ``htmlcov`` which ignores any errors that may arise and ignores modules for which test coverage is -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Fri Mar 14 00:31:56 2014 From: python-checkins at python.org (eric.araujo) Date: Fri, 14 Mar 2014 00:31:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Update_maintainers_entrie?= =?utf-8?q?s_for_distutils?= Message-ID: <3flRC01nZNz7LjY@mail.python.org> http://hg.python.org/devguide/rev/6f21b877f2b5 changeset: 675:6f21b877f2b5 user: ?ric Araujo date: Thu Mar 13 19:31:49 2014 -0400 summary: Update maintainers entries for distutils files: experts.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/experts.rst b/experts.rst --- a/experts.rst +++ b/experts.rst @@ -96,7 +96,7 @@ decimal facundobatista, rhettinger, mark.dickinson difflib tim.peters (inactive) dis ncoghlan* -distutils tarek, eric.araujo +distutils eric.araujo doctest tim.peters (inactive) dummy_threading brett.cannon email barry, r.david.murray* @@ -160,7 +160,6 @@ optparse aronacher os loewis ossaudiodev -packaging tarek*, eric.araujo*, alexis parser benjamin.peterson pathlib pitrou* pdb georg.brandl* -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Fri Mar 14 01:19:35 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 01:19:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_linkify_some_p?= =?utf-8?q?ip_things=2C_and_explain_why_non-inheriting_is_good=2E?= Message-ID: <3flSFz6Zkkz7LjR@mail.python.org> http://hg.python.org/cpython/rev/ee635e7d213d changeset: 89646:ee635e7d213d user: R David Murray date: Thu Mar 13 19:53:38 2014 -0400 summary: whatsnew: linkify some pip things, and explain why non-inheriting is good. files: Doc/library/venv.rst | 2 ++ Doc/using/unix.rst | 2 ++ Doc/whatsnew/3.4.rst | 27 +++++++++++++++++---------- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -76,6 +76,8 @@ without there needing to be any reference to its venv in ``PATH``. +.. _venv-api: + API --- diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst --- a/Doc/using/unix.rst +++ b/Doc/using/unix.rst @@ -60,6 +60,8 @@ prompt. +.. _building-python-on-unix: + Building Python =============== diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -193,15 +193,16 @@ X.Y stands for the version of the Python installation), along with the ``pip`` Python package and its dependencies. -The :mod:`venv` module and the :command:`pyvenv` utility make use of this -module to make ``pip`` readily available in virtual environments. When -using the command line interface, ``pip`` is installed by default, while -for the module API installation of ``pip`` must be requested explicitly. - -For CPython source builds on POSIX systems, the ``make install`` and -``make altinstall`` commands bootstrap ``pip`` by default. This behaviour -can be controlled through configure options, and overridden through -Makefile options. +The :ref:`pyvenv ` command line utility and the :mod:`venv` +module make use of the :mod:`ensurepip` module to make ``pip`` readily +available in virtual environments. When using the command line utility, ``pip`` +is installed by default, while when using the :mod:`venv` module +:ref:`venv-api` installation of ``pip`` must be requested explicitly. + +For CPython :ref:`source builds on POSIX systems `, +the ``make install`` and ``make altinstall`` commands bootstrap ``pip`` by +default. This behaviour can be controlled through configure options, and +overridden through Makefile options. On Windows and Mac OS X, the CPython installers now offer the option to install ``pip`` along with CPython itself. @@ -251,7 +252,13 @@ ----------------------------------------------------------- :pep:`446` makes newly created file descriptors :ref:`non-inheritable -`. New functions and methods: +`. In general, this is the behavior an application will +want: when launching a new process, having currently open files also +open in the new process can lead to all sorts of hard to find bugs, +and potentially to security issues. + +However, there are occasions when inheritance is desired. To support +these cases, the following new functions and methods are available: * :func:`os.get_inheritable`, :func:`os.set_inheritable` * :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 01:55:23 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 01:55:23 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzc0NzU6IFJlbW92?= =?utf-8?q?e_references_to_=27=2Etransform=27_from_transform_codec_docstri?= =?utf-8?q?ngs=2E?= Message-ID: <3flT3H6hqCz7LjY@mail.python.org> http://hg.python.org/cpython/rev/d7950e916f20 changeset: 89647:d7950e916f20 branch: 3.3 parent: 89643:066b896b6132 user: R David Murray date: Thu Mar 13 20:54:30 2014 -0400 summary: #7475: Remove references to '.transform' from transform codec docstrings. files: Lib/encodings/base64_codec.py | 3 +-- Lib/encodings/hex_codec.py | 3 +-- Lib/encodings/quopri_codec.py | 3 +-- Lib/encodings/rot_13.py | 3 +-- Lib/encodings/uu_codec.py | 3 +-- Lib/encodings/zlib_codec.py | 3 +-- 6 files changed, 6 insertions(+), 12 deletions(-) diff --git a/Lib/encodings/base64_codec.py b/Lib/encodings/base64_codec.py --- a/Lib/encodings/base64_codec.py +++ b/Lib/encodings/base64_codec.py @@ -1,7 +1,6 @@ """Python 'base64_codec' Codec - base64 content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/hex_codec.py b/Lib/encodings/hex_codec.py --- a/Lib/encodings/hex_codec.py +++ b/Lib/encodings/hex_codec.py @@ -1,7 +1,6 @@ """Python 'hex_codec' Codec - 2-digit hex content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -1,7 +1,6 @@ """Codec for quoted-printable encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. """ import codecs diff --git a/Lib/encodings/rot_13.py b/Lib/encodings/rot_13.py --- a/Lib/encodings/rot_13.py +++ b/Lib/encodings/rot_13.py @@ -1,8 +1,7 @@ #!/usr/bin/env python """ Python Character Mapping Codec for ROT13. -This codec de/encodes from str to str and is therefore usable with -str.transform() and str.untransform(). +This codec de/encodes from str to str. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/uu_codec.py b/Lib/encodings/uu_codec.py --- a/Lib/encodings/uu_codec.py +++ b/Lib/encodings/uu_codec.py @@ -1,7 +1,6 @@ """Python 'uu_codec' Codec - UU content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). Some details were adapted from uu.py which was written by Lance Ellinghouse and diff --git a/Lib/encodings/zlib_codec.py b/Lib/encodings/zlib_codec.py --- a/Lib/encodings/zlib_codec.py +++ b/Lib/encodings/zlib_codec.py @@ -1,7 +1,6 @@ """Python 'zlib_codec' Codec - zlib compression encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 01:55:25 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 01:55:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Merge_=237475=3A_Remove_references_to_=27=2Etransform=27?= =?utf-8?q?_from_transform_codec_docstrings=2E?= Message-ID: <3flT3K0s5Sz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/83d54ab5c696 changeset: 89648:83d54ab5c696 parent: 89646:ee635e7d213d parent: 89647:d7950e916f20 user: R David Murray date: Thu Mar 13 20:55:09 2014 -0400 summary: Merge #7475: Remove references to '.transform' from transform codec docstrings. files: Lib/encodings/base64_codec.py | 3 +-- Lib/encodings/hex_codec.py | 3 +-- Lib/encodings/quopri_codec.py | 3 +-- Lib/encodings/rot_13.py | 3 +-- Lib/encodings/uu_codec.py | 3 +-- Lib/encodings/zlib_codec.py | 3 +-- 6 files changed, 6 insertions(+), 12 deletions(-) diff --git a/Lib/encodings/base64_codec.py b/Lib/encodings/base64_codec.py --- a/Lib/encodings/base64_codec.py +++ b/Lib/encodings/base64_codec.py @@ -1,7 +1,6 @@ """Python 'base64_codec' Codec - base64 content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/hex_codec.py b/Lib/encodings/hex_codec.py --- a/Lib/encodings/hex_codec.py +++ b/Lib/encodings/hex_codec.py @@ -1,7 +1,6 @@ """Python 'hex_codec' Codec - 2-digit hex content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -1,7 +1,6 @@ """Codec for quoted-printable encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. """ import codecs diff --git a/Lib/encodings/rot_13.py b/Lib/encodings/rot_13.py --- a/Lib/encodings/rot_13.py +++ b/Lib/encodings/rot_13.py @@ -1,8 +1,7 @@ #!/usr/bin/env python """ Python Character Mapping Codec for ROT13. -This codec de/encodes from str to str and is therefore usable with -str.transform() and str.untransform(). +This codec de/encodes from str to str. Written by Marc-Andre Lemburg (mal at lemburg.com). """ diff --git a/Lib/encodings/uu_codec.py b/Lib/encodings/uu_codec.py --- a/Lib/encodings/uu_codec.py +++ b/Lib/encodings/uu_codec.py @@ -1,7 +1,6 @@ """Python 'uu_codec' Codec - UU content transfer encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). Some details were adapted from uu.py which was written by Lance Ellinghouse and diff --git a/Lib/encodings/zlib_codec.py b/Lib/encodings/zlib_codec.py --- a/Lib/encodings/zlib_codec.py +++ b/Lib/encodings/zlib_codec.py @@ -1,7 +1,6 @@ """Python 'zlib_codec' Codec - zlib compression encoding. -This codec de/encodes from bytes to bytes and is therefore usable with -bytes.transform() and bytes.untransform(). +This codec de/encodes from bytes to bytes. Written by Marc-Andre Lemburg (mal at lemburg.com). """ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 03:17:48 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 03:17:48 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_small_tweaks_t?= =?utf-8?q?o_codecs_entry=2E?= Message-ID: <3flVtN2JDPz7LjN@mail.python.org> http://hg.python.org/cpython/rev/6ddd2c2108f2 changeset: 89649:6ddd2c2108f2 user: R David Murray date: Thu Mar 13 20:56:27 2014 -0400 summary: whatsnew: small tweaks to codecs entry. files: Doc/whatsnew/3.4.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -289,10 +289,10 @@ but were previously only discoverable through runtime introspection. Unlike the convenience methods on :class:`str`, :class:`bytes` and -:class:`bytearray`, these convenience functions support arbitrary codecs -in both Python 2 and Python 3, rather than being limited to Unicode text -encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions -(in Python 2). +:class:`bytearray`, the :mod:`codecs` convenience functions support arbitrary +codecs in both Python 2 and Python 3, rather than being limited to Unicode text +encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions (in +Python 2). In Python 3.4, the interpreter is able to identify the known non-text encodings provided in the standard library and direct users towards these @@ -315,7 +315,7 @@ In a related change, whenever it is feasible without breaking backwards compatibility, exceptions raised during encoding and decoding operations -will be wrapped in a chained exception of the same type that mentions the +are wrapped in a chained exception of the same type that mentions the name of the codec responsible for producing the error:: >>> import codecs @@ -361,7 +361,7 @@ The binary and text transforms provided in the standard library are detailed in :ref:`binary-transforms` and :ref:`text-transforms`. -(Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`, +(Contributed by Nick Coghlan in :issue:`7475`, :issue:`17827`, :issue:`17828` and :issue:`19619`) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 03:17:49 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 03:17:49 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_difflib=2Eisbj?= =?utf-8?q?unk_=26c_were_removed=2C_not_deprecated=2E?= Message-ID: <3flVtP3ndtz7LjN@mail.python.org> http://hg.python.org/cpython/rev/46d2c9021ee9 changeset: 89650:46d2c9021ee9 user: R David Murray date: Thu Mar 13 21:34:54 2014 -0400 summary: whatsnew: difflib.isbjunk &c were removed, not deprecated. Also move NEWS item to correct position (it was in 3.3). files: Doc/whatsnew/3.4.rst | 10 ++++++---- Misc/NEWS | 5 +++-- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2051,10 +2051,6 @@ Deprecations in the Python API ------------------------------ -* :meth:`difflib.SequenceMatcher.isbjunk` and - :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and - ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object. - * As mentioned in :ref:`whatsnew-pep-451`, a number of :mod:`importilb` methods and functions are deprecated: :meth:`importlib.find_loader` is replaced by :func:`importlib.util.find_spec`; @@ -2194,6 +2190,12 @@ that do not have a __format__ method that handles it. See :issue:`7994` for background. +* :meth:`difflib.SequenceMatcher.isbjunk` and + :meth:`difflib.SequenceMatcher.isbpopular` were deprecated in 3.2, and have + now been removed: use ``x in sm.bjunk`` and + ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object + (:issue:`13248`). + Code Cleanups ------------- diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3077,6 +3077,9 @@ - Issue #17467: add readline and readlines support to mock_open in unittest.mock. +- Issue #13248: removed deprecated and undocumented difflib.isbjunk, + isbpopular. + - Issue #17192: Update the ctypes module's libffi to v3.0.13. This specifically addresses a stack misalignment issue on x86 and issues on some more recent platforms. @@ -4512,8 +4515,6 @@ Library ------- -- Issue #13248: removed deprecated and undocumented difflib.isbjunk, isbpopular. - - Issue #13370: Ensure that ctypes works on Mac OS X when Python is compiled using the clang compiler. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 03:17:50 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 03:17:50 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_link_from_modu?= =?utf-8?q?le=5Fspec_section_to_Deprecation=2C_for_replacements=2E?= Message-ID: <3flVtQ5YCSz7LjX@mail.python.org> http://hg.python.org/cpython/rev/f2c257820a73 changeset: 89651:f2c257820a73 user: R David Murray date: Thu Mar 13 22:17:28 2014 -0400 summary: whatsnew: link from module_spec section to Deprecation, for replacements. files: Doc/whatsnew/3.4.rst | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -382,7 +382,9 @@ Furthermore, they should be transparent to everyone but importer authors. Key finder and loader methods have been deprecated, but they will continue working. New importers should use the new methods described in the PEP. Existing -importers should be updated to implement the new methods. +importers should be updated to implement the new methods. See the +:ref:`deprecated-3.4` section for a list of methods that should be replaced and +their replacements. Other Language Changes @@ -2038,6 +2040,8 @@ multiple threads. (Contributed by Antoine Pitrou in :issue:`18756`.) +.. _deprecated-3.4: + Deprecated ========== -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Fri Mar 14 09:54:35 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 14 Mar 2014 09:54:35 +0100 Subject: [Python-checkins] Daily reference leaks (f2c257820a73): sum=8 Message-ID: results for f2c257820a73 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_site leaked [2, 0, 0] references, sum=2 test_site leaked [2, 0, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogcZQGDQ', '-x'] From python-checkins at python.org Fri Mar 14 14:42:31 2014 From: python-checkins at python.org (vinay.sajip) Date: Fri, 14 Mar 2014 14:42:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2320918=3A_Added_h?= =?utf-8?q?andling_for_exceptions_during_fallback_output_of_logging?= Message-ID: <3flp4R3f8Rz7LjM@mail.python.org> http://hg.python.org/cpython/rev/73c2a70e4b35 changeset: 89652:73c2a70e4b35 user: Vinay Sajip date: Fri Mar 14 13:42:19 2014 +0000 summary: Closes #20918: Added handling for exceptions during fallback output of logging exceptions. files: Lib/logging/__init__.py | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -1,4 +1,4 @@ -# Copyright 2001-2013 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2014 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -18,7 +18,7 @@ Logging package for Python. Based on PEP 282 and comments thereto in comp.lang.python. -Copyright (C) 2001-2013 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2014 Vinay Sajip. All Rights Reserved. To use, simply 'import logging' and log away! """ @@ -42,6 +42,7 @@ __author__ = "Vinay Sajip " __status__ = "production" +# The following module attributes are no longer updated. __version__ = "0.5.1.2" __date__ = "07 February 2010" @@ -902,8 +903,15 @@ sys.stderr.write('Logged from file %s, line %s\n' % ( record.filename, record.lineno)) # Issue 18671: output logging message and arguments - sys.stderr.write('Message: %r\n' - 'Arguments: %s\n' % (record.msg, record.args)) + try: + sys.stderr.write('Message: %r\n' + 'Arguments: %s\n' % (record.msg, + record.args)) + except Exception: + sys.stderr.write('Unable to print the message and arguments' + ' - possible formatting error.\nUse the' + ' traceback above to help find the error.\n' + ) except OSError: #pragma: no cover pass # see issue 5971 finally: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 14:46:42 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 14 Mar 2014 14:46:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320884=3A_whatsnew?= =?utf-8?q?=3A_Frozen_modules_don=27t_set_=5F=5Ffile=5F=5F_anymore=2E?= Message-ID: <3flp9G5Rk1z7LjN@mail.python.org> http://hg.python.org/cpython/rev/b626f4978a28 changeset: 89653:b626f4978a28 user: Brett Cannon date: Fri Mar 14 09:46:28 2014 -0400 summary: Issue #20884: whatsnew: Frozen modules don't set __file__ anymore. files: Doc/whatsnew/3.4.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -2275,6 +2275,14 @@ to determine if a module is a package or not is to use``hasattr(module, '__path__')`` (:issue:`18065`). +* Frozen modules no longer define a ``__file__`` attribute. It's semantically + incorrect for frozen modules to set the attribute as they are not loaded from + any explicit location. If you must know that a module comes from frozen code + then you can see if the module's ``__spec__.location`` is set to ``'frozen'``, + check if the loader is a subclass of + :class:`importlib.machinery.FrozenImporter`, + or if Python 2 compatibility is necessary you can use :func:`imp.is_frozen`. + * :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path it would write to is a symlink or a non-regular file. This is to act as a warning that import will overwrite those files with a regular file regardless -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Mar 14 15:20:26 2014 From: python-checkins at python.org (vinay.sajip) Date: Fri, 14 Mar 2014 15:20:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_Doc/tools/_subdirs_t?= =?utf-8?q?o_=2Ehgignore=2E?= Message-ID: <3flpwB0Njyz7LjX@mail.python.org> http://hg.python.org/cpython/rev/edb12dad7bf6 changeset: 89654:edb12dad7bf6 user: Vinay Sajip date: Fri Mar 14 14:20:09 2014 +0000 summary: Added Doc/tools/ subdirs to .hgignore. files: .hgignore | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -24,6 +24,11 @@ reflog.txt$ tags$ Lib/plat-mac/errors.rsrc.df.rsrc +Doc/tools/sphinx/ +Doc/tools/docutils/ +Doc/tools/jinja/ +Doc/tools/jinja2/ +Doc/tools/pygments/ Misc/python.pc Misc/python-config.sh$ Modules/Setup$ -- Repository URL: http://hg.python.org/cpython From benjamin at python.org Fri Mar 14 16:10:24 2014 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 14 Mar 2014 08:10:24 -0700 Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. In-Reply-To: <3flpwB0Njyz7LjX@mail.python.org> References: <3flpwB0Njyz7LjX@mail.python.org> Message-ID: <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> On Fri, Mar 14, 2014, at 07:20 AM, vinay.sajip wrote: > http://hg.python.org/cpython/rev/edb12dad7bf6 > changeset: 89654:edb12dad7bf6 > user: Vinay Sajip > date: Fri Mar 14 14:20:09 2014 +0000 > summary: > Added Doc/tools/ subdirs to .hgignore. > > files: > .hgignore | 5 +++++ > 1 files changed, 5 insertions(+), 0 deletions(-) > > > diff --git a/.hgignore b/.hgignore > --- a/.hgignore > +++ b/.hgignore > @@ -24,6 +24,11 @@ > reflog.txt$ > tags$ > Lib/plat-mac/errors.rsrc.df.rsrc > +Doc/tools/sphinx/ > +Doc/tools/docutils/ > +Doc/tools/jinja/ > +Doc/tools/jinja2/ > +Doc/tools/pygments/ Why are you readding these when they were apparently purposely removed in eef7899ea7ab? From zachary.ware+pydev at gmail.com Fri Mar 14 16:52:28 2014 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Fri, 14 Mar 2014 10:52:28 -0500 Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. In-Reply-To: <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> References: <3flpwB0Njyz7LjX@mail.python.org> <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> Message-ID: On Fri, Mar 14, 2014 at 10:10 AM, Benjamin Peterson wrote: > On Fri, Mar 14, 2014, at 07:20 AM, vinay.sajip wrote: >> diff --git a/.hgignore b/.hgignore >> --- a/.hgignore >> +++ b/.hgignore >> @@ -24,6 +24,11 @@ >> reflog.txt$ >> tags$ >> Lib/plat-mac/errors.rsrc.df.rsrc >> +Doc/tools/sphinx/ >> +Doc/tools/docutils/ >> +Doc/tools/jinja/ >> +Doc/tools/jinja2/ >> +Doc/tools/pygments/ > > Why are you readding these when they were apparently purposely removed > in eef7899ea7ab? It is rather ugly to build the docs in a 3.3 checkout, then update to default and run 'hg status' without those entries in .hgignore. I can understand not wanting to be bombarded by all of the untracked files, but it's also pretty easy to enable the purge extension and do "hg purge Doc/tools". -- Zach From benjamin at python.org Fri Mar 14 16:55:30 2014 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 14 Mar 2014 08:55:30 -0700 Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. In-Reply-To: References: <3flpwB0Njyz7LjX@mail.python.org> <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> Message-ID: <1394812530.8956.94532457.00F2EB6E@webmail.messagingengine.com> On Fri, Mar 14, 2014, at 08:52 AM, Zachary Ware wrote: > On Fri, Mar 14, 2014 at 10:10 AM, Benjamin Peterson > wrote: > > On Fri, Mar 14, 2014, at 07:20 AM, vinay.sajip wrote: > >> diff --git a/.hgignore b/.hgignore > >> --- a/.hgignore > >> +++ b/.hgignore > >> @@ -24,6 +24,11 @@ > >> reflog.txt$ > >> tags$ > >> Lib/plat-mac/errors.rsrc.df.rsrc > >> +Doc/tools/sphinx/ > >> +Doc/tools/docutils/ > >> +Doc/tools/jinja/ > >> +Doc/tools/jinja2/ > >> +Doc/tools/pygments/ > > > > Why are you readding these when they were apparently purposely removed > > in eef7899ea7ab? > > It is rather ugly to build the docs in a 3.3 checkout, then update to > default and run 'hg status' without those entries in .hgignore. I can > understand not wanting to be bombarded by all of the untracked files, > but it's also pretty easy to enable the purge extension and do "hg > purge Doc/tools". This is why everyone should use the share extension and have separate working copies for every branch. :) From python-checkins at python.org Fri Mar 14 17:40:49 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 14 Mar 2014 17:40:49 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Assign_PEP_=23465_to_=22Dedic?= =?utf-8?q?ated_infix_operators_for_matrix_multiplication_and?= Message-ID: <3flt294MxPz7Ljp@mail.python.org> http://hg.python.org/peps/rev/733c83581fbe changeset: 5406:733c83581fbe user: Guido van Rossum date: Fri Mar 14 09:40:42 2014 -0700 summary: Assign PEP #465 to "Dedicated infix operators for matrix multiplication and matrix power" by Nathaniel J. Smith. files: pep-0465.txt | 1223 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 1223 insertions(+), 0 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt new file mode 100644 --- /dev/null +++ b/pep-0465.txt @@ -0,0 +1,1223 @@ +PEP: 465 +Title: Dedicated infix operators for matrix multiplication and matrix power +Version: $Revision$ +Last-Modified: $Date$ +Author: Nathaniel J. Smith +Status: Draft +Type: Standards Track +Python-Version: 3.5 +Content-Type: text/x-rst +Created: 20-Feb-2014 +Post-History: 13-Mar-2014 + +Abstract +======== + +This PEP proposes two new binary operators dedicated to matrix +multiplication and matrix power, spelled ``@`` and ``@@`` +respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) + + +Specification +============= + +Two new binary operators are added to the Python language, together +with corresponding in-place versions: + +======= ========================= =============================== + Op Precedence/associativity Methods +======= ========================= =============================== +``@`` Same as ``*`` ``__matmul__``, ``__rmatmul__`` +``@@`` Same as ``**`` ``__matpow__``, ``__rmatpow__`` +``@=`` n/a ``__imatmul__`` +``@@=`` n/a ``__imatpow__`` +======= ========================= =============================== + +No implementations of these methods are added to the builtin or +standard library types. However, a number of projects have reached +consensus on the recommended semantics for these operations; see +`Intended usage details`_ below. + + +Motivation +========== + +Executive summary +----------------- + +In numerical code, there are two important operations which compete +for use of Python's ``*`` operator: elementwise multiplication, and +matrix multiplication. In the nearly twenty years since the Numeric +library was first proposed, there have been many attempts to resolve +this tension [#hugunin]_; none have been really satisfactory. +Currently, most numerical Python code uses ``*`` for elementwise +multiplication, and function/method syntax for matrix multiplication; +however, this leads to ugly and unreadable code in common +circumstances. The problem is bad enough that significant amounts of +code continue to use the opposite convention (which has the virtue of +producing ugly and unreadable code in *different* circumstances), and +this API fragmentation across codebases then creates yet more +problems. There does not seem to be any *good* solution to the +problem of designing a numerical API within current Python syntax -- +only a landscape of options that are bad in different ways. The +minimal change to Python syntax which is sufficient to resolve these +problems is the addition of a single new infix operator for matrix +multiplication. + +Matrix multiplication has a singular combination of features which +distinguish it from other binary operations, which together provide a +uniquely compelling case for the addition of a dedicated infix +operator: + +* Just as for the existing numerical operators, there exists a vast + body of prior art supporting the use of infix notation for matrix + multiplication across all fields of mathematics, science, and + engineering; ``@`` harmoniously fills a hole in Python's existing + operator system. + +* ``@`` greatly clarifies real-world code. + +* ``@`` provides a smoother onramp for less experienced users, who are + particularly harmed by hard-to-read code and API fragmentation. + +* ``@`` benefits a substantial and growing portion of the Python user + community. + +* ``@`` will be used frequently -- in fact, evidence suggests it may + be used more frequently than ``//`` or the bitwise operators. + +* ``@`` allows the Python numerical community to reduce fragmentation, + and finally standardize on a single consensus duck type for all + numerical array objects. + +And, given the existence of ``@``, it makes more sense than not to +have ``@@``, ``@=``, and ``@@=``, so they are added as well. + + +Background: What's wrong with the status quo? +--------------------------------------------- + +When we crunch numbers on a computer, we usually have lots and lots of +numbers to deal with. Trying to deal with them one at a time is +cumbersome and slow -- especially when using an interpreted language. +Instead, we want the ability to write down simple operations that +apply to large collections of numbers all at once. The *n-dimensional +array* is the basic object that all popular numeric computing +environments use to make this possible. Python has several libraries +that provide such arrays, with numpy being at present the most +prominent. + +When working with n-dimensional arrays, there are two different ways +we might want to define multiplication. One is elementwise +multiplication:: + + [[1, 2], [[11, 12], [[1 * 11, 2 * 12], + [3, 4]] x [13, 14]] = [3 * 13, 4 * 14]] + +and the other is `matrix multiplication`_: + +.. _matrix multiplication: https://en.wikipedia.org/wiki/Matrix_multiplication + +:: + + [[1, 2], [[11, 12], [[1 * 11 + 2 * 13, 1 * 12 + 2 * 14], + [3, 4]] x [13, 14]] = [3 * 11 + 4 * 13, 3 * 12 + 4 * 14]] + +Elementwise multiplication is useful because it lets us easily and +quickly perform many multiplications on a large collection of values, +without writing a slow and cumbersome ``for`` loop. And this works as +part of a very general schema: when using the array objects provided +by numpy or other numerical libraries, all Python operators work +elementwise on arrays of all dimensionalities. The result is that one +can write functions using straightforward code like ``a * b + c / d``, +treating the variables as if they were simple values, but then +immediately use this function to efficiently perform this calculation +on large collections of values, while keeping them organized using +whatever arbitrarily complex array layout works best for the problem +at hand. + +Matrix multiplication is more of a special case. It's only defined on +2d arrays (also known as "matrices"), and multiplication is the only +operation that has a meaningful "matrix" version -- "matrix addition" +is the same as elementwise addition; there is no such thing as "matrix +bitwise-or" or "matrix floordiv"; "matrix division" can be defined but +is not very useful, etc. However, matrix multiplication is still used +very heavily across all numerical application areas; mathematically, +it's one of the most fundamental operations there is. + +Because Python syntax currently allows for only a single +multiplication operator ``*``, libraries providing array-like objects +must decide: either use ``*`` for elementwise multiplication, or use +``*`` for matrix multiplication. And, unfortunately, it turns out +that when doing general-purpose number crunching, both operations are +used frequently, and there are major advantages to using infix rather +than function call syntax in both cases. Thus it is not at all clear +which convention is optimal, or even acceptable; often it varies on a +case-by-case basis. + +Nonetheless, network effects mean that it is very important that we +pick *just one* convention. In numpy, for example, it is technically +possible to switch between the conventions, because numpy provides two +different types with different ``__mul__`` methods. For +``numpy.ndarray`` objects, ``*`` performs elementwise multiplication, +and matrix multiplication must use a function call (``numpy.dot``). +For ``numpy.matrix`` objects, ``*`` performs matrix multiplication, +and elementwise multiplication requires function syntax. Writing code +using ``numpy.ndarray`` works fine. Writing code using +``numpy.matrix`` also works fine. But trouble begins as soon as we +try to integrate these two pieces of code together. Code that expects +an ``ndarray`` and gets a ``matrix``, or vice-versa, may crash or +return incorrect results. Keeping track of which functions expect +which types as inputs, and return which types as outputs, and then +converting back and forth all the time, is incredibly cumbersome and +impossible to get right at any scale. Functions that defensively try +to handle both types as input and DTRT, find themselves floundering +into a swamp of ``isinstance`` and ``if`` statements. + +PEP 238 split ``/`` into two operators: ``/`` and ``//``. Imagine the +chaos that would have resulted if it had instead split ``int`` into +two types: ``classic_int``, whose ``__div__`` implemented floor +division, and ``new_int``, whose ``__div__`` implemented true +division. This, in a more limited way, is the situation that Python +number-crunchers currently find themselves in. + +In practice, the vast majority of projects have settled on the +convention of using ``*`` for elementwise multiplication, and function +call syntax for matrix multiplication (e.g., using ``numpy.ndarray`` +instead of ``numpy.matrix``). This reduces the problems caused by API +fragmentation, but it doesn't eliminate them. The strong desire to +use infix notation for matrix multiplication has caused a number of +specialized array libraries to continue to use the opposing convention +(e.g., scipy.sparse, pyoperators, pyviennacl) despite the problems +this causes, and ``numpy.matrix`` itself still gets used in +introductory programming courses, often appears in StackOverflow +answers, and so forth. Well-written libraries thus must continue to +be prepared to deal with both types of objects, and, of course, are +also stuck using unpleasant funcall syntax for matrix multiplication. +After nearly two decades of trying, the numerical community has still +not found any way to resolve these problems within the constraints of +current Python syntax (see `Rejected alternatives to adding a new +operator`_ below). + +This PEP proposes the minimum effective change to Python syntax that +will allow us to drain this swamp. It splits ``*`` into two +operators, just as was done for ``/``: ``*`` for elementwise +multiplication, and ``@`` for matrix multiplication. (Why not the +reverse? Because this way is compatible with the existing consensus, +and because it gives us a consistent rule that all the built-in +numeric operators also apply in an elementwise manner to arrays; the +reverse convention would lead to more special cases.) + +So that's why matrix multiplication doesn't and can't just use ``*``. +Now, in the the rest of this section, we'll explain why it nonetheless +meets the high bar for adding a new operator. + + +Why should matrix multiplication be infix? +------------------------------------------ + +Right now, most numerical code in Python uses syntax like +``numpy.dot(a, b)`` or ``a.dot(b)`` to perform matrix multiplication. +This obviously works, so why do people make such a fuss about it, even +to the point of creating API fragmentation and compatibility swamps? + +Matrix multiplication shares two features with ordinary arithmetic +operations like addition and multiplication on numbers: (a) it is used +very heavily in numerical programs -- often multiple times per line of +code -- and (b) it has an ancient and universally adopted tradition of +being written using infix syntax. This is because, for typical +formulas, this notation is dramatically more readable than any +function call syntax. Here's an example to demonstrate: + +One of the most useful tools for testing a statistical hypothesis is +the linear hypothesis test for OLS regression models. It doesn't +really matter what all those words I just said mean; if we find +ourselves having to implement this thing, what we'll do is look up +some textbook or paper on it, and encounter many mathematical formulas +that look like: + +.. math:: + + S = (H \beta - r)^T (H V H^T)^{-1} (H \beta - r) + +Here the various variables are all vectors or matrices (details for +the curious: [#lht]_). + +Now we need to write code to perform this calculation. In current +numpy, matrix multiplication can be performed using either the +function or method call syntax. Neither provides a particularly +readable translation of the formula:: + + import numpy as np + from numpy.linalg import inv, solve + + # Using dot function: + S = np.dot((np.dot(H, beta) - r).T, + np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r)) + + # Using dot method: + S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r) + +With the ``@`` operator, the direct translation of the above formula +becomes:: + + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +Notice that there is now a transparent, 1-to-1 mapping between the +symbols in the original formula and the code that implements it. + +Of course, an experienced programmer will probably notice that this is +not the best way to compute this expression. The repeated computation +of :math:`H \beta - r` should perhaps be factored out; and, +expressions of the form ``dot(inv(A), B)`` should almost always be +replaced by the more numerically stable ``solve(A, B)``. When using +``@``, performing these two refactorings gives us:: + + # Version 1 (as above) + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + + # Version 2 + trans_coef = H @ beta - r + S = trans_coef.T @ inv(H @ V @ H.T) @ trans_coef + + # Version 3 + S = trans_coef.T @ solve(H @ V @ H.T, trans_coef) + +Notice that when comparing between each pair of steps, it's very easy +to see exactly what was changed. If we apply the equivalent +transformations to the code using the .dot method, then the changes +are much harder to read out or verify for correctness:: + + # Version 1 (as above) + S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r) + + # Version 2 + trans_coef = H.dot(beta) - r + S = trans_coef.T.dot(inv(H.dot(V).dot(H.T))).dot(trans_coef) + + # Version 3 + S = trans_coef.T.dot(solve(H.dot(V).dot(H.T)), trans_coef) + +Readability counts! The statements using ``@`` are shorter, contain +more whitespace, can be directly and easily compared both to each +other and to the textbook formula, and contain only meaningful +parentheses. This last point is particularly important for +readability: when using function-call syntax, the required parentheses +on every operation create visual clutter that makes it very difficult +to parse out the overall structure of the formula by eye, even for a +relatively simple formula like this one. Eyes are terrible at parsing +non-regular languages. I made and caught many errors while trying to +write out the 'dot' formulas above. I know they still contain at +least one error, maybe more. (Exercise: find it. Or them.) The +``@`` examples, by contrast, are not only correct, they're obviously +correct at a glance. + +If we are even more sophisticated programmers, and writing code that +we expect to be reused, then considerations of speed or numerical +accuracy might lead us to prefer some particular order of evaluation. +Because ``@`` makes it possible to omit irrelevant parentheses, we can +be certain that if we *do* write something like ``(H @ V) @ H.T``, +then our readers will know that the parentheses must have been added +intentionally to accomplish some meaningful purpose. In the ``dot`` +examples, it's impossible to know which nesting decisions are +important, and which are arbitrary. + +Infix ``@`` dramatically improves matrix code usability at all stages +of programmer interaction. + + +Transparent syntax is especially crucial for non-expert programmers +------------------------------------------------------------------- + +A large proportion of scientific code is written by people who are +experts in their domain, but are not experts in programming. And +there are many university courses run each year with titles like "Data +analysis for social scientists" which assume no programming +background, and teach some combination of mathematical techniques, +introduction to programming, and the use of programming to implement +these mathematical techniques, all within a 10-15 week period. These +courses are more and more often being taught in Python rather than +special-purpose languages like R or Matlab. + +For these kinds of users, whose programming knowledge is fragile, the +existence of a transparent mapping between formulas and code often +means the difference between succeeding and failing to write that code +at all. This is so important that such classes often use the +``numpy.matrix`` type which defines ``*`` to mean matrix +multiplication, even though this type is buggy and heavily +disrecommended by the rest of the numpy community for the +fragmentation that it causes. This pedagogical use case is, in fact, +the *only* reason ``numpy.matrix`` remains a supported part of numpy. +Adding ``@`` will benefit both beginning and advanced users with +better syntax; and furthermore, it will allow both groups to +standardize on the same notation from the start, providing a smoother +on-ramp to expertise. + + +But isn't matrix multiplication a pretty niche requirement? +----------------------------------------------------------- + +The world is full of continuous data, and computers are increasingly +called upon to work with it in sophisticated ways. Arrays are the +lingua franca of finance, machine learning, 3d graphics, computer +vision, robotics, operations research, econometrics, meteorology, +computational linguistics, recommendation systems, neuroscience, +astronomy, bioinformatics (including genetics, cancer research, drug +discovery, etc.), physics engines, quantum mechanics, geophysics, +network analysis, and many other application areas. In most or all of +these areas, Python is rapidly becoming a dominant player, in large +part because of its ability to elegantly mix traditional discrete data +structures (hash tables, strings, etc.) on an equal footing with +modern numerical data types and algorithms. + +We all live in our own little sub-communities, so some Python users +may be surprised to realize the sheer extent to which Python is used +for number crunching -- especially since much of this particular +sub-community's activity occurs outside of traditional Python/FOSS +channels. So, to give some rough idea of just how many numerical +Python programmers are actually out there, here are two numbers: In +2013, there were 7 international conferences organized specifically on +numerical Python [#scipy-conf]_ [#pydata-conf]_. At PyCon 2014, ~20% +of the tutorials appear to involve the use of matrices +[#pycon-tutorials]_. + +To quantify this further, we used Github's "search" function to look +at what modules are actually imported across a wide range of +real-world code (i.e., all the code on Github). We checked for +imports of several popular stdlib modules, a variety of numerically +oriented modules, and various other extremely high-profile modules +like django and lxml (the latter of which is the #1 most downloaded +package on PyPI). Starred lines indicate packages which export array- +or matrix-like objects which will adopt ``@`` if this PEP is +approved:: + + Count of Python source files on Github matching given search terms + (as of 2014-04-10, ~21:00 UTC) + ================ ========== =============== ======= =========== + module "import X" "from X import" total total/numpy + ================ ========== =============== ======= =========== + sys 2374638 63301 2437939 5.85 + os 1971515 37571 2009086 4.82 + re 1294651 8358 1303009 3.12 + numpy ************** 337916 ********** 79065 * 416981 ******* 1.00 + warnings 298195 73150 371345 0.89 + subprocess 281290 63644 344934 0.83 + django 62795 219302 282097 0.68 + math 200084 81903 281987 0.68 + threading 212302 45423 257725 0.62 + pickle+cPickle 215349 22672 238021 0.57 + matplotlib 119054 27859 146913 0.35 + sqlalchemy 29842 82850 112692 0.27 + pylab *************** 36754 ********** 41063 ** 77817 ******* 0.19 + scipy *************** 40829 ********** 28263 ** 69092 ******* 0.17 + lxml 19026 38061 57087 0.14 + zlib 40486 6623 47109 0.11 + multiprocessing 25247 19850 45097 0.11 + requests 30896 560 31456 0.08 + jinja2 8057 24047 32104 0.08 + twisted 13858 6404 20262 0.05 + gevent 11309 8529 19838 0.05 + pandas ************** 14923 *********** 4005 ** 18928 ******* 0.05 + sympy 2779 9537 12316 0.03 + theano *************** 3654 *********** 1828 *** 5482 ******* 0.01 + ================ ========== =============== ======= =========== + +These numbers should be taken with several grains of salt (see +footnote for discussion: [#github-details]_), but, to the extent they +can be trusted, they suggest that ``numpy`` might be the single +most-imported non-stdlib module in the entire Pythonverse; it's even +more-imported than such stdlib stalwarts as ``subprocess``, ``math``, +``pickle``, and ``threading``. And numpy users represent only a +subset of the broader numerical community that will benefit from the +``@`` operator. Matrices may once have been a niche data type +restricted to Fortran programs running in university labs and military +clusters, but those days are long gone. Number crunching is a +mainstream part of modern Python usage. + +In addition, there is some precedence for adding an infix operator to +handle a more-specialized arithmetic operation: the floor division +operator ``//``, like the bitwise operators, is very useful under +certain circumstances when performing exact calculations on discrete +values. But it seems likely that there are many Python programmers +who have never had reason to use ``//`` (or, for that matter, the +bitwise operators). ``@`` is no more niche than ``//``. + + +So ``@`` is good for matrix formulas, but how common are those really? +---------------------------------------------------------------------- + +We've seen that ``@`` makes matrix formulas dramatically easier to +work with for both experts and non-experts, that matrix formulas +appear in many important applications, and that numerical libraries +like numpy are used by a substantial proportion of Python's user base. +But numerical libraries aren't just about matrix formulas, and being +important doesn't necessarily mean taking up a lot of code: if matrix +formulas only occured in one or two places in the average +numerically-oriented project, then it still wouldn't be worth adding a +new operator. So how common is matrix multiplication, really? + +When the going gets tough, the tough get empirical. To get a rough +estimate of how useful the ``@`` operator will be, the table below +shows the rate at which different Python operators are actually used +in the stdlib, and also in two high-profile numerical packages -- the +scikit-learn machine learning library, and the nipy neuroimaging +library -- normalized by source lines of code (SLOC). Rows are sorted +by the 'combined' column, which pools all three code bases together. +The combined column is thus strongly weighted towards the stdlib, +which is much larger than both projects put together (stdlib: 411575 +SLOC, scikit-learn: 50924 SLOC, nipy: 37078 SLOC). [#sloc-details]_ + +The ``dot`` row (marked ``******``) counts how common matrix multiply +operations are in each codebase. + +:: + + ==== ====== ============ ==== ======== + op stdlib scikit-learn nipy combined + ==== ====== ============ ==== ======== + = 2969 5536 4932 3376 / 10,000 SLOC + - 218 444 496 261 + + 224 201 348 231 + == 177 248 334 196 + * 156 284 465 192 + % 121 114 107 119 + ** 59 111 118 68 + != 40 56 74 44 + / 18 121 183 41 + > 29 70 110 39 + += 34 61 67 39 + < 32 62 76 38 + >= 19 17 17 18 + <= 18 27 12 18 + dot ***** 0 ********** 99 ** 74 ****** 16 + | 18 1 2 15 + & 14 0 6 12 + << 10 1 1 8 + // 9 9 1 8 + -= 5 21 14 8 + *= 2 19 22 5 + /= 0 23 16 4 + >> 4 0 0 3 + ^ 3 0 0 3 + ~ 2 4 5 2 + |= 3 0 0 2 + &= 1 0 0 1 + //= 1 0 0 1 + ^= 1 0 0 0 + **= 0 2 0 0 + %= 0 0 0 0 + <<= 0 0 0 0 + >>= 0 0 0 0 + ==== ====== ============ ==== ======== + +These two numerical packages alone contain ~780 uses of matrix +multiplication. Within these packages, matrix multiplication is used +more heavily than most comparison operators (``<`` ``!=`` ``<=`` +``>=``). Even when we dilute these counts by including the stdlib +into our comparisons, matrix multiplication is still used more often +in total than any of the bitwise operators, and 2x as often as ``//``. +This is true even though the stdlib, which contains a fair amount of +integer arithmetic and no matrix operations, makes up more than 80% of +the combined code base. + +By coincidence, the numeric libraries make up approximately the same +proportion of the 'combined' codebase as numeric tutorials make up of +PyCon 2014's tutorial schedule, which suggests that the 'combined' +column may not be *wildly* unrepresentative of new Python code in +general. While it's impossible to know for certain, from this data it +seems entirely possible that across all Python code currently being +written, matrix multiplication is already used more often than ``//`` +and the bitwise operations. + + +But isn't it weird to add an operator with no stdlib uses? +---------------------------------------------------------- + +It's certainly unusual (though ``Ellipsis`` was also added without any +stdlib uses). But the important thing is whether a change will +benefit users, not where the software is being downloaded from. It's +clear from the above that ``@`` will be used, and used heavily. And +this PEP provides the critical piece that will allow the Python +numerical community to finally reach consensus on a standard duck type +for all array-like objects, which is a necessary precondition to ever +adding a numerical array type to the stdlib. + + +Matrix power and in-place operators +----------------------------------- + +The primary motivation for this PEP is ``@``; the other proposed +operators don't have nearly as much impact. The matrix power operator +``@@`` is useful and well-defined, but not really necessary. It is +still included, though, for consistency: if we have an ``@`` that is +analogous to ``*``, then it would be weird and surprising to *not* +have an ``@@`` that is analogous to ``**``. Similarly, the in-place +operators ``@=`` and ``@@=`` provide limited value -- it's more common +to write ``a = (b @ a)`` than it is to write ``a = (a @ b)``, and +in-place matrix operations still generally have to allocate +substantial temporary storage -- but they are included for +completeness and symmetry. + + +Compatibility considerations +============================ + +Currently, the only legal use of the ``@`` token in Python code is at +statement beginning in decorators. The new operators are all infix; +the one place they can never occur is at statement beginning. +Therefore, no existing code will be broken by the addition of these +operators, and there is no possible parsing ambiguity between +decorator-@ and the new operators. + +Another important kind of compatibility is the mental cost paid by +users to update their understanding of the Python language after this +change, particularly for users who do not work with matrices and thus +do not benefit. Here again, ``@`` has minimal impact: even +comprehensive tutorials and references will only need to add a +sentence or two to fully document this PEP's changes for a +non-numerical audience. + + +Intended usage details +====================== + +This section is informative, rather than normative -- it documents the +consensus of a number of libraries that provide array- or matrix-like +objects on how the ``@`` and ``@@`` operators will be implemented. + +This section uses the numpy terminology for describing arbitrary +multidimensional arrays of data, because it is a superset of all other +commonly used models. In this model, the *shape* of any array is +represented by a tuple of integers. Because matrices are +two-dimensional, they have len(shape) == 2, while 1d vectors have +len(shape) == 1, and scalars have shape == (), i.e., they are "0 +dimensional". Any array contains prod(shape) total entries. Notice +that `prod(()) == 1`_ (for the same reason that sum(()) == 0); scalars +are just an ordinary kind of array, not a special case. Notice also +that we distinguish between a single scalar value (shape == (), +analogous to ``1``), a vector containing only a single entry (shape == +(1,), analogous to ``[1]``), a matrix containing only a single entry +(shape == (1, 1), analogous to ``[[1]]``), etc., so the dimensionality +of any array is always well-defined. Other libraries with more +restricted representations (e.g., those that support 2d arrays only) +might implement only a subset of the functionality described here. + +.. _prod(()) == 1: https://en.wikipedia.org/wiki/Empty_product + +Semantics +--------- + +The recommended semantics for ``@`` for different inputs are: + +* 2d inputs are conventional matrices, and so the semantics are + obvious: we apply conventional matrix multiplication. If we write + ``arr(2, 3)`` to represent an arbitrary 2x3 array, then ``arr(3, 4) + @ arr(4, 5)`` returns an array with shape (3, 5). + +* 1d vector inputs are promoted to 2d by prepending or appending a '1' + to the shape, the operation is performed, and then the added + dimension is removed from the output. The 1 is always added on the + "outside" of the shape: prepended for left arguments, and appended + for right arguments. The result is that matrix @ vector and vector + @ matrix are both legal (assuming compatible shapes), and both + return 1d vectors; vector @ vector returns a scalar. This is + clearer with examples. + + * ``arr(2, 3) @ arr(3, 1)`` is a regular matrix product, and returns + an array with shape (2, 1), i.e., a column vector. + + * ``arr(2, 3) @ arr(3)`` performs the same computation as the + previous (i.e., treats the 1d vector as a matrix containing a + single *column*, shape = (3, 1)), but returns the result with + shape (2,), i.e., a 1d vector. + + * ``arr(1, 3) @ arr(3, 2)`` is a regular matrix product, and returns + an array with shape (1, 2), i.e., a row vector. + + * ``arr(3) @ arr(3, 2)`` performs the same computation as the + previous (i.e., treats the 1d vector as a matrix containing a + single *row*, shape = (1, 3)), but returns the result with shape + (2,), i.e., a 1d vector. + + * ``arr(1, 3) @ arr(3, 1)`` is a regular matrix product, and returns + an array with shape (1, 1), i.e., a single value in matrix form. + + * ``arr(3) @ arr(3)`` performs the same computation as the + previous, but returns the result with shape (), i.e., a single + scalar value, not in matrix form. So this is the standard inner + product on vectors. + + An infelicity of this definition for 1d vectors is that it makes + ``@`` non-associative in some cases (``(Mat1 @ vec) @ Mat2`` != + ``Mat1 @ (vec @ Mat2)``). But this seems to be a case where + practicality beats purity: non-associativity only arises for strange + expressions that would never be written in practice; if they are + written anyway then there is a consistent rule for understanding + what will happen (``Mat1 @ vec @ Mat2`` is parsed as ``(Mat1 @ vec) + @ Mat2``, just like ``a - b - c``); and, not supporting 1d vectors + would rule out many important use cases that do arise very commonly + in practice. No-one wants to explain to new users why to solve the + simplest linear system in the obvious way, they have to type + ``(inv(A) @ b[:, np.newaxis]).flatten()`` instead of ``inv(A) @ b``, + or perform an ordinary least-squares regression by typing + ``solve(X.T @ X, X @ y[:, np.newaxis]).flatten()`` instead of + ``solve(X.T @ X, X @ y)``. No-one wants to type ``(a[np.newaxis, :] + @ b[:, np.newaxis])[0, 0]`` instead of ``a @ b`` every time they + compute an inner product, or ``(a[np.newaxis, :] @ Mat @ b[:, + np.newaxis])[0, 0]`` for general quadratic forms instead of ``a @ + Mat @ b``. In addition, sage and sympy (see below) use these + non-associative semantics with an infix matrix multiplication + operator (they use ``*``), and they report that they haven't + experienced any problems caused by it. + +* For inputs with more than 2 dimensions, we treat the last two + dimensions as being the dimensions of the matrices to multiply, and + 'broadcast' across the other dimensions. This provides a convenient + way to quickly compute many matrix products in a single operation. + For example, ``arr(10, 2, 3) @ arr(10, 3, 4)`` performs 10 separate + matrix multiplies, each of which multiplies a 2x3 and a 3x4 matrix + to produce a 2x4 matrix, and then returns the 10 resulting matrices + together in an array with shape (10, 2, 4). The intuition here is + that we treat these 3d arrays of numbers as if they were 1d arrays + *of matrices*, and then apply matrix multiplication in an + elementwise manner, where now each 'element' is a whole matrix. + Note that broadcasting is not limited to perfectly aligned arrays; + in more complicated cases, it allows several simple but powerful + tricks for controlling how arrays are aligned with each other; see + [#broadcasting]_ for details. (In particular, it turns out that + when broadcasting is taken into account, the standard scalar * + matrix product is a special case of the elementwise multiplication + operator ``*``.) + + If one operand is >2d, and another operand is 1d, then the above + rules apply unchanged, with 1d->2d promotion performed before + broadcasting. E.g., ``arr(10, 2, 3) @ arr(3)`` first promotes to + ``arr(10, 2, 3) @ arr(3, 1)``, then broadcasts the right argument to + create the aligned operation ``arr(10, 2, 3) @ arr(10, 3, 1)``, + multiplies to get an array with shape (10, 2, 1), and finally + removes the added dimension, returning an array with shape (10, 2). + Similarly, ``arr(2) @ arr(10, 2, 3)`` produces an intermediate array + with shape (10, 1, 3), and a final array with shape (10, 3). + +* 0d (scalar) inputs raise an error. Scalar * matrix multiplication + is a mathematically and algorithmically distinct operation from + matrix @ matrix multiplication, and is already covered by the + elementwise ``*`` operator. Allowing scalar @ matrix would thus + both require an unnecessary special case, and violate TOOWTDI. + +The recommended semantics for ``@@`` are:: + + def __matpow__(self, n): + if not isinstance(n, numbers.Integral): + raise TypeError("@@ not implemented for fractional powers") + if n == 0: + return identity_matrix_with_shape(self.shape) + elif n < 0: + return inverse(self) @ (self @@ (n + 1)) + else: + return self @ (self @@ (n - 1)) + +(Of course we expect that much more efficient implementations will be +used in practice.) Notice that if given an appropriate definition of +``identity_matrix_with_shape``, then this definition will +automatically handle >2d arrays appropriately. Notice also that with +this definition, ``vector @@ 2`` gives the squared Euclidean length of +the vector, a commonly used value. Also, while it is rarely useful to +explicitly compute inverses or other negative powers in standard +immediate-mode dense matrix code, these computations are natural when +doing symbolic or deferred-mode computations (as in e.g. sympy, +theano, numba, numexpr); therefore, negative powers are fully +supported. Fractional powers, though, bring in variety of +`mathematical complications`_, so we leave it to individual projects +to decide whether they want to try to define some reasonable semantics +for fractional inputs. + +.. _`mathematical complications`: +https://en.wikipedia.org/wiki/Square_root_of_a_matrix + + +Adoption +-------- + +We group existing Python projects which provide array- or matrix-like +types based on what API they currently use for elementwise and matrix +multiplication. + +**Projects which currently use * for *elementwise* multiplication, and +function/method calls for *matrix* multiplication:** + +The developers of the following projects have expressed an intention +to implement ``@`` and ``@@`` on their array-like types using the +above semantics: + +* numpy +* pandas +* blaze +* theano + +The following projects have been alerted to the existence of the PEP, +but it's not yet known what they plan to do if it's accepted. We +don't anticipate that they'll have any objections, though, since +everything proposed here is consistent with how they already do +things: + +* pycuda +* panda3d + +**Projects which currently use * for *matrix* multiplication, and +function/method calls for *elementwise* multiplication:** + +The following projects have expressed an intention, if this PEP is +accepted, to migrate from their current API to the elementwise-``*``, +matmul-``@`` convention (i.e., this is a list of projects whose API +fragmentation will probably be eliminated if this PEP is accepted): + +* numpy (``numpy.matrix``) +* scipy.sparse +* pyoperators +* pyviennacl + +The following projects have been alerted to the existence of the PEP, +but it's not known what they plan to do if it's accepted (i.e., this +is a list of projects whose API fragmentation may or may not be +eliminated if this PEP is accepted): + +* cvxopt + +**Projects which currently use * for *matrix* multiplication, and +which do not implement elementwise multiplication at all:** + +There are several projects which implement matrix types, but from a +very different perspective than the numerical libraries discussed +above. These projects focus on computational methods for analyzing +matrices in the sense of abstract mathematical objects (i.e., linear +maps over free modules over rings), rather than as big bags full of +numbers that need crunching. And it turns out that from the abstract +math point of view, there isn't much use for elementwise operations in +the first place; as discussed in the Background section above, +elementwise operations are motivated by the bag-of-numbers approach. +The different goals of these projects mean that they don't encounter +the basic problem that this PEP exists to address, making it mostly +irrelevant to them; while they appear superficially similar, they're +actually doing something quite different. They use ``*`` for matrix +multiplication (and for group actions, and so forth), and if this PEP +is accepted, their expressed intention is to continue doing so, while +perhaps adding ``@`` and ``@@`` on matrices as aliases for ``*`` and +``**``: + +* sympy +* sage + +If you know of any actively maintained Python libraries which provide +an interface for working with numerical arrays or matrices, and which +are not listed above, then please let the PEP author know: +njs at pobox.com + + +Rationale for specification details +=================================== + +Choice of operator +------------------ + +Why ``@`` instead of some other punctuation symbol? It doesn't matter +much, and there isn't any consensus across other programming languages +about how this operator should be named [#matmul-other-langs]_, but +``@`` has a few advantages: + +* ``@`` is a friendly character that Pythoneers are already used to + typing in decorators, and its use in email addresses means it is + more likely to be easily accessible across keyboard layouts than + some other characters (e.g. ``$`` or non-ASCII characters). + +* The mATrices mnemonic is cute. + +* It's round like ``*`` and :math:`\cdot`. + +* The use of a single-character token makes ``@@`` possible, which is + a nice bonus. + +* The swirly shape is reminiscent of the simultaneous sweeps over rows + and columns that define matrix multiplication; its asymmetry is + evocative of its non-commutative nature. + + +(Non)-Definitions for built-in types +------------------------------------ + +No ``__matmul__`` or ``__matpow__`` are defined for builtin numeric +types (``float``, ``int``, etc.) or for the ``numbers.Number`` +hierarchy, because these types represent scalars, and the consensus +semantics for ``@`` are that it should raise an error on scalars. + +We do not -- for now -- define a ``__matmul__`` method on the standard +``memoryview`` or ``array.array`` objects, for several reasons. +First, there is currently no way to create multidimensional memoryview +objects using only the stdlib, and array objects cannot represent +multidimensional data at all, which makes ``__matmul__`` much less +useful. Second, providing a quality implementation of matrix +multiplication is highly non-trivial. Naive nested loop +implementations are very slow and providing one in CPython would just +create a trap for users. But the alternative -- providing a modern, +competitive matrix multiply -- would require that CPython link to a +BLAS library, which brings a set of new complications. In particular, +several popular BLAS libraries (including the one that ships by +default on OS X) currently break the use of ``multiprocessing`` +[#blas-fork]_. And finally, we'd have to add quite a bit beyond +``__matmul__`` before ``memoryview`` or ``array.array`` would be +useful for numeric work -- like elementwise versions of the other +arithmetic operators, just to start. Put together, these +considerations mean that the cost/benefit of adding ``__matmul__`` to +these types just isn't there, so for now we'll continue to delegate +these problems to numpy and friends, and defer a more systematic +solution to a future proposal. + +There are also non-numeric Python builtins which define ``__mul__`` +(``str``, ``list``, ...). We do not define ``__matmul__`` for these +types either, because why would we even do that. + + +Unresolved issues +----------------- + +Associativity of ``@`` +'''''''''''''''''''''' + +It's been suggested that ``@`` should be right-associative, on the +grounds that for expressions like ``Mat @ Mat @ vec``, the two +different evaluation orders produce the same result, but the +right-associative order ``Mat @ (Mat @ vec)`` will be faster and use +less memory than the left-associative order ``(Mat @ Mat) @ vec``. +(Matrix-vector multiplication is much cheaper than matrix-matrix +multiplication). It would be a shame if users found themselves +required to use an overabundance of parentheses to achieve acceptable +speed/memory usage in common situations, but, it's not currently clear +whether such cases actually are common enough to override Python's +general rule of left-associativity, or even whether they're more +common than the symmetric cases where left-associativity would be +faster (though this does seem intuitively plausible). The only way to +answer this is probably to do an audit of some real-world uses and +check how often the associativity matters in practice; if this PEP is +accepted in principle, then we should probably do this check before +finalizing it. + + +Rejected alternatives to adding a new operator +============================================== + +Over the past few decades, the Python numeric community has explored a +variety of ways to resolve the tension between matrix and elementwise +multiplication operations. PEP 211 and PEP 225, both proposed in 2000 +and last seriously discussed in 2008 [#threads-2008]_, were early +attempts to add new operators to solve this problem, but suffered from +serious flaws; in particular, at that time the Python numerical +community had not yet reached consensus on the proper API for array +objects, or on what operators might be needed or useful (e.g., PEP 225 +proposes 6 new operators with unspecified semantics). Experience +since then has now led to consensus that the best solution, for both +numeric Python and core Python, is to add a single infix operator for +matrix multiply (together with the other new operators this implies +like ``@=``). + +We review some of the rejected alternatives here. + +**Use a second type that defines __mul__ as matrix multiplication:** +As discussed above (`Background: What's wrong with the status quo?`_), +this has been tried this for many years via the ``numpy.matrix`` type +(and its predecessors in Numeric and numarray). The result is a +strong consensus among both numpy developers and developers of +downstream packages that ``numpy.matrix`` should essentially never be +used, because of the problems caused by having conflicting duck types +for arrays. (Of course one could then argue we should *only* define +``__mul__`` to be matrix multiplication, but then we'd have the same +problem with elementwise multiplication.) There have been several +pushes to remove ``numpy.matrix`` entirely; the only counter-arguments +have come from educators who find that its problems are outweighed by +the need to provide a simple and clear mapping between mathematical +notation and code for novices (see `Transparent syntax is especially +crucial for non-expert programmers`_). But, of course, starting out +newbies with a dispreferred syntax and then expecting them to +transition later causes its own problems. The two-type solution is +worse than the disease. + +**Add lots of new operators, or add a new generic syntax for defining +infix operators:** In addition to being generally un-Pythonic and +repeatedly rejected by BDFL fiat, this would be using a sledgehammer +to smash a fly. The scientific python community has consensus that +adding one operator for matrix multiplication is enough to fix the one +otherwise unfixable pain point. (In retrospect, we all think PEP 225 +was a bad idea too -- or at least far more complex than it needed to +be.) + +**Add a new @ (or whatever) operator that has some other meaning in +general Python, and then overload it in numeric code:** This was the +approach taken by PEP 211, which proposed defining ``@`` to be the +equivalent of ``itertools.product``. The problem with this is that +when taken on its own terms, adding an infix operator for +``itertools.product`` is just silly. (During discussions of this PEP, +a similar suggestion was made to define ``@`` as a general purpose +function composition operator, and this suffers from the same problem; +``functools.compose`` isn't even useful enough to exist.) Matrix +multiplication has a uniquely strong rationale for inclusion as an +infix operator. There almost certainly don't exist any other binary +operations that will ever justify adding any other infix operators to +Python. + +**Add a .dot method to array types so as to allow "pseudo-infix" +A.dot(B) syntax:** This has been in numpy for some years, and in many +cases it's better than dot(A, B). But it's still much less readable +than real infix notation, and in particular still suffers from an +extreme overabundance of parentheses. See `Why should matrix +multiplication be infix?`_ above. + +**Use a 'with' block to toggle the meaning of * within a single code +block**: E.g., numpy could define a special context object so that +we'd have:: + + c = a * b # element-wise multiplication + with numpy.mul_as_dot: + c = a * b # matrix multiplication + +However, this has two serious problems: first, it requires that every +array-like type's ``__mul__`` method know how to check some global +state (``numpy.mul_is_currently_dot`` or whatever). This is fine if +``a`` and ``b`` are numpy objects, but the world contains many +non-numpy array-like objects. So this either requires non-local +coupling -- every numpy competitor library has to import numpy and +then check ``numpy.mul_is_currently_dot`` on every operation -- or +else it breaks duck-typing, with the above code doing radically +different things depending on whether ``a`` and ``b`` are numpy +objects or some other sort of object. Second, and worse, ``with`` +blocks are dynamically scoped, not lexically scoped; i.e., any +function that gets called inside the ``with`` block will suddenly find +itself executing inside the mul_as_dot world, and crash and burn +horribly -- if you're lucky. So this is a construct that could only +be used safely in rather limited cases (no function calls), and which +would make it very easy to shoot yourself in the foot without warning. + +**Use a language preprocessor that adds extra numerically-oriented +operators and perhaps other syntax:** (As per recent BDFL suggestion: +[#preprocessor]_) This suggestion seems based on the idea that +numerical code needs a wide variety of syntax additions. In fact, +given ``@``, most numerical users don't need any other operators or +syntax; it solves the one really painful problem that cannot be solved +by other means, and that causes painful reverberations through the +larger ecosystem. Defining a new language (presumably with its own +parser which would have to be kept in sync with Python's, etc.), just +to support a single binary operator, is neither practical nor +desireable. In the numerical context, Python's competition is +special-purpose numerical languages (Matlab, R, IDL, etc.). Compared +to these, Python's killer feature is exactly that one can mix +specialized numerical code with code for XML parsing, web page +generation, database access, network programming, GUI libraries, and +so forth, and we also gain major benefits from the huge variety of +tutorials, reference material, introductory classes, etc., which use +Python. Fragmenting "numerical Python" from "real Python" would be a +major source of confusion. A major motivation for this PEP is to +*reduce* fragmentation. Having to set up a preprocessor would be an +especially prohibitive complication for unsophisticated users. And we +use Python because we like Python! We don't want +almost-but-not-quite-Python. + +**Use overloading hacks to define a "new infix operator" like *dot*, +as in a well-known Python recipe:** (See: [#infix-hack]_) Beautiful is +better than ugly. This is... not beautiful. And not Pythonic. And +especially unfriendly to beginners, who are just trying to wrap their +heads around the idea that there's a coherent underlying system behind +these magic incantations that they're learning, when along comes an +evil hack like this that violates that system, creates bizarre error +messages when accidentally misused, and whose underlying mechanisms +can't be understood without deep knowledge of how object oriented +systems work. We've considered promoting this as a general solution, +and perhaps if the PEP is rejected we'll revisit this option, but so +far the numeric community has mostly elected to leave this one on the +shelf. + + +References +========== + +.. [#preprocessor] From a comment by GvR on a G+ post by GvR; the + comment itself does not seem to be directly linkable: +https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u +.. [#infix-hack] http://code.activestate.com/recipes/384122-infix-operators/ + http://www.sagemath.org/doc/reference/misc/sage/misc/decorators.html#sage.misc.decorators.infix_operator +.. [#scipy-conf] http://conference.scipy.org/past.html +.. [#pydata-conf] http://pydata.org/events/ +.. [#lht] In this formula, :math:`\beta` is a vector or matrix of + regression coefficients, :math:`V` is the estimated + variance/covariance matrix for these coefficients, and we want to + test the null hypothesis that :math:`H\beta = r`; a large :math:`S` + then indicates that this hypothesis is unlikely to be true. For + example, in an analysis of human height, the vector :math:`\beta` + might contain one value which was the the average height of the + measured men, and another value which was the average height of the + measured women, and then setting :math:`H = [1, -1], r = 0` would + let us test whether men and women are the same height on + average. Compare to eq. 2.139 in + http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode17.html + + Example code is adapted from +https://github.com/rerpy/rerpy/blob/0d274f85e14c3b1625acb22aed1efa85d122ecb7/rerpy/incremental_ls.py#L202 + +.. [#pycon-tutorials] Out of the 36 tutorials scheduled for PyCon 2014 + (https://us.pycon.org/2014/schedule/tutorials/), we guess that the + 8 below will almost certainly deal with matrices: + + * Dynamics and control with Python + + * Exploring machine learning with Scikit-learn + + * How to formulate a (science) problem and analyze it using Python + code + + * Diving deeper into Machine Learning with Scikit-learn + + * Data Wrangling for Kaggle Data Science Competitions ? An etude + + * Hands-on with Pydata: how to build a minimal recommendation + engine. + + * Python for Social Scientists + + * Bayesian statistics made simple + + In addition, the following tutorials could easily involve matrices: + + * Introduction to game programming + + * mrjob: Snakes on a Hadoop *("We'll introduce some data science + concepts, such as user-user similarity, and show how to calculate + these metrics...")* + + * Mining Social Web APIs with IPython Notebook + + * Beyond Defaults: Creating Polished Visualizations Using Matplotlib + + This gives an estimated range of 8 to 12 / 36 = 22% to 33% of + tutorials dealing with matrices; saying ~20% then gives us some + wiggle room in case our estimates are high. + +.. [#sloc-details] SLOCs were defined as physical lines which contain + at least one token that is not a COMMENT, NEWLINE, ENCODING, + INDENT, or DEDENT. Counts were made by using ``tokenize`` module + from Python 3.2.3 to examine the tokens in all files ending ``.py`` + underneath some directory. Only tokens which occur at least once + in the source trees are included in the table. The counting script + will be available as an auxiliary file once this PEP is submitted; + until then, it can be found here: + https://gist.github.com/njsmith/9157645 + + Matrix multiply counts were estimated by counting how often certain + tokens which are used as matrix multiply function names occurred in + each package. In principle this could create false positives, but + as far as I know the counts are exact; it's unlikely that anyone is + using ``dot`` as a variable name when it's also the name of one of + the most widely-used numpy functions. + + All counts were made using the latest development version of each + project as of 21 Feb 2014. + + 'stdlib' is the contents of the Lib/ directory in commit + d6aa3fa646e2 to the cpython hg repository, and treats the following + tokens as indicating matrix multiply: n/a. + + 'scikit-learn' is the contents of the sklearn/ directory in commit + 69b71623273ccfc1181ea83d8fb9e05ae96f57c7 to the scikit-learn + repository (https://github.com/scikit-learn/scikit-learn), and + treats the following tokens as indicating matrix multiply: ``dot``, + ``fast_dot``, ``safe_sparse_dot``. + + 'nipy' is the contents of the nipy/ directory in commit + 5419911e99546401b5a13bd8ccc3ad97f0d31037 to the nipy repository + (https://github.com/nipy/nipy/), and treats the following tokens as + indicating matrix multiply: ``dot``. + +.. [#blas-fork] BLAS libraries have a habit of secretly spawning + threads, even when used from single-threaded programs. And threads + play very poorly with ``fork()``; the usual symptom is that + attempting to perform linear algebra in a child process causes an + immediate deadlock. + +.. [#threads-2008] http://fperez.org/py4science/numpy-pep225/numpy-pep225.html + +.. [#broadcasting] http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html + +.. [#matmul-other-langs] +http://mail.scipy.org/pipermail/scipy-user/2014-February/035499.html + +.. [#github-details] Counts were produced by manually entering the + string ``"import foo"`` or ``"from foo import"`` (with quotes) into + the Github code search page, e.g.: + https://github.com/search?q=%22import+numpy%22&ref=simplesearch&type=Code + on 2014-04-10 at ~21:00 UTC. The reported values are the numbers + given in the "Languages" box on the lower-left corner, next to + "Python". This also causes some undercounting (e.g., leaving out + Cython code, and possibly one should also count HTML docs and so + forth), but these effects are negligible (e.g., only ~1% of numpy + usage appears to occur in Cython code, and probably even less for + the other modules listed). The use of this box is crucial, + however, because these counts appear to be stable, while the + "overall" counts listed at the top of the page ("We've found ___ + code results") are highly variable even for a single search -- + simply reloading the page can cause this number to vary by a factor + of 2 (!!). (They do seem to settle down if one reloads the page + repeatedly, but nonetheless this is spooky enough that it seemed + better to avoid these numbers.) + + These numbers should of course be taken with multiple grains of + salt; it's not clear how representative Github is of Python code in + general, and limitations of the search tool make it impossible to + get precise counts. AFAIK this is the best data set currently + available, but it'd be nice if it were better. In particular: + + * Lines like ``import sys, os`` will only be counted in the ``sys`` + row. + + * A file containing both ``import X`` and ``from X import`` will be + counted twice + + * Imports of the form ``from X.foo import ...`` are missed. We + could catch these by instead searching for "from X", but this is + a common phrase in English prose, so we'd end up with false + positives from comments, strings, etc. For many of the modules + considered this shouldn't matter too much -- for example, the + stdlib modules have flat namespaces -- but it might especially + lead to undercounting of django, scipy, and twisted. + + Also, it's possible there exist other non-stdlib modules we didn't + think to test that are even more-imported than numpy -- though we + tried quite a few of the obvious suspects. If you find one, let us + know! The modules tested here were chosen based on a combination + of intuition and the top-100 list at pypi-ranking.info. + + Fortunately, it doesn't really matter if it turns out that numpy + is, say, merely the *third* most-imported non-stdlib module, since + the point is just that numeric programming is a common and + mainstream activity. + + Finally, we should point out the obvious: whether a package is + import**ed** is rather different from whether it's import**ant**. + No-one's claiming numpy is "the most important package" or anything + like that. Certainly more packages depend on distutils, e.g., then + depend on numpy -- and far fewer source files import distutils than + import numpy. But this is fine for our present purposes. Most + source files don't import distutils because most source files don't + care how they're distributed, so long as they are; these source + files thus don't care about details of how distutils' API works. + This PEP is in some sense about changing how numpy's and related + packages' APIs work, so the relevant metric is to look at source + files that are choosing to directly interact with that API, which + is sort of like what we get by looking at import statements. + +.. [#hugunin] The first such proposal occurs in Jim Hugunin's very + first email to the matrix SIG in 1995, which lays out the first + draft of what became Numeric. He suggests using ``*`` for + elementwise multiplication, and ``%`` for matrix multiplication: + https://mail.python.org/pipermail/matrix-sig/1995-August/000002.html + + +Copyright +========= + +This document has been placed in the public domain. -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Fri Mar 14 17:43:14 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 14 Mar 2014 17:43:14 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_mark-up_errors_I_introduc?= =?utf-8?q?ed_by_copy/pasting=2E?= Message-ID: <3flt4y5bCYz7Ljg@mail.python.org> http://hg.python.org/peps/rev/8053a66b0bab changeset: 5407:8053a66b0bab user: Guido van Rossum date: Fri Mar 14 09:43:12 2014 -0700 summary: Fix mark-up errors I introduced by copy/pasting. files: pep-0465.txt | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -733,7 +733,7 @@ for fractional inputs. .. _`mathematical complications`: -https://en.wikipedia.org/wiki/Square_root_of_a_matrix + https://en.wikipedia.org/wiki/Square_root_of_a_matrix Adoption @@ -1039,7 +1039,7 @@ .. [#preprocessor] From a comment by GvR on a G+ post by GvR; the comment itself does not seem to be directly linkable: -https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u + https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u .. [#infix-hack] http://code.activestate.com/recipes/384122-infix-operators/ http://www.sagemath.org/doc/reference/misc/sage/misc/decorators.html#sage.misc.decorators.infix_operator .. [#scipy-conf] http://conference.scipy.org/past.html @@ -1058,7 +1058,7 @@ http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode17.html Example code is adapted from -https://github.com/rerpy/rerpy/blob/0d274f85e14c3b1625acb22aed1efa85d122ecb7/rerpy/incremental_ls.py#L202 + https://github.com/rerpy/rerpy/blob/0d274f85e14c3b1625acb22aed1efa85d122ecb7/rerpy/incremental_ls.py#L202 .. [#pycon-tutorials] Out of the 36 tutorials scheduled for PyCon 2014 (https://us.pycon.org/2014/schedule/tutorials/), we guess that the @@ -1144,7 +1144,7 @@ .. [#broadcasting] http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html .. [#matmul-other-langs] -http://mail.scipy.org/pipermail/scipy-user/2014-February/035499.html + http://mail.scipy.org/pipermail/scipy-user/2014-February/035499.html .. [#github-details] Counts were produced by manually entering the string ``"import foo"`` or ``"from foo import"`` (with quotes) into -- Repository URL: http://hg.python.org/peps From zachary.ware+pydev at gmail.com Fri Mar 14 18:08:22 2014 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Fri, 14 Mar 2014 12:08:22 -0500 Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. In-Reply-To: <1394812530.8956.94532457.00F2EB6E@webmail.messagingengine.com> References: <3flpwB0Njyz7LjX@mail.python.org> <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> <1394812530.8956.94532457.00F2EB6E@webmail.messagingengine.com> Message-ID: On Fri, Mar 14, 2014 at 10:55 AM, Benjamin Peterson wrote: > On Fri, Mar 14, 2014, at 08:52 AM, Zachary Ware wrote: >> On Fri, Mar 14, 2014 at 10:10 AM, Benjamin Peterson >> wrote: >> > Why are you readding these when they were apparently purposely removed >> > in eef7899ea7ab? >> >> It is rather ugly to build the docs in a 3.3 checkout, then update to >> default and run 'hg status' without those entries in .hgignore. I can >> understand not wanting to be bombarded by all of the untracked files, >> but it's also pretty easy to enable the purge extension and do "hg >> purge Doc/tools". > > This is why everyone should use the share extension and have separate > working copies for every branch. :) I've been using that setup for a few months now (and cursing myself for not switching sooner), but I've still found myself annoyed by this issue a time or two since the doc build chain change. I personally don't mind whether those entries are in .hgignore or not, but I don't see the harm in them sticking around (at least until 3.3 is out of maintenance; updating from 2.7 to default is a great source of problems even aside from this issue). -- Zach From solipsis at pitrou.net Fri Mar 14 18:12:30 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 18:12:30 +0100 Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. References: <3flpwB0Njyz7LjX@mail.python.org> <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> <1394812530.8956.94532457.00F2EB6E@webmail.messagingengine.com> Message-ID: <20140314181230.063e4f6a@fsol> On Fri, 14 Mar 2014 12:08:22 -0500 Zachary Ware wrote: > On Fri, Mar 14, 2014 at 10:55 AM, Benjamin Peterson wrote: > > On Fri, Mar 14, 2014, at 08:52 AM, Zachary Ware wrote: > >> On Fri, Mar 14, 2014 at 10:10 AM, Benjamin Peterson > >> wrote: > >> > Why are you readding these when they were apparently purposely removed > >> > in eef7899ea7ab? > >> > >> It is rather ugly to build the docs in a 3.3 checkout, then update to > >> default and run 'hg status' without those entries in .hgignore. I can > >> understand not wanting to be bombarded by all of the untracked files, > >> but it's also pretty easy to enable the purge extension and do "hg > >> purge Doc/tools". > > > > This is why everyone should use the share extension and have separate > > working copies for every branch. :) > > I've been using that setup for a few months now (and cursing myself > for not switching sooner), but I've still found myself annoyed by this > issue a time or two since the doc build chain change. I personally > don't mind whether those entries are in .hgignore or not, but I don't > see the harm in them sticking around (at least until 3.3 is out of > maintenance; updating from 2.7 to default is a great source of > problems even aside from this issue). Indeed, I think it's ok to keep them. Regards Antoine. From python-checkins at python.org Fri Mar 14 18:46:00 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 14 Mar 2014 18:46:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_465=3A_Updated_section_?= =?utf-8?q?=22Choice_of_operator=22=2E?= Message-ID: <3flvTN2mmXz7Ljg@mail.python.org> http://hg.python.org/peps/rev/790f47e5419b changeset: 5408:790f47e5419b user: Guido van Rossum date: Fri Mar 14 10:45:59 2014 -0700 summary: PEP 465: Updated section "Choice of operator". files: pep-0465.txt | 105 +++++++++++++++++++++++++++++++++----- 1 files changed, 89 insertions(+), 16 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -820,26 +820,99 @@ Choice of operator ------------------ -Why ``@`` instead of some other punctuation symbol? It doesn't matter -much, and there isn't any consensus across other programming languages -about how this operator should be named [#matmul-other-langs]_, but -``@`` has a few advantages: +Why ``@`` instead of some other spelling? There isn't any consensus +across other programming languages about how this operator should be +named [#matmul-other-langs]_; here we discuss the various options. -* ``@`` is a friendly character that Pythoneers are already used to - typing in decorators, and its use in email addresses means it is - more likely to be easily accessible across keyboard layouts than - some other characters (e.g. ``$`` or non-ASCII characters). +Restricting ourselves only to symbols present on US English keyboards, +the punctuation characters that don't already have a meaning in Python +expression context are: ``@``, backtick, ``$``, ``!``, and ``?``. Of +these options, ``@`` is clearly the best; ``!`` and ``?`` are already +heavily freighted with inapplicable meanings in the programming +context, backtick has been banned from Python by BDFL pronouncement +(see PEP 3099), and ``$`` is uglier, even more dissimilar to ``*`` and +:math:`\cdot`, and has Perl/PHP baggage. ``$`` is probably the +second-best option of these, though. + +Symbols which are not present on US English keyboards start at a +significant disadvantage (having to spend 5 minutes at the beginning +of every numeric Python tutorial just going over keyboard layouts is +not a hassle anyone really wants). Plus, even if we somehow overcame +the typing problem, it's not clear there are any that are actually +better than ``@``. Some options that have been suggested include: + +* U+00D7 MULTIPLICATION SIGN: ``A ? B`` +* U+22C5 DOT OPERATOR: ``A ? B`` +* U+2297 CIRCLED TIMES: ``A ? B`` +* U+00B0 DEGREE: ``A ? B`` + +What we need, though, is an operator that means "matrix +multiplication, as opposed to scalar/elementwise multiplication". +There is no conventional symbol for these in mathematics or +programming, where these operations are usually distinguished by +context. (And U+2297 CIRCLED TIMES is actually used conventionally to +mean exactly the opposite: elementwise multiplication -- the "Hadamard +product" -- as opposed to matrix multiplication). ``@`` at least has +the virtue that it *looks* like a funny non-commutative operator; a +naive user who knows maths but not programming couldn't look at ``A * +B`` versus ``A ? B``, or ``A * B`` versus ``A ? B``, or ``A * B`` +versus ``A ? B`` and guess which one is the usual multiplication, and +which one is the special case. + +Finally, there is the option of using multi-character tokens. Some +options: + +* Matlab uses a ``.*`` operator. Aside from being visually confusable + with ``*``, this would be a terrible choice for us because in + Matlab, ``*`` means matrix multiplication and ``.*`` means + elementwise multiplication, so using ``.*`` for matrix + multiplication would make us exactly backwards from what Matlab + users expect. + +* APL apparently used ``+.?``, which by combining a multi-character + token, confusing attribute-access-like . syntax, and a unicode + character, ranks somewhere below U+2603 SNOWMAN on our candidate + list. If we like the idea of combining addition and multiplication + operators as being evocative of how matrix multiplication actually + works, then something like ``+*`` could be used -- though this may + be too easy to confuse with ``*+``, which is just multiplication + combined with the unary ``+`` operator. + +* PEP 211 suggested ``~*`` and ``~**``. This has the downside that it + sort of suggests that there is a unary ``*`` operator that is being + combined with unary ``~``, but it could work. + +* R uses ``%*%`` for matrix multiplication. In R this forms part of a + general extensible infix system in which all tokens of the form + ``%foo%`` are user-defined binary operators. We could steal the + token without stealing the system. + +* Some other plausible candidates that have been suggested: ``><`` (= + ascii drawing of the multiplication sign ?); the footnote operators + ``[*]`` and ``[**]`` or ``|*|`` and ``|**|`` (but when used in + context, the use of vertical grouping symbols tends to recreate the + nested parentheses visual clutter that was noted as one of the major + downsides of the function syntax we're trying to get away from); + ``^*`` and ``^^``. + +So, it doesn't matter much, but ``@`` seems as good or better than any +of the alternatives: + +* It's a friendly character that Pythoneers are already used to typing + in decorators, but the decorator usage and the math expression + usage are sufficiently dissimilar that it would be hard to confuse + them in practice. + +* It's widely accessible across keyboard layouts (and thanks to its + use in email addresses, this is true even of weird keyboards like + those in phones). + +* It's round like ``*`` and :math:`\cdot`. * The mATrices mnemonic is cute. -* It's round like ``*`` and :math:`\cdot`. - -* The use of a single-character token makes ``@@`` possible, which is - a nice bonus. - -* The swirly shape is reminiscent of the simultaneous sweeps over rows - and columns that define matrix multiplication; its asymmetry is - evocative of its non-commutative nature. +* The use of a single-character token reduces the line-noise effect, + and makes ``@@`` possible, which is a nice bonus. (Non)-Definitions for built-in types -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Fri Mar 14 21:17:13 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 14 Mar 2014 21:17:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_put_python-version_after_crea?= =?utf-8?q?ted?= Message-ID: <3flyqs4ZpKz7LjN@mail.python.org> http://hg.python.org/peps/rev/b7cdb9a7e146 changeset: 5409:b7cdb9a7e146 user: Benjamin Peterson date: Fri Mar 14 15:17:05 2014 -0500 summary: put python-version after created files: pep-0465.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -5,9 +5,9 @@ Author: Nathaniel J. Smith Status: Draft Type: Standards Track -Python-Version: 3.5 Content-Type: text/x-rst Created: 20-Feb-2014 +Python-Version: 3.5 Post-History: 13-Mar-2014 Abstract -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Fri Mar 14 23:31:23 2014 From: python-checkins at python.org (r.david.murray) Date: Fri, 14 Mar 2014 23:31:23 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_final_edit_pas?= =?utf-8?q?s=2E?= Message-ID: <3fm1pg6ddQz7LjX@mail.python.org> http://hg.python.org/cpython/rev/cc27535a7c00 changeset: 89655:cc27535a7c00 user: R David Murray date: Fri Mar 14 18:31:07 2014 -0400 summary: whatsnew: final edit pass. files: Doc/includes/email-alternative-new-api.py | 2 +- Doc/library/exceptions.rst | 1 - Doc/library/smtplib.rst | 3 + Doc/whatsnew/3.4.rst | 179 +++++---- 4 files changed, 94 insertions(+), 91 deletions(-) diff --git a/Doc/includes/email-alternative-new-api.py b/Doc/includes/email-alternative-new-api.py --- a/Doc/includes/email-alternative-new-api.py +++ b/Doc/includes/email-alternative-new-api.py @@ -19,7 +19,7 @@ [1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718 ---?ric +--Pep? """) # Add the html version. This converts the message into a multipart/alternative diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -265,7 +265,6 @@ :exc:`mmap.error` have been merged into :exc:`OSError`. .. versionchanged:: 3.4 - The :attr:`filename` attribute is now the original file name passed to the function, instead of the name encoded to or decoded from the filesystem encoding. Also, the :attr:`filename2` attribute was added. diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -117,6 +117,9 @@ Subclass of :exc:`OSError` that is the base exception class for all the other exceptions provided by this module. + .. versionchanged:: 3.4 + SMTPException became subclass of :exc:`OSError` + .. exception:: SMTPServerDisconnected diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -406,10 +406,10 @@ ``__main__.__file__`` when a script has been executed directly using a relative path (Contributed by Brett Cannon in :issue:`18416`). -* Now all the UTF-\* codecs (except UTF-7) reject surrogates during both +* All the UTF-\* codecs (except UTF-7) now reject surrogates during both encoding and decoding unless the ``surrogatepass`` error handler is used, - with the exception of the UTF-16 decoder that accepts valid surrogate pairs, - and the UTF-16 encoder that produces them while encoding non-BMP characters. + with the exception of the UTF-16 decoder (which accepts valid surrogate pairs) + and the UTF-16 encoder (which produces them while encoding non-BMP characters). Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in :issue:`12892`. @@ -563,7 +563,7 @@ The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to trace memory blocks allocated by Python. It provides the following information: -* Traceback where an object was allocated +* Trace where an object was allocated * Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks * Compute the differences between two snapshots to detect memory leaks @@ -643,7 +643,7 @@ :func:`~base64.b85encode`, and :func:`~base64.b85decode` provide the ability to encode and decode binary data from and to ``Ascii85`` and the git/mercurial ``Base85`` formats, respectively. The ``a85`` functions have options that can -be sued to make them compatible with the variants of the ``Ascii85`` encoding, +be used to make them compatible with the variants of the ``Ascii85`` encoding, including the Adobe variant. (Contributed by Martin Morrison, the Mercurial project, Serhiy Storchaka, and Antoine Pitrou in :issue:`17618`.) @@ -861,15 +861,15 @@ :pep:`443` -- Single-dispatch generic functions PEP written and implemented by ?ukasz Langa. +:func:`~functools.total_ordering` now supports a return value of +:const:`NotImplemented` from the underlying comparison function. (Contributed +by Katie Miller in :issue:`10042`.) + A pure-python version of the :func:`~functools.partial` function is now in the stdlib; in CPython it is overridden by the C accelerated version, but it is available for other implementations to use. (Contributed by Brian Thorne in :issue:`12428`.) -:func:`~functools.total_ordering` now supports a return value of -:const:`NotImplemented` from the underlying comparison function. (Contributed -by Katie Miller in :issue:`10042`.) - gc -- @@ -1093,29 +1093,17 @@ mmap ---- -mmap objects can now be weakref'ed. (Contributed by Valerie Lambert in +mmap objects can now be :mod:`weakref`\ ed. (Contributed by Valerie Lambert in :issue:`4885`.) -mock ----- - -:mod:`~unittest.mock` objects now inspect their specification signatures when -matching calls, which means an argument can now be matched by either position -or name, instead of only by position. (Contributed by Antoine Pitrou in -:issue:`17015`.) - -:func:`~mock.mock_open` objects now have ``readline`` and ``readlines`` -methods. (Contributed by Toshio Kuratomi in :issue:`17467`.) - - multiprocessing --------------- .. _whatsnew-multiprocessing-no-fork: -On Unix, two new :ref:`start methods ` -(``spawn`` and ``forkserver``) have been added for starting processes using +On Unix two new :ref:`start methods `, +(``spawn`` and ``forkserver``, have been added for starting processes using :mod:`multiprocessing`. These make the mixing of processes with threads more robust, and the ``spawn`` method matches the semantics that multiprocessing has always used on Windows. New function @@ -1141,7 +1129,7 @@ :mod:`multiprocessing` now relies on :mod:`runpy` (which implements the ``-m`` switch) to initialise ``__main__`` appropriately in child processes when using the ``spawn`` or ``forkserver`` start methods. This resolves some -edge cases where combining multiprocessing, the ``-m`` command line switch +edge cases where combining multiprocessing, the ``-m`` command line switch, and explicit relative imports could cause obscure failures in child processes. (Contributed by Nick Coghlan in :issue:`19946`) @@ -1149,29 +1137,29 @@ operator -------- -There is now a pure-python version of the :mod:`operator` module available for -reference and for use by alternate implementations of Python. (Contributed by -Zachary Ware in :issue:`16694`.) - New function :func:`~operator.length_hint` provides an implementation of the specification for how the :meth:`~object.__length_hint__` special method should be used, as part of the :pep:`424` formal specification of this language feature. (Contributed by Armin Ronacher in :issue:`16148`.) +There is now a pure-python version of the :mod:`operator` module available for +reference and for use by alternate implementations of Python. (Contributed by +Zachary Ware in :issue:`16694`.) + os -- -New functions to get and set the :ref:`inheritable flag ` of a file -descriptors or a Windows handle: - -* :func:`os.get_inheritable`, :func:`os.set_inheritable` -* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` - -The :mod:`os` module now provides a :func:`~os.cpu_count` function, analogous to -the :func:`multiprocessing.cpu_count` function (which is now implemented in -terms of the new :mod:`os` function). (Contributed by Trent Nelson, Yogesh -Chaudhari, Victor Stinner, and Charles-Fran?ois Natali in :issue:`17914`.) +There are new functions to get and set the :ref:`inheritable flag +` of a file descriptor (:func:`os.get_inheritable`, +:func:`os.set_inheritable`) or a Windows handle +(:func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`). + +New function :func:`~os.cpu_count` reports the number of CPUs available on the +platform on which Python is running (or ``None`` if the count can't be +determined). The :func:`multiprocessing.cpu_count` function is now implemented +in terms of this function). (Contributed by Trent Nelson, Yogesh Chaudhari, +Victor Stinner, and Charles-Fran?ois Natali in :issue:`17914`.) :func:`os.path.samestat` is now available on the Windows platform (and the :func:`os.path.samefile` implementation is now shared between Unix and @@ -1181,10 +1169,10 @@ root on Windows. (Contributed by Tim Golden in :issue:`9035`.) :func:`os.open` supports two new flags on platforms that provide them, -:data:`~os.O_TMPFILE` (unnamed temporary file, as of 3.4.0 release available -only on Linux systems with a kernel version of 3.11 or newer that have uapi -headers), and :data:`~os.O_PATH` (un-opened file descriptor). (Contributed by -Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) +:data:`~os.O_PATH` (un-opened file descriptor), and :data:`~os.O_TMPFILE` +(unnamed temporary file; as of 3.4.0 release available only on Linux systems +with a kernel version of 3.11 or newer that have uapi headers). (Contributed +by Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) pdb @@ -1214,7 +1202,7 @@ :mod:`pickle` now supports (but does not use by default) a new pickle protocol, protocol 4. This new protocol addresses a number of issues that were present in previous protocols, such as the serialization of nested classes, very large -strings and containers, or classes whose :meth:`__new__` method takes +strings and containers, and classes whose :meth:`__new__` method takes keyword-only arguments. It also provides some efficiency improvements. .. seealso:: @@ -1269,11 +1257,11 @@ pydoc ----- -The :mod:`pydoc` module is now based directly on the -:func:`inspect.signature` introspection API, allowing it to provide -signature information for a wider variety of callable objects. This change -also means that ``__wrapped__`` attributes are now taken into account when -display help information (Contributed by Larry Hastings in :issue:`19674`) +The :mod:`pydoc` module is now based directly on the :func:`inspect.signature` +introspection API, allowing it to provide signature information for a wider +variety of callable objects. This change also means that ``__wrapped__`` +attributes are now taken into account when displaying help information +(Contributed by Larry Hastings in :issue:`19674`) The :mod:`pydoc` module no longer displays the ``self`` parameter for already bound methods. Instead, it aims to always display the exact current @@ -1362,7 +1350,7 @@ ----- The :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now -accept a *map* keyword argument, which if specified is passed in to +accept a *map* keyword argument which, if specified, is passed in to :class:`asynchat.async_chat` as its *map* argument. This allows an application to avoid affecting the global socket map. (Contributed by Vinay Sajip in :issue:`11959`.) @@ -1374,7 +1362,7 @@ :exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows both socket level errors and SMTP protocol level errors to be caught in one try/except statement by code that only cares whether or not an error occurred. -(:issue:`2118`). +(Contributed by Ned Jackson Lovely in :issue:`2118`). socket @@ -1400,10 +1388,10 @@ sqlite3 ------- -A new boolean parameter, *uri*, to the :func:`~sqlite3.connect` function can -be used to indicate that the *database* parameter is a ``uri`` (see -the `SQLite URI documentation `_). -(Contributed by poq in :issue:`13773`.) +A new boolean parameter to the :func:`~sqlite3.connect` function, *uri*, can be +used to indicate that the *database* parameter is a ``uri`` (see the `SQLite +URI documentation `_). (Contributed by poq in +:issue:`13773`.) ssl @@ -1418,6 +1406,18 @@ .. _whatsnew34-sslcontext: +New function :func:`~ssl.create_default_context` provides a standard way to +obtain an :class:`~ssl.SSLContext` whose settings are intended to be a +reasonable balance between compatibility and security. These settings are +more stringent than the defaults provided by the :class:`~ssl.SSLContext` +constructor, and may be adjusted in the future, without prior deprecation, if +best-practice security requirements change. The new recommended best +practice for using stdlib libraries that support SSL is to use +:func:`~ssl.create_default_context` to obtain an :class:`~ssl.SSLContext` +object, modify it if needed, and then pass it as the *context* argument +of the appropriate stdlib API. (Contributed by Christian Heimes +in :issue:`19689`.) + :class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_verify_locations` accepts a new optional argument *cadata*, which can be used to provide PEM or DER encoded certificates directly via strings or bytes, respectively. @@ -1435,7 +1435,7 @@ ``X.509`` certs, ``X.509 CA`` certs, and certificate revocation lists (``crl``\ s), as well as a :meth:`~ssl.SSLContext.get_ca_certs` method that returns a list of the loaded ``CA`` certificates. (Contributed by Christian Heimes in -and :issue:`18147`.) +:issue:`18147`.) If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has an new attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the @@ -1454,14 +1454,6 @@ (``purpose=``:data:`~ssl.Purpose.CLIENT_AUTH`). (Contributed by Christian Heimes in :issue:`19292`.) -New function :func:`~ssl.create_default_context` provides a standard way to -obtain an :class:`~ssl.SSLContext` whose settings are intended to be a -reasonable balance between compatibility and security. These settings are -more stringent than the defaults provided by the :class:`~ssl.SSLContext` -constructor, and may be adjusted in the future, without prior deprecation, if -best-practice security requirements change. (Contributed by Christian Heimes -in :issue:`19689`.) - .. _whatsnew34-win-cert-store: Two new windows-only functions, :func:`~ssl.enum_certificates` and @@ -1485,19 +1477,20 @@ The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C implementation is required as most of the values aren't standardized and -platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) - -The module supports new file types: door, event port and whiteout. +are platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) + +The module supports new :mod:`~stat.ST_MODE` flags, :mod:`~stat.S_IFDOOR`, +:attr:`~stat.S_IFPORT`, and :attr:`~stat.S_IFWHT`. (Contributed by +Christian Hiemes in :issue:`11016`.) struct ------ -:mod:`struct` now supports the streamed unpacking of a buffer containing -repeated instances of a given format of data. Both a module level -:mod:`~struct.iter_unpack` function and a :meth:`struct.Struct.iter_unpack` -method on compiled formats have been added. (Contributed by Antoine Pitrou in -:issue:`17804`.) +New function :mod:`~struct.iter_unpack` and a new +:meth:`struct.Struct.iter_unpack` method on compiled formats provide streamed +unpacking of a buffer containing repeated instances of a given format of data. +(Contributed by Antoine Pitrou in :issue:`17804`.) subprocess @@ -1508,8 +1501,8 @@ (Contributed by Zack Weinberg in :issue:`16624`.) :func:`~subprocess.getstatus` and :func:`~subprocess.getstatusoutput` now -work on Windows. This change was actually made in 3.3.4. (Contributed -by Tim Golden in :issue:`10197`.) +work on Windows. This change was actually inadvertently made in 3.3.4. +(Contributed by Tim Golden in :issue:`10197`.) sunau @@ -1536,9 +1529,9 @@ --- New function :func:`sys.getallocatedblocks` returns the current number of -blocks allocated by the interpreter (in CPython with the default +blocks allocated by the interpreter. (In CPython with the default ``--with-pymalloc`` setting, this is allocations made through the -:c:func:`PyObject_Malloc` API). This can be useful for tracking memory leaks, +:c:func:`PyObject_Malloc` API.) This can be useful for tracking memory leaks, especially if automated via a test suite. (Contributed by Antoine Pitrou in :issue:`13390`.) @@ -1692,6 +1685,14 @@ Test discovery now works with namespace packages (Contributed by Claudiu Popa in :issue:`17457`.) +:mod:`unittest.mock` objects now inspect their specification signatures when +matching calls, which means an argument can now be matched by either position +or name, instead of only by position. (Contributed by Antoine Pitrou in +:issue:`17015`.) + +:func:`~mock.mock_open` objects now have ``readline`` and ``readlines`` +methods. (Contributed by Toshio Kuratomi in :issue:`17467`.) + venv ---- @@ -1844,7 +1845,7 @@ implemented in C. Some standard library extension modules have been converted to use Argument -Clinic in Python 3.4, and :mod:`pydoc` and :mod:`inspect` has been updated +Clinic in Python 3.4, and :mod:`pydoc` and :mod:`inspect` have been updated accordingly. It is expected that signature metadata for programmatic introspection will @@ -1879,7 +1880,7 @@ marked as accepting ``const char *`` rather than ``char *`` (Contributed by Serhiy Storchaka in :issue:`1772673`). -* New shell version of ``python-config``; can be used even when a python +* A new shell version of ``python-config`` can be used even when a python interpreter is not available (for example, in cross compilation scenarios). * :c:func:`PyUnicode_FromFormat` now supports width and precision @@ -2086,8 +2087,9 @@ * The :mod:`formatter` module is pending deprecation and is slated for removal in Python 3.6. -* MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will - require an explicit digest name or constructor as *digestmod* argument. +* ``MD5`` as the default *digestmod* for the :func:`hmac.new` function is + deprecated. Python 3.6 will require an explicit digest name or constructor as + *digestmod* argument. * The internal ``Netrc`` class in the :mod:`ftplib` module has been documented as deprecated in its docstring for quite some time. It now emits a @@ -2099,9 +2101,6 @@ * The *strict* argument of :class:`~html.parser.HTMLParser` is deprecated. -* The :func:`hmac.new` *digestmod* keyword having a default value (currently - ``MD5``) is deprecated. - * The :mod:`plistlib` :func:`~plistlib.readPlist`, :func:`~plistlib.writePlist`, :func:`~plistlib.readPlistFromBytes`, and :func:`~plistlib.writePlistToBytes` functions are deprecated in favor of the @@ -2117,7 +2116,7 @@ appropriate uses of :class:`io.TextIOWrapper` (if needed) and its *newline* argument. -* The *parser* argument of :func:`~xml.etree.ElementTree.iterparse` has +* The *parser* argument of :func:`xml.etree.ElementTree.iterparse` has been deprecated, as has the *html* argument of :func:`~xml.etree.ElementTree.XMLParser`. To prepare for the removal of the latter, all arguments to ``XMLParser`` should be passed by keyword. @@ -2308,7 +2307,7 @@ no ``__wrapped__`` attribute. * :func:`inspect.getfullargspec` has been reimplemented on top of - :func`inspect.signature` and hence handles a much wider variety of callable + :func:`inspect.signature` and hence handles a much wider variety of callable objects than it did in the past. It is expected that additional builtin and extension module callables will gain signature metadata over the course of the Python 3.4 series. Code that assumes that @@ -2321,8 +2320,10 @@ iterating through :data:`sys.path_importer_cache` based on :data:`sys.path` will not find all keys. A module's ``__file__`` when imported in the current working directory will also now have an absolute path, including when using - ``-m`` with the interpreter (this does not influence when the path to a file - is specified on the command-line) (:issue:`18416`). + ``-m`` with the interpreter (except for ``__main__.__file__`` when a script + has been executed directly using a relative path) (Contributed by Brett + Cannon in :issue:`18416`). is specified on the command-line) + (:issue:`18416`). * The removal of the *strict* argument to :class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` changes the meaning of the @@ -2383,7 +2384,7 @@ perpetually alive (for example, database connections kept in thread-local storage). (:issue:`17094`.) -* Parameter names in ``__annotations__`` dict are now mangled properly, +* Parameter names in ``__annotations__`` dicts are now mangled properly, similarly to ``__kwdefaults__``. (Contributed by Yury Selivanov in :issue:`20625`). @@ -2408,7 +2409,7 @@ * The :func:`re.split`, :func:`re.findall`, and :func:`re.sub` functions, and the :meth:`~re.match.group` and :meth:`~re.match.groups` methods of - :class:``match`` objects now always return a *bytes* object when the string + ``match`` objects now always return a *bytes* object when the string to be matched is a :term:`bytes-like object`. Previously the return type matched the input type, so if your code was depending on the return value being, say, a ``bytearray``, you will need to change your code. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 00:13:10 2014 From: python-checkins at python.org (vinay.sajip) Date: Sat, 15 Mar 2014 00:13:10 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Reverted_change_introduced?= =?utf-8?q?_in_edb12dad7bf6=2E?= Message-ID: <3fm2kt0VwWz7LjP@mail.python.org> http://hg.python.org/cpython/rev/63d2d0c15b37 changeset: 89656:63d2d0c15b37 user: Vinay Sajip date: Fri Mar 14 23:13:00 2014 +0000 summary: Reverted change introduced in edb12dad7bf6. files: .hgignore | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -24,11 +24,6 @@ reflog.txt$ tags$ Lib/plat-mac/errors.rsrc.df.rsrc -Doc/tools/sphinx/ -Doc/tools/docutils/ -Doc/tools/jinja/ -Doc/tools/jinja2/ -Doc/tools/pygments/ Misc/python.pc Misc/python-config.sh$ Modules/Setup$ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 02:15:37 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 02:15:37 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_cast_negative_numbers_to_s?= =?utf-8?q?ize=5Ft_before_shifting_them_=28=2320929=29?= Message-ID: <3fm5S921mfz7LkP@mail.python.org> http://hg.python.org/cpython/rev/e68f156ea0e6 changeset: 89657:e68f156ea0e6 user: Benjamin Peterson date: Fri Mar 14 20:15:29 2014 -0500 summary: cast negative numbers to size_t before shifting them (#20929) files: Include/objimpl.h | 2 +- Misc/NEWS | 2 ++ 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Include/objimpl.h b/Include/objimpl.h --- a/Include/objimpl.h +++ b/Include/objimpl.h @@ -265,7 +265,7 @@ #define _PyGCHead_REFS(g) ((g)->gc.gc_refs >> _PyGC_REFS_SHIFT) #define _PyGCHead_SET_REFS(g, v) do { \ (g)->gc.gc_refs = ((g)->gc.gc_refs & ~_PyGC_REFS_MASK) \ - | (v << _PyGC_REFS_SHIFT); \ + | (((size_t)(v)) << _PyGC_REFS_SHIFT); \ } while (0) #define _PyGCHead_DECREF(g) ((g)->gc.gc_refs -= 1 << _PyGC_REFS_SHIFT) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -8,6 +8,8 @@ Core and Builtins ----------------- +- Issue #20929: Add a type cast to avoid shifting a negative number. + - Issue #20731: Properly position in source code files even if they are opened in text mode. Patch by Serhiy Storchaka. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 03:47:59 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 03:47:59 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogZG9uJ3QgZG8gcG9p?= =?utf-8?q?nter_arithmetic_with_signed_numbers?= Message-ID: <3fm7Vl329mz7LjP@mail.python.org> http://hg.python.org/cpython/rev/d1b392141123 changeset: 89658:d1b392141123 branch: 3.3 parent: 89647:d7950e916f20 user: Benjamin Peterson date: Fri Mar 14 21:47:23 2014 -0500 summary: don't do pointer arithmetic with signed numbers files: Objects/longobject.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -36,7 +36,8 @@ static PyObject * get_small_int(sdigit ival) { - PyObject *v = (PyObject*)(small_ints + ival + NSMALLNEGINTS); + assert(-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS); + PyObject *v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; Py_INCREF(v); #ifdef COUNT_ALLOCS if (ival >= 0) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 03:48:00 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 03:48:00 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fm7Vm4kJWz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/061a9679ea2b changeset: 89659:061a9679ea2b parent: 89657:e68f156ea0e6 parent: 89658:d1b392141123 user: Benjamin Peterson date: Fri Mar 14 21:47:36 2014 -0500 summary: merge 3.3 files: Objects/longobject.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -37,7 +37,8 @@ static PyObject * get_small_int(sdigit ival) { - PyObject *v = (PyObject*)(small_ints + ival + NSMALLNEGINTS); + assert(-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS); + PyObject *v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; Py_INCREF(v); #ifdef COUNT_ALLOCS if (ival >= 0) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 03:55:05 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 03:55:05 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fix_c89_declar?= =?utf-8?q?ation_order?= Message-ID: <3fm7fx3W2Wz7LjR@mail.python.org> http://hg.python.org/cpython/rev/8877c6b39461 changeset: 89660:8877c6b39461 branch: 3.3 parent: 89658:d1b392141123 user: Benjamin Peterson date: Fri Mar 14 21:53:51 2014 -0500 summary: fix c89 declaration order files: Objects/longobject.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -36,8 +36,9 @@ static PyObject * get_small_int(sdigit ival) { + PyObject *v; assert(-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS); - PyObject *v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; + v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; Py_INCREF(v); #ifdef COUNT_ALLOCS if (ival >= 0) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 03:55:06 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 03:55:06 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_avoid_referenc?= =?utf-8?q?ing_out-of-bounds_memory?= Message-ID: <3fm7fy5FWfz7LjR@mail.python.org> http://hg.python.org/cpython/rev/ea27f0ed741f changeset: 89661:ea27f0ed741f branch: 3.3 user: Benjamin Peterson date: Fri Mar 14 21:54:31 2014 -0500 summary: avoid referencing out-of-bounds memory files: Objects/listobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -2036,7 +2036,7 @@ if (keys != NULL) { for (i = 0; i < saved_ob_size; i++) Py_DECREF(keys[i]); - if (keys != &ms.temparray[saved_ob_size+1]) + if (saved_ob_size >= MERGESTATE_TEMP_SIZE/2) PyMem_FREE(keys); } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 03:55:08 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 03:55:08 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fm7g001NTz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/7ce22d0899e4 changeset: 89662:7ce22d0899e4 parent: 89659:061a9679ea2b parent: 89661:ea27f0ed741f user: Benjamin Peterson date: Fri Mar 14 21:54:41 2014 -0500 summary: merge 3.3 files: Objects/listobject.c | 2 +- Objects/longobject.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -2043,7 +2043,7 @@ if (keys != NULL) { for (i = 0; i < saved_ob_size; i++) Py_DECREF(keys[i]); - if (keys != &ms.temparray[saved_ob_size+1]) + if (saved_ob_size >= MERGESTATE_TEMP_SIZE/2) PyMem_FREE(keys); } diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -37,8 +37,9 @@ static PyObject * get_small_int(sdigit ival) { + PyObject *v; assert(-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS); - PyObject *v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; + v = (PyObject *)&small_ints[ival + NSMALLNEGINTS]; Py_INCREF(v); #ifdef COUNT_ALLOCS if (ival >= 0) -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Mar 15 09:51:44 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 15 Mar 2014 09:51:44 +0100 Subject: [Python-checkins] Daily reference leaks (e68f156ea0e6): sum=0 Message-ID: results for e68f156ea0e6 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_site leaked [-2, 2, -2] references, sum=-2 test_site leaked [-2, 2, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogA6ySLy', '-x'] From python-checkins at python.org Sat Mar 15 17:03:25 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 15 Mar 2014 17:03:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzIwOTMzOiBBdCBs?= =?utf-8?q?east_one_place_maps_=27test=27_to_=27localhost=27=2E=2E=2Efix_t?= =?utf-8?q?est=2E?= Message-ID: <3fmT8Y29Mcz7LjM@mail.python.org> http://hg.python.org/cpython/rev/6a78bbd0fda4 changeset: 89663:6a78bbd0fda4 branch: 3.3 parent: 89661:ea27f0ed741f user: R David Murray date: Sat Mar 15 12:00:14 2014 -0400 summary: #20933: At least one place maps 'test' to 'localhost'...fix test. Discovery and patch by Wenzhu Man. University of Waterloo apparently maps the local name 'test' to localhost, which is in the bypass list, causing the test to fail. So change 'test' to a name unlikely to get mapped to localhost. files: Lib/test/test_urllib2.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1209,7 +1209,8 @@ self.assertTrue(_proxy_bypass_macosx_sysconf(host, bypass), 'expected bypass of %s to be True' % host) # Check hosts that should not trigger the proxy bypass - for host in ('abc.foo.bar', 'bar.com', '127.0.0.2', '10.11.0.1', 'test'): + for host in ('abc.foo.bar', 'bar.com', '127.0.0.2', '10.11.0.1', + 'notinbypass'): self.assertFalse(_proxy_bypass_macosx_sysconf(host, bypass), 'expected bypass of %s to be False' % host) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 17:03:26 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 15 Mar 2014 17:03:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2320933=3A_At_least_one_place_maps_=27test=27_?= =?utf-8?b?dG8gJ2xvY2FsaG9zdCcuLi5maXggdGVzdC4=?= Message-ID: <3fmT8Z5gnnz7Ljq@mail.python.org> http://hg.python.org/cpython/rev/586de8d96817 changeset: 89664:586de8d96817 parent: 89662:7ce22d0899e4 parent: 89663:6a78bbd0fda4 user: R David Murray date: Sat Mar 15 12:03:02 2014 -0400 summary: Merge: #20933: At least one place maps 'test' to 'localhost'...fix test. files: Lib/test/test_urllib2.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1227,7 +1227,8 @@ self.assertTrue(_proxy_bypass_macosx_sysconf(host, bypass), 'expected bypass of %s to be True' % host) # Check hosts that should not trigger the proxy bypass - for host in ('abc.foo.bar', 'bar.com', '127.0.0.2', '10.11.0.1', 'test'): + for host in ('abc.foo.bar', 'bar.com', '127.0.0.2', '10.11.0.1', + 'notinbypass'): self.assertFalse(_proxy_bypass_macosx_sysconf(host, bypass), 'expected bypass of %s to be False' % host) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 17:50:25 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 17:50:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_remove_unused_?= =?utf-8?q?zero_constants?= Message-ID: <3fmVBn3Krsz7LjP@mail.python.org> http://hg.python.org/cpython/rev/a3f80c22a441 changeset: 89665:a3f80c22a441 branch: 3.3 parent: 89663:6a78bbd0fda4 user: Benjamin Peterson date: Sat Mar 15 11:49:49 2014 -0500 summary: remove unused zero constants files: Modules/_decimal/_decimal.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -3928,9 +3928,6 @@ return ret; } -static mpd_uint_t data_zero[1] = {0}; -static const mpd_t zero = {MPD_STATIC|MPD_CONST_DATA, 0, 1, 1, 1, data_zero}; - static PyObject * nm_mpd_qpow(PyObject *base, PyObject *exp, PyObject *mod) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 17:50:26 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 17:50:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_only_define_ze?= =?utf-8?q?ro_constant_when_it_is_used?= Message-ID: <3fmVBp57K7z7Ljn@mail.python.org> http://hg.python.org/cpython/rev/3f93907ab8e1 changeset: 89666:3f93907ab8e1 branch: 3.3 user: Benjamin Peterson date: Sat Mar 15 11:50:00 2014 -0500 summary: only define zero constant when it is used files: Modules/_math.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Modules/_math.c b/Modules/_math.c --- a/Modules/_math.c +++ b/Modules/_math.c @@ -22,7 +22,9 @@ static const double ln2 = 6.93147180559945286227E-01; static const double two_pow_m28 = 3.7252902984619141E-09; /* 2**-28 */ static const double two_pow_p28 = 268435456.0; /* 2**28 */ +#ifndef Py_NAN static const double zero = 0.0; +#endif /* acosh(x) * Method : -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 17:50:27 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 17:50:27 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fmVBq6xXtz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/85cb7c37503e changeset: 89667:85cb7c37503e parent: 89664:586de8d96817 parent: 89666:3f93907ab8e1 user: Benjamin Peterson date: Sat Mar 15 11:50:14 2014 -0500 summary: merge 3.3 files: Modules/_decimal/_decimal.c | 3 --- Modules/_math.c | 2 ++ 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -3928,9 +3928,6 @@ return ret; } -static mpd_uint_t data_zero[1] = {0}; -static const mpd_t zero = {MPD_STATIC|MPD_CONST_DATA, 0, 1, 1, 1, data_zero}; - static PyObject * nm_mpd_qpow(PyObject *base, PyObject *exp, PyObject *mod) { diff --git a/Modules/_math.c b/Modules/_math.c --- a/Modules/_math.c +++ b/Modules/_math.c @@ -22,7 +22,9 @@ static const double ln2 = 6.93147180559945286227E-01; static const double two_pow_m28 = 3.7252902984619141E-09; /* 2**-28 */ static const double two_pow_p28 = 268435456.0; /* 2**28 */ +#ifndef Py_NAN static const double zero = 0.0; +#endif /* acosh(x) * Method : -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 18:22:03 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 18:22:03 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_avoid_referenc?= =?utf-8?q?ing_past_the_bounds_of_an_array?= Message-ID: <3fmVvH5CTQz7LjW@mail.python.org> http://hg.python.org/cpython/rev/f5e9495df22f changeset: 89668:f5e9495df22f branch: 3.3 parent: 89666:3f93907ab8e1 user: Benjamin Peterson date: Sat Mar 15 12:21:28 2014 -0500 summary: avoid referencing past the bounds of an array files: Objects/listobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -1963,7 +1963,7 @@ if (keys[i] == NULL) { for (i=i-1 ; i>=0 ; i--) Py_DECREF(keys[i]); - if (keys != &ms.temparray[saved_ob_size+1]) + if (saved_ob_size >= MERGESTATE_TEMP_SIZE/2) PyMem_FREE(keys); goto keyfunc_fail; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 18:22:05 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 18:22:05 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fmVvK0Mfxz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/5c1bacba828d changeset: 89669:5c1bacba828d parent: 89667:85cb7c37503e parent: 89668:f5e9495df22f user: Benjamin Peterson date: Sat Mar 15 12:21:47 2014 -0500 summary: merge 3.3 files: Objects/listobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -1970,7 +1970,7 @@ if (keys[i] == NULL) { for (i=i-1 ; i>=0 ; i--) Py_DECREF(keys[i]); - if (keys != &ms.temparray[saved_ob_size+1]) + if (saved_ob_size >= MERGESTATE_TEMP_SIZE/2) PyMem_FREE(keys); goto keyfunc_fail; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 18:29:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 18:29:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_remove_runtime?= =?utf-8?q?=5Flibrary=5Fdirs_for_=5Fsqlite=3B_it_isn=27t_needed?= Message-ID: <3fmW3x73nSz7LjW@mail.python.org> http://hg.python.org/cpython/rev/7cb8faf45539 changeset: 89670:7cb8faf45539 branch: 3.3 parent: 89668:f5e9495df22f user: Benjamin Peterson date: Sat Mar 15 12:29:04 2014 -0500 summary: remove runtime_library_dirs for _sqlite; it isn't needed files: setup.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1128,7 +1128,6 @@ define_macros=sqlite_defines, include_dirs=include_dirs, library_dirs=sqlite_libdir, - runtime_library_dirs=sqlite_libdir, extra_link_args=sqlite_extra_link_args, libraries=["sqlite3",])) else: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 18:29:35 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 18:29:35 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_remove_runtime?= =?utf-8?q?=5Flibrary=5Fdirs_for_=5Fsqlite=3B_it_isn=27t_needed?= Message-ID: <3fmW3z1nkPz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/71da05fa2d8a changeset: 89671:71da05fa2d8a branch: 2.7 parent: 89642:67ada6ab7fe2 user: Benjamin Peterson date: Sat Mar 15 12:29:04 2014 -0500 summary: remove runtime_library_dirs for _sqlite; it isn't needed files: setup.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1187,7 +1187,6 @@ include_dirs=["Modules/_sqlite", sqlite_incdir], library_dirs=sqlite_libdir, - runtime_library_dirs=sqlite_libdir, extra_link_args=sqlite_extra_link_args, libraries=["sqlite3",])) else: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 18:29:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 18:29:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fmW403SDTz7Ljw@mail.python.org> http://hg.python.org/cpython/rev/fe0ad06da28c changeset: 89672:fe0ad06da28c parent: 89669:5c1bacba828d parent: 89670:7cb8faf45539 user: Benjamin Peterson date: Sat Mar 15 12:29:21 2014 -0500 summary: merge 3.3 files: setup.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1138,7 +1138,6 @@ define_macros=sqlite_defines, include_dirs=include_dirs, library_dirs=sqlite_libdir, - runtime_library_dirs=sqlite_libdir, extra_link_args=sqlite_extra_link_args, libraries=["sqlite3",])) else: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:19:53 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 15 Mar 2014 21:19:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwOTM5?= =?utf-8?q?=3A_Fix_test=5Fgeturl_failure_in_test=5Furllibnet_due_to?= Message-ID: <3fmZrT6cCHz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/9b02f23bc129 changeset: 89673:9b02f23bc129 branch: 2.7 parent: 89671:71da05fa2d8a user: Ned Deily date: Sat Mar 15 13:14:15 2014 -0700 summary: Issue #20939: Fix test_geturl failure in test_urllibnet due to new redirect of http://www.python.org/ to https://www.python.org. files: Lib/test/test_urllibnet.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -92,7 +92,7 @@ def test_geturl(self): # Make sure same URL as opened is returned by geturl. - URL = "http://www.python.org/" + URL = "https://www.python.org/" open_url = self.urlopen(URL) try: gotten_url = open_url.geturl() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -316,6 +316,9 @@ - Issue #20605: Make test_socket getaddrinfo OS X segfault test more robust. +- Issue #20939: Fix test_geturl failure in test_urllibnet due to + new redirect of http://www.python.org/ to https://www.python.org. + Documentation ------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:19:55 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 15 Mar 2014 21:19:55 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwOTM5?= =?utf-8?q?=3A_Fix_test=5Fgeturl_failure_in_test=5Furllibnet_due_to?= Message-ID: <3fmZrW24vQz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/030efedd70ca changeset: 89674:030efedd70ca branch: 3.3 parent: 89670:7cb8faf45539 user: Ned Deily date: Sat Mar 15 13:15:31 2014 -0700 summary: Issue #20939: Fix test_geturl failure in test_urllibnet due to new redirect of http://www.python.org/ to https://www.python.org. files: Lib/test/test_urllibnet.py | 2 +- Misc/NEWS | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -80,7 +80,7 @@ def test_geturl(self): # Make sure same URL as opened is returned by geturl. - URL = "http://www.python.org/" + URL = "https://www.python.org/" with self.urlopen(URL) as open_url: gotten_url = open_url.geturl() self.assertEqual(gotten_url, URL) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,12 @@ as documented. The pattern and source keyword parameters are left as deprecated aliases. +Tests +----- + +- Issue #20939: Fix test_geturl failure in test_urllibnet due to + new redirect of http://www.python.org/ to https://www.python.org. + What's New in Python 3.3.5? =========================== -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:19:56 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 15 Mar 2014 21:19:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320939=3A_merge_from_3=2E3?= Message-ID: <3fmZrX4Vz7z7Lk0@mail.python.org> http://hg.python.org/cpython/rev/9358a41b245b changeset: 89675:9358a41b245b parent: 89672:fe0ad06da28c parent: 89674:030efedd70ca user: Ned Deily date: Sat Mar 15 13:19:20 2014 -0700 summary: Issue #20939: merge from 3.3 files: Lib/test/test_urllibnet.py | 2 +- Misc/NEWS | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -80,7 +80,7 @@ def test_geturl(self): # Make sure same URL as opened is returned by geturl. - URL = "http://www.python.org/" + URL = "https://www.python.org/" with self.urlopen(URL) as open_url: gotten_url = open_url.geturl() self.assertEqual(gotten_url, URL) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -72,6 +72,10 @@ - Issue #20743: Fix a reference leak in test_tcl. +- Issue #20939: Fix test_geturl failure in test_urllibnet due to + new redirect of http://www.python.org/ to https://www.python.org. + + Build ----- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:24:30 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 21:24:30 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_send_people_to?= =?utf-8?q?_the_right_editors_page_=28=2320938=29?= Message-ID: <3fmZxp4kSRz7LjX@mail.python.org> http://hg.python.org/cpython/rev/772404a89b88 changeset: 89676:772404a89b88 branch: 2.7 parent: 89673:9b02f23bc129 user: Benjamin Peterson date: Sat Mar 15 15:23:32 2014 -0500 summary: send people to the right editors page (#20938) files: Doc/faq/general.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -471,7 +471,8 @@ Emacs users will be happy to know that there is a very good Python mode for Emacs. All of these programming environments provide syntax highlighting, auto-indenting, and access to the interactive interpreter while coding. Consult -http://www.python.org/editors/ for a full list of Python editing environments. +`the Python wiki `_ for a full list +of Python editing environments. If you want to discuss Python's use in education, you may be interested in joining `the edu-sig mailing list -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:24:32 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 21:24:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_send_people_to?= =?utf-8?q?_the_right_editors_page_=28=2320938=29?= Message-ID: <3fmZxr086bz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/014c2a62c168 changeset: 89677:014c2a62c168 branch: 3.3 parent: 89674:030efedd70ca user: Benjamin Peterson date: Sat Mar 15 15:23:32 2014 -0500 summary: send people to the right editors page (#20938) files: Doc/faq/general.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -471,7 +471,8 @@ Emacs users will be happy to know that there is a very good Python mode for Emacs. All of these programming environments provide syntax highlighting, auto-indenting, and access to the interactive interpreter while coding. Consult -http://www.python.org/editors/ for a full list of Python editing environments. +`the Python wiki `_ for a full list +of Python editing environments. If you want to discuss Python's use in education, you may be interested in joining `the edu-sig mailing list -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Mar 15 21:24:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 15 Mar 2014 21:24:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4z?= Message-ID: <3fmZxs2fC8z7Ljd@mail.python.org> http://hg.python.org/cpython/rev/f4c15e978483 changeset: 89678:f4c15e978483 parent: 89675:9358a41b245b parent: 89677:014c2a62c168 user: Benjamin Peterson date: Sat Mar 15 15:24:10 2014 -0500 summary: merge 3.3 files: Doc/faq/general.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -471,7 +471,8 @@ Emacs users will be happy to know that there is a very good Python mode for Emacs. All of these programming environments provide syntax highlighting, auto-indenting, and access to the interactive interpreter while coding. Consult -http://www.python.org/editors/ for a full list of Python editing environments. +`the Python wiki `_ for a full list +of Python editing environments. If you want to discuss Python's use in education, you may be interested in joining `the edu-sig mailing list -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 02:15:36 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 16 Mar 2014 02:15:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_Clarify_headin?= =?utf-8?q?g=2E?= Message-ID: <3fmjPh6xg4z7LjP@mail.python.org> http://hg.python.org/cpython/rev/21ecc3d52806 changeset: 89679:21ecc3d52806 user: R David Murray date: Sat Mar 15 21:15:18 2014 -0400 summary: whatsnew: Clarify heading. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -84,7 +84,7 @@ * No new syntax features were added in Python 3.4. -New expected features for Python implementations: +New Features: * :ref:`pip should always be "available" ` (:pep:`453`). * :ref:`Newly created file descriptors are non-inheritable ` -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 02:16:56 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sun, 16 Mar 2014 02:16:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2UgIzE2NjY1?= =?utf-8?q?=3A_improve_documentation_for_hex=28=29=2E__Patch_by_Jessica_Mc?= =?utf-8?q?Kellar=2E?= Message-ID: <3fmjRD06Tvz7LjP@mail.python.org> http://hg.python.org/cpython/rev/464c22bf9fe1 changeset: 89680:464c22bf9fe1 branch: 3.3 parent: 89677:014c2a62c168 user: Antoine Pitrou date: Sun Mar 16 02:12:20 2014 +0100 summary: Close #16665: improve documentation for hex(). Patch by Jessica McKellar. files: Doc/library/functions.rst | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -608,9 +608,19 @@ .. function:: hex(x) - Convert an integer number to a hexadecimal string. The result is a valid Python - expression. If *x* is not a Python :class:`int` object, it has to define an - :meth:`__index__` method that returns an integer. + Convert an integer number to a lowercase hexadecimal string + prefixed with "0x", for example: + + >>> hex(255) + '0xff' + >>> hex(-42) + '-0x2a' + + If x is not a Python :class:`int` object, it has to define an __index__() + method that returns an integer. + + See also :func:`int` for converting a hexadecimal string to an + integer using a base of 16. .. note:: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 02:16:57 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sun, 16 Mar 2014 02:16:57 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2UgIzE2NjY1?= =?utf-8?q?=3A_improve_documentation_for_hex=28=29=2E__Patch_by_Jessica_Mc?= =?utf-8?q?Kellar=2E?= Message-ID: <3fmjRF23Bmz7LjP@mail.python.org> http://hg.python.org/cpython/rev/d14beaf03f55 changeset: 89681:d14beaf03f55 branch: 2.7 parent: 89676:772404a89b88 user: Antoine Pitrou date: Sun Mar 16 02:12:20 2014 +0100 summary: Close #16665: improve documentation for hex(). Patch by Jessica McKellar. files: Doc/library/functions.rst | 17 +++++++++++++++-- 1 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -596,8 +596,21 @@ .. function:: hex(x) - Convert an integer number (of any size) to a hexadecimal string. The result is a - valid Python expression. + Convert an integer number (of any size) to a lowercase hexadecimal string + prefixed with "0x", for example: + + >>> hex(255) + '0xff' + >>> hex(-42) + '-0x2a' + >>> hex(1L) + '0x1L' + + If x is not a Python :class:`int` or :class:`long` object, it has to + define an __index__() method that returns an integer. + + See also :func:`int` for converting a hexadecimal string to an + integer using a base of 16. .. note:: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 02:16:58 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sun, 16 Mar 2014 02:16:58 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?q?=29=3A_Close_=2316665=3A_improve_documentation_for_hex=28=29=2E?= =?utf-8?q?__Patch_by_Jessica_McKellar=2E?= Message-ID: <3fmjRG40Jrz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/c267f4eb8173 changeset: 89682:c267f4eb8173 parent: 89679:21ecc3d52806 parent: 89680:464c22bf9fe1 user: Antoine Pitrou date: Sun Mar 16 02:13:07 2014 +0100 summary: Close #16665: improve documentation for hex(). Patch by Jessica McKellar. files: Doc/library/functions.rst | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -617,9 +617,19 @@ .. function:: hex(x) - Convert an integer number to a hexadecimal string. The result is a valid Python - expression. If *x* is not a Python :class:`int` object, it has to define an - :meth:`__index__` method that returns an integer. + Convert an integer number to a lowercase hexadecimal string + prefixed with "0x", for example: + + >>> hex(255) + '0xff' + >>> hex(-42) + '-0x2a' + + If x is not a Python :class:`int` object, it has to define an __index__() + method that returns an integer. + + See also :func:`int` for converting a hexadecimal string to an + integer using a base of 16. .. note:: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 02:41:51 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sun, 16 Mar 2014 02:41:51 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Issue_=2320926=3A_Enhance?= =?utf-8?q?_Quick_Start_portion_of_instructions_for_Windows=2E__Patch?= Message-ID: <3fmjzz1tVRz7LjN@mail.python.org> http://hg.python.org/devguide/rev/a775fe9e7f86 changeset: 676:a775fe9e7f86 user: Antoine Pitrou date: Sun Mar 16 02:41:37 2014 +0100 summary: Issue #20926: Enhance Quick Start portion of instructions for Windows. Patch by Kathleen Weaver. files: index.rst | 3 ++- setup.rst | 3 +++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/index.rst b/index.rst --- a/index.rst +++ b/index.rst @@ -27,7 +27,8 @@ and :menuselection:`Build --> Build Solution`. Run :file:`Tools\\buildbot\\external.bat` or :file:`Tools\\buildbot\\external-amd64.bat` to download and compile 3rd - party libraries. + party libraries. Note: Visual Studio may throw errors, but Python will + be built. 3. :doc:`Run the tests `:: diff --git a/setup.rst b/setup.rst --- a/setup.rst +++ b/setup.rst @@ -217,6 +217,9 @@ Windows ''''''' +The readme included in the solution has more details, especially on the +software needed to resolve the below mentioned build errors. + **Python 3.3** and later use Microsoft Visual Studio 2010. You can download Microsoft Visual C++ 2010 Express `from Microsoft's site `_. -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Sun Mar 16 04:16:01 2014 From: python-checkins at python.org (larry.hastings) Date: Sun, 16 Mar 2014 04:16:01 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320931=3A_Fix_othe?= =?utf-8?q?r_confusingly-worded_heading=2E?= Message-ID: <3fmm4d2jq8z7LjM@mail.python.org> http://hg.python.org/cpython/rev/3488a9fbe09e changeset: 89683:3488a9fbe09e user: Larry Hastings date: Sat Mar 15 20:15:55 2014 -0700 summary: Issue #20931: Fix other confusingly-worded heading. files: Doc/whatsnew/3.4.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -174,8 +174,8 @@ -New Expected Features for Python Implementations -================================================ +New Features +============ .. _whatsnew-pep-453: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 04:31:26 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 16 Mar 2014 04:31:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew=3A_Fix_Features_s?= =?utf-8?q?ection_title=2C_use_title/sentence_case_consistently=2E?= Message-ID: <3fmmQQ2Y4yz7LjT@mail.python.org> http://hg.python.org/cpython/rev/7e78e602ec73 changeset: 89684:7e78e602ec73 user: R David Murray date: Sat Mar 15 23:27:30 2014 -0400 summary: whatsnew: Fix Features section title, use title/sentence case consistently. closes #20931 files: Doc/whatsnew/3.4.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -84,9 +84,9 @@ * No new syntax features were added in Python 3.4. -New Features: - -* :ref:`pip should always be "available" ` (:pep:`453`). +Other new features: + +* :ref:`pip should always be available ` (:pep:`453`). * :ref:`Newly created file descriptors are non-inheritable ` (:pep:`446`). * command line option for :ref:`isolated mode ` @@ -116,7 +116,7 @@ * :mod:`tracemalloc`: :ref:`Trace Python memory allocations ` (:pep:`454`). -Significantly Improved Library Modules: +Significantly improved library modules: * :ref:`Single-dispatch generic functions ` in :mod:`functools` (:pep:`443`). @@ -182,7 +182,7 @@ PEP 453: Explicit Bootstrapping of PIP in Python Installations -------------------------------------------------------------- -Bootstrapping pip by default +Bootstrapping pip By Default ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard @@ -214,7 +214,7 @@ __ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors -Documentation changes +Documentation Changes ~~~~~~~~~~~~~~~~~~~~~ As part of this change, the :ref:`installing-index` and @@ -2223,7 +2223,7 @@ that may require changes to your code. -Changes in 'python' command behavior +Changes in 'python' Command Behavior ------------------------------------ * In a posix shell, setting the :envvar:`PATH` environment variable to -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 04:59:09 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 16 Mar 2014 04:59:09 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Bring_PEP_453_What=27s_New?= =?utf-8?q?_entry_up_to_date?= Message-ID: <3fmn2P5GdBz7LjT@mail.python.org> http://hg.python.org/cpython/rev/74144b0e7858 changeset: 89685:74144b0e7858 user: Nick Coghlan date: Sun Mar 16 13:55:19 2014 +1000 summary: Bring PEP 453 What's New entry up to date We changed a few details of the pip integration over the course of the Python 3.4 beta and release candidate cycle, but I hadn't updated the What's New entry appropriately. The Py2/Py3 split on POSIX and the generally challenges of dealing with parallel installs means we haven't attained complete consistency across all environments, but hopefully this updated text is clearer about the end result. Relevant to issue #20909. files: Doc/whatsnew/3.4.rst | 55 +++++++++++++++++++------------ 1 files changed, 34 insertions(+), 21 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -187,16 +187,23 @@ The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard cross-platform mechanism to bootstrap the pip installer into Python -installations and virtual environments. - -By default, the scripts ``pipX`` and ``pipX.Y`` will be installed (where -X.Y stands for the version of the Python installation), along with the -``pip`` Python package and its dependencies. +installations and virtual environments. The version of ``pip`` included +with Python 3.4.0 is ``pip`` 1.5.4, and future 3.4.x maintenance releases +will update the bundled version to the latest version of ``pip`` that is +available at the time of creating the release candidate. + +By default, the commands ``pipX`` and ``pipX.Y`` will be installed on all +platforms (where X.Y stands for the version of the Python installation), +along with the ``pip`` Python package and its dependencies. On Windows and +in virtual environments on all platforms, the unversioned ``pip`` command +will also be installed. On other platforms, the system wide unversioned +``pip`` command typically refers to the separately installed Python 2 +version. The :ref:`pyvenv ` command line utility and the :mod:`venv` module make use of the :mod:`ensurepip` module to make ``pip`` readily -available in virtual environments. When using the command line utility, ``pip`` -is installed by default, while when using the :mod:`venv` module +available in virtual environments. When using the command line utility, +``pip`` is installed by default, while when using the :mod:`venv` module :ref:`venv-api` installation of ``pip`` must be requested explicitly. For CPython :ref:`source builds on POSIX systems `, @@ -204,15 +211,31 @@ default. This behaviour can be controlled through configure options, and overridden through Makefile options. -On Windows and Mac OS X, the CPython installers now offer the option to -install ``pip`` along with CPython itself. +On Windows and Mac OS X, the CPython installers now default to installing +``pip`` along with CPython itself (users may opt out of installing it +during the installation process). Window users will need to opt in to the +automatic ``PATH`` modifications to have ``pip`` available from the command +line by default, otherwise it can still be accessed through the Python +launcher for Windows as ``py -m pip``. As `discussed in the PEP`__, platform packagers may choose not to install -``pip`` by default, as long as the command ``pip``, when invoked, provides -clear and simple directions on how to install ``pip`` on the platform. +these commands by default, as long as, when invoked, they provide clear and +simple directions on how to install them on that platform (usually using +the system package manager). __ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors +.. note:: + + To avoid conflicts between parallel Python 2 and Python 3 installations, + only the versioned ``pip3`` and ``pip3.4`` commands are bootstrapped by + default when ``ensurepip`` is invoked directly - the ``--default-pip`` + option is needed to also request the unversioned ``pip`` command. + ``pyvenv`` and the Windows installer ensure that the unqualified ``pip`` + command is made available in those environments, and ``pip`` can always be + invoked via the ``-m`` switch rather than directly to avoid ambiguity on + systems with multiple Python installations. + Documentation Changes ~~~~~~~~~~~~~~~~~~~~~ @@ -229,16 +252,6 @@ versions of those guides remaining available as :ref:`install-index` and :ref:`distutils-index`. -.. note:: - - To avoid conflicts between parallel Python 2 and Python 3 installations, - only the versioned ``pip3`` and ``pip3.4`` commands are bootstrapped by - default when ``ensurepip`` is invoked directly (including by the CPython - installers). ``pyvenv`` ensures that the unqualified ``pip`` command is - made available in virtual environments, and ``pip`` can always be - invoked via the ``-m`` switch rather than directly to avoid ambiguity on - systems with multiple Python installations. - .. seealso:: :pep:`453` -- Explicit bootstrapping of pip in Python installations -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 05:43:21 2014 From: python-checkins at python.org (nick.coghlan) Date: Sun, 16 Mar 2014 05:43:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_statistics_module_=26_new_has?= =?utf-8?q?h_are_in_3=2E4?= Message-ID: <3fmp1P0K4Cz7LjW@mail.python.org> http://hg.python.org/peps/rev/37a8fdcf4940 changeset: 5410:37a8fdcf4940 user: Nick Coghlan date: Sun Mar 16 14:43:06 2014 +1000 summary: statistics module & new hash are in 3.4 files: pep-0450.txt | 2 +- pep-0456.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0450.txt b/pep-0450.txt --- a/pep-0450.txt +++ b/pep-0450.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Steven D'Aprano -Status: Accepted +Status: Final Type: Standards Track Content-Type: text/plain Created: 01-Aug-2013 diff --git a/pep-0456.txt b/pep-0456.txt --- a/pep-0456.txt +++ b/pep-0456.txt @@ -4,7 +4,7 @@ Last-Modified: $Date$ Author: Christian Heimes BDFL-Delegate: Nick Coghlan -Status: Accepted +Status: Final Type: Standards Track Content-Type: text/x-rst Created: 27-Sep-2013 -- Repository URL: http://hg.python.org/peps From solipsis at pitrou.net Sun Mar 16 09:54:52 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 16 Mar 2014 09:54:52 +0100 Subject: [Python-checkins] Daily reference leaks (c267f4eb8173): sum=0 Message-ID: results for c267f4eb8173 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 4] memory blocks, sum=4 test_site leaked [0, -2, 0] references, sum=-2 test_site leaked [0, -2, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogIA27Yk', '-x'] From vinay_sajip at yahoo.co.uk Sat Mar 15 00:06:50 2014 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 14 Mar 2014 23:06:50 +0000 (GMT) Subject: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. In-Reply-To: <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> References: <3flpwB0Njyz7LjX@mail.python.org> <1394809824.29257.94514229.6A393CDD@webmail.messagingengine.com> Message-ID: <1394838410.68259.YahooMailNeo@web172402.mail.ir2.yahoo.com> Sorry, my mistake - I should have checked more carefully. Thank you for pointing me to the commit where they were removed - I'll install a venv with Sphinx, revert the change and remove those files from my repo. Regards, Vinay Sajip ________________________________ From: Benjamin Peterson To: python-dev at python.org; python-checkins at python.org; vinay_sajip at yahoo.co.uk Sent: Friday, 14 March 2014, 15:10 Subject: Re: [Python-checkins] cpython: Added Doc/tools/ subdirs to .hgignore. On Fri, Mar 14, 2014, at 07:20 AM, vinay.sajip wrote: > http://hg.python.org/cpython/rev/edb12dad7bf6 > changeset:? 89654:edb12dad7bf6 > user:? ? ? ? Vinay Sajip > date:? ? ? ? Fri Mar 14 14:20:09 2014 +0000 > summary: >? Added Doc/tools/ subdirs to .hgignore. > > files: >? .hgignore |? 5 +++++ >? 1 files changed, 5 insertions(+), 0 deletions(-) > > > diff --git a/.hgignore b/.hgignore > --- a/.hgignore > +++ b/.hgignore > @@ -24,6 +24,11 @@ >? reflog.txt$ >? tags$ >? Lib/plat-mac/errors.rsrc.df.rsrc > +Doc/tools/sphinx/ > +Doc/tools/docutils/ > +Doc/tools/jinja/ > +Doc/tools/jinja2/ > +Doc/tools/pygments/ Why are you readding these when they were apparently purposely removed in eef7899ea7ab? -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Sun Mar 16 17:31:12 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 16 Mar 2014 17:31:12 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fix_ctypes_tes?= =?utf-8?q?t_alignment_assumptions_=28closes_=2320946=29?= Message-ID: <3fn5k82jkHz7LjM@mail.python.org> http://hg.python.org/cpython/rev/2150053d77ca changeset: 89686:2150053d77ca branch: 3.3 parent: 89680:464c22bf9fe1 user: Benjamin Peterson date: Sun Mar 16 10:07:26 2014 +0100 summary: fix ctypes test alignment assumptions (closes #20946) Patch by Andreas Schwab. files: Lib/ctypes/test/test_bitfields.py | 2 +- Lib/ctypes/test/test_structures.py | 18 +++++++++--------- Misc/NEWS | 2 ++ Modules/_ctypes/cfield.c | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/Lib/ctypes/test/test_bitfields.py b/Lib/ctypes/test/test_bitfields.py --- a/Lib/ctypes/test/test_bitfields.py +++ b/Lib/ctypes/test/test_bitfields.py @@ -207,7 +207,7 @@ class X(Structure): _fields_ = [("a", c_byte, 4), ("b", c_int, 32)] - self.assertEqual(sizeof(X), sizeof(c_int)*2) + self.assertEqual(sizeof(X), alignment(c_int)+sizeof(c_int)) def test_mixed_3(self): class X(Structure): diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py --- a/Lib/ctypes/test/test_structures.py +++ b/Lib/ctypes/test/test_structures.py @@ -83,7 +83,7 @@ class Y(Structure): _fields_ = [("x", c_char * 3), ("y", c_int)] - self.assertEqual(alignment(Y), calcsize("i")) + self.assertEqual(alignment(Y), alignment(c_int)) self.assertEqual(sizeof(Y), calcsize("3si")) class SI(Structure): @@ -175,13 +175,6 @@ self.assertEqual(sizeof(X), 10) self.assertEqual(X.b.offset, 2) - class X(Structure): - _fields_ = [("a", c_byte), - ("b", c_longlong)] - _pack_ = 4 - self.assertEqual(sizeof(X), 12) - self.assertEqual(X.b.offset, 4) - import struct longlong_size = struct.calcsize("q") longlong_align = struct.calcsize("bq") - longlong_size @@ -189,9 +182,16 @@ class X(Structure): _fields_ = [("a", c_byte), ("b", c_longlong)] + _pack_ = 4 + self.assertEqual(sizeof(X), min(4, longlong_align) + longlong_size) + self.assertEqual(X.b.offset, min(4, longlong_align)) + + class X(Structure): + _fields_ = [("a", c_byte), + ("b", c_longlong)] _pack_ = 8 - self.assertEqual(sizeof(X), longlong_align + longlong_size) + self.assertEqual(sizeof(X), min(8, longlong_align) + longlong_size) self.assertEqual(X.b.offset, min(8, longlong_align)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,8 @@ Tests ----- +- Issue #20946: Correct alignment assumptions of some ctypes tests. + - Issue #20939: Fix test_geturl failure in test_urllibnet due to new redirect of http://www.python.org/ to https://www.python.org. diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -1640,9 +1640,9 @@ /* #define CHAR_ALIGN (sizeof(s_char) - sizeof(char)) #define SHORT_ALIGN (sizeof(s_short) - sizeof(short)) -#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define LONG_ALIGN (sizeof(s_long) - sizeof(long)) */ +#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define FLOAT_ALIGN (sizeof(s_float) - sizeof(float)) #define DOUBLE_ALIGN (sizeof(s_double) - sizeof(double)) #define LONGDOUBLE_ALIGN (sizeof(s_long_double) - sizeof(long double)) @@ -1684,8 +1684,8 @@ ffi_type ffi_type_uint16 = { 2, 2, FFI_TYPE_UINT16 }; ffi_type ffi_type_sint16 = { 2, 2, FFI_TYPE_SINT16 }; -ffi_type ffi_type_uint32 = { 4, 4, FFI_TYPE_UINT32 }; -ffi_type ffi_type_sint32 = { 4, 4, FFI_TYPE_SINT32 }; +ffi_type ffi_type_uint32 = { 4, INT_ALIGN, FFI_TYPE_UINT32 }; +ffi_type ffi_type_sint32 = { 4, INT_ALIGN, FFI_TYPE_SINT32 }; ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 }; ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 }; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 17:31:13 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 16 Mar 2014 17:31:13 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_ctypes_tes?= =?utf-8?q?t_alignment_assumptions_=28closes_=2320946=29?= Message-ID: <3fn5k96ZfMz7Ljx@mail.python.org> http://hg.python.org/cpython/rev/e5a09b09bb51 changeset: 89687:e5a09b09bb51 branch: 2.7 parent: 89681:d14beaf03f55 user: Benjamin Peterson date: Sun Mar 16 10:07:26 2014 +0100 summary: fix ctypes test alignment assumptions (closes #20946) Patch by Andreas Schwab. files: Lib/ctypes/test/test_bitfields.py | 2 +- Lib/ctypes/test/test_structures.py | 18 +++++++++--------- Misc/NEWS | 2 ++ Modules/_ctypes/cfield.c | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/Lib/ctypes/test/test_bitfields.py b/Lib/ctypes/test/test_bitfields.py --- a/Lib/ctypes/test/test_bitfields.py +++ b/Lib/ctypes/test/test_bitfields.py @@ -207,7 +207,7 @@ class X(Structure): _fields_ = [("a", c_byte, 4), ("b", c_int, 32)] - self.assertEqual(sizeof(X), sizeof(c_int)*2) + self.assertEqual(sizeof(X), alignment(c_int)+sizeof(c_int)) def test_mixed_3(self): class X(Structure): diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py --- a/Lib/ctypes/test/test_structures.py +++ b/Lib/ctypes/test/test_structures.py @@ -83,7 +83,7 @@ class Y(Structure): _fields_ = [("x", c_char * 3), ("y", c_int)] - self.assertEqual(alignment(Y), calcsize("i")) + self.assertEqual(alignment(Y), alignment(c_int)) self.assertEqual(sizeof(Y), calcsize("3si")) class SI(Structure): @@ -175,13 +175,6 @@ self.assertEqual(sizeof(X), 10) self.assertEqual(X.b.offset, 2) - class X(Structure): - _fields_ = [("a", c_byte), - ("b", c_longlong)] - _pack_ = 4 - self.assertEqual(sizeof(X), 12) - self.assertEqual(X.b.offset, 4) - import struct longlong_size = struct.calcsize("q") longlong_align = struct.calcsize("bq") - longlong_size @@ -189,9 +182,16 @@ class X(Structure): _fields_ = [("a", c_byte), ("b", c_longlong)] + _pack_ = 4 + self.assertEqual(sizeof(X), min(4, longlong_align) + longlong_size) + self.assertEqual(X.b.offset, min(4, longlong_align)) + + class X(Structure): + _fields_ = [("a", c_byte), + ("b", c_longlong)] _pack_ = 8 - self.assertEqual(sizeof(X), longlong_align + longlong_size) + self.assertEqual(sizeof(X), min(8, longlong_align) + longlong_size) self.assertEqual(X.b.offset, min(8, longlong_align)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -271,6 +271,8 @@ Tests ----- +- Issue #20946: Correct alignment assumptions of some ctypes tests. + - Issue #20743: Fix a reference leak in test_tcl. - Issue #20510: Rewrote test_exit in test_sys to match existing comments, diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -1687,9 +1687,9 @@ /* #define CHAR_ALIGN (sizeof(s_char) - sizeof(char)) #define SHORT_ALIGN (sizeof(s_short) - sizeof(short)) -#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define LONG_ALIGN (sizeof(s_long) - sizeof(long)) */ +#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define FLOAT_ALIGN (sizeof(s_float) - sizeof(float)) #define DOUBLE_ALIGN (sizeof(s_double) - sizeof(double)) #define LONGDOUBLE_ALIGN (sizeof(s_long_double) - sizeof(long double)) @@ -1731,8 +1731,8 @@ ffi_type ffi_type_uint16 = { 2, 2, FFI_TYPE_UINT16 }; ffi_type ffi_type_sint16 = { 2, 2, FFI_TYPE_SINT16 }; -ffi_type ffi_type_uint32 = { 4, 4, FFI_TYPE_UINT32 }; -ffi_type ffi_type_sint32 = { 4, 4, FFI_TYPE_SINT32 }; +ffi_type ffi_type_uint32 = { 4, INT_ALIGN, FFI_TYPE_UINT32 }; +ffi_type ffi_type_sint32 = { 4, INT_ALIGN, FFI_TYPE_SINT32 }; ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 }; ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 }; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 17:31:15 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 16 Mar 2014 17:31:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMjA5NDYp?= Message-ID: <3fn5kC2VCXz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/3736bf94535c changeset: 89688:3736bf94535c parent: 89685:74144b0e7858 parent: 89686:2150053d77ca user: Benjamin Peterson date: Sun Mar 16 11:30:54 2014 -0500 summary: merge 3.3 (#20946) files: Lib/ctypes/test/test_bitfields.py | 2 +- Lib/ctypes/test/test_structures.py | 18 +++++++++--------- Misc/NEWS | 2 ++ Modules/_ctypes/cfield.c | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/Lib/ctypes/test/test_bitfields.py b/Lib/ctypes/test/test_bitfields.py --- a/Lib/ctypes/test/test_bitfields.py +++ b/Lib/ctypes/test/test_bitfields.py @@ -207,7 +207,7 @@ class X(Structure): _fields_ = [("a", c_byte, 4), ("b", c_int, 32)] - self.assertEqual(sizeof(X), sizeof(c_int)*2) + self.assertEqual(sizeof(X), alignment(c_int)+sizeof(c_int)) def test_mixed_3(self): class X(Structure): diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py --- a/Lib/ctypes/test/test_structures.py +++ b/Lib/ctypes/test/test_structures.py @@ -83,7 +83,7 @@ class Y(Structure): _fields_ = [("x", c_char * 3), ("y", c_int)] - self.assertEqual(alignment(Y), calcsize("i")) + self.assertEqual(alignment(Y), alignment(c_int)) self.assertEqual(sizeof(Y), calcsize("3si")) class SI(Structure): @@ -175,13 +175,6 @@ self.assertEqual(sizeof(X), 10) self.assertEqual(X.b.offset, 2) - class X(Structure): - _fields_ = [("a", c_byte), - ("b", c_longlong)] - _pack_ = 4 - self.assertEqual(sizeof(X), 12) - self.assertEqual(X.b.offset, 4) - import struct longlong_size = struct.calcsize("q") longlong_align = struct.calcsize("bq") - longlong_size @@ -189,9 +182,16 @@ class X(Structure): _fields_ = [("a", c_byte), ("b", c_longlong)] + _pack_ = 4 + self.assertEqual(sizeof(X), min(4, longlong_align) + longlong_size) + self.assertEqual(X.b.offset, min(4, longlong_align)) + + class X(Structure): + _fields_ = [("a", c_byte), + ("b", c_longlong)] _pack_ = 8 - self.assertEqual(sizeof(X), longlong_align + longlong_size) + self.assertEqual(sizeof(X), min(8, longlong_align) + longlong_size) self.assertEqual(X.b.offset, min(8, longlong_align)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -72,6 +72,8 @@ - Issue #20743: Fix a reference leak in test_tcl. +- Issue #20946: Correct alignment assumptions of some ctypes tests. + - Issue #20939: Fix test_geturl failure in test_urllibnet due to new redirect of http://www.python.org/ to https://www.python.org. diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -1640,9 +1640,9 @@ /* #define CHAR_ALIGN (sizeof(s_char) - sizeof(char)) #define SHORT_ALIGN (sizeof(s_short) - sizeof(short)) -#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define LONG_ALIGN (sizeof(s_long) - sizeof(long)) */ +#define INT_ALIGN (sizeof(s_int) - sizeof(int)) #define FLOAT_ALIGN (sizeof(s_float) - sizeof(float)) #define DOUBLE_ALIGN (sizeof(s_double) - sizeof(double)) #define LONGDOUBLE_ALIGN (sizeof(s_long_double) - sizeof(long double)) @@ -1684,8 +1684,8 @@ ffi_type ffi_type_uint16 = { 2, 2, FFI_TYPE_UINT16 }; ffi_type ffi_type_sint16 = { 2, 2, FFI_TYPE_SINT16 }; -ffi_type ffi_type_uint32 = { 4, 4, FFI_TYPE_UINT32 }; -ffi_type ffi_type_sint32 = { 4, 4, FFI_TYPE_SINT32 }; +ffi_type ffi_type_uint32 = { 4, INT_ALIGN, FFI_TYPE_UINT32 }; +ffi_type ffi_type_sint32 = { 4, INT_ALIGN, FFI_TYPE_SINT32 }; ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 }; ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 }; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 21:29:53 2014 From: python-checkins at python.org (victor.stinner) Date: Sun, 16 Mar 2014 21:29:53 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320950=3A_Fix_typo?= =?utf-8?q?_asyncio_doc=2C_wait=28=29_has_no_self_parameter?= Message-ID: <3fnC1Y5X1Gz7LjS@mail.python.org> http://hg.python.org/cpython/rev/1009cf8cb304 changeset: 89689:1009cf8cb304 user: Victor Stinner date: Sun Mar 16 21:29:31 2014 +0100 summary: Issue #20950: Fix typo asyncio doc, wait() has no self parameter self parameter is implicit. Mention also that communicate() and wait() are coroutines. files: Doc/library/asyncio-subprocess.rst | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -107,6 +107,8 @@ The data read is buffered in memory, so do not use this method if the data size is large or unlimited. + This method is a :ref:`coroutine `. + .. method:: kill() Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to @@ -129,11 +131,13 @@ to the child. On Windows the Win32 API function :c:func:`TerminateProcess` is called to stop the child. - .. method:: wait(self): + .. method:: wait(): Wait for child process to terminate. Set and return :attr:`returncode` attribute. + This method is a :ref:`coroutine `. + Example ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 21:56:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 16 Mar 2014 21:56:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogaW1wcm92ZSBzZXRf?= =?utf-8?q?tunnel_docs_=28closes_=2311448=29?= Message-ID: <3fnCcN22PYz7LjW@mail.python.org> http://hg.python.org/cpython/rev/68a257ca6be6 changeset: 89690:68a257ca6be6 branch: 3.3 parent: 89686:2150053d77ca user: Benjamin Peterson date: Sun Mar 16 15:55:39 2014 -0500 summary: improve set_tunnel docs (closes #11448) Patch by Ryan Kelly, karl, and Nikolaus Rath. files: Doc/library/http.client.rst | 22 ++++++++++++++++++---- 1 files changed, 18 insertions(+), 4 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -451,11 +451,25 @@ .. method:: HTTPConnection.set_tunnel(host, port=None, headers=None) - Set the host and the port for HTTP Connect Tunnelling. Normally used when it - is required to a HTTPS Connection through a proxy server. + Set the host and the port for HTTP Connect Tunnelling. This allows running + the connection through a proxy server. - The headers argument should be a mapping of extra HTTP headers to send - with the CONNECT request. + The host and port arguments specify the endpoint of the tunneled connection + (i.e. the address included in the CONNECT request, *not* the address of the + proxy server). + + The headers argument should be a mapping of extra HTTP headers to send with + the CONNECT request. + + For example, to tunnel through a HTTPS proxy server running locally on port + 8080, we would pass the address of the proxy to the :class:`HTTPSConnection` + constructor, and the address of the host that we eventually want to reach to + the :meth:`~HTTPConnection.set_tunnel` method:: + + >>> import http.client + >>> conn = http.client.HTTPSConnection("localhost", 8080) + >>> conn.set_tunnel("www.python.org") + >>> conn.request("HEAD","/index.html") .. versionadded:: 3.2 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Mar 16 21:56:37 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 16 Mar 2014 21:56:37 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy4zICgjMTE0NDgp?= Message-ID: <3fnCcP3jFRz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/5cab0ada97b2 changeset: 89691:5cab0ada97b2 parent: 89689:1009cf8cb304 parent: 89690:68a257ca6be6 user: Benjamin Peterson date: Sun Mar 16 15:56:24 2014 -0500 summary: merge 3.3 (#11448) files: Doc/library/http.client.rst | 22 ++++++++++++++++++---- 1 files changed, 18 insertions(+), 4 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -451,11 +451,25 @@ .. method:: HTTPConnection.set_tunnel(host, port=None, headers=None) - Set the host and the port for HTTP Connect Tunnelling. Normally used when it - is required to a HTTPS Connection through a proxy server. + Set the host and the port for HTTP Connect Tunnelling. This allows running + the connection through a proxy server. - The headers argument should be a mapping of extra HTTP headers to send - with the CONNECT request. + The host and port arguments specify the endpoint of the tunneled connection + (i.e. the address included in the CONNECT request, *not* the address of the + proxy server). + + The headers argument should be a mapping of extra HTTP headers to send with + the CONNECT request. + + For example, to tunnel through a HTTPS proxy server running locally on port + 8080, we would pass the address of the proxy to the :class:`HTTPSConnection` + constructor, and the address of the host that we eventually want to reach to + the :meth:`~HTTPConnection.set_tunnel` method:: + + >>> import http.client + >>> conn = http.client.HTTPSConnection("localhost", 8080) + >>> conn.set_tunnel("www.python.org") + >>> conn.request("HEAD","/index.html") .. versionadded:: 3.2 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:23:56 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 17 Mar 2014 07:23:56 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_allow-branch_hook_update_?= =?utf-8?q?instructions=2E?= Message-ID: <3fnSC02Schz7LjW@mail.python.org> http://hg.python.org/peps/rev/9b9d06e19586 changeset: 5411:9b9d06e19586 user: Georg Brandl date: Mon Mar 17 07:23:38 2014 +0100 summary: Add allow-branch hook update instructions. files: pep-0101.txt | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -38,6 +38,9 @@ * Access to ``dinsdale.python.org``, the Python web host. You'll be uploading files directly here. + * Shell access to ``hg.python.org``, the Python Mercurial host. You'll + have to adapt repository configuration there. + * Write access to the website SVN repository Instructions here: @@ -266,6 +269,12 @@ $ hg update deadbeef # revision ID noted down before $ hg branch 3.2 + ___ When you want to push back your new branch to the main CPython + repository, the new branch name must be added to the "allow-branches" + hook configuration, which protects against stray named branches being + pushed. Login to hg.python.org and edit (as the "hg" user) + ``/data/hg/repos/cpython/.hg/hgrc`` to that effect. + ___ Push your commits to the remote release clone. $ hg push ssh://hg.python.org/releasing/... -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Mar 17 07:31:09 2014 From: python-checkins at python.org (georg.brandl) Date: Mon, 17 Mar 2014 07:31:09 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_version_switcher_update_i?= =?utf-8?q?nstructions=2E?= Message-ID: <3fnSMK08w9z7Lk3@mail.python.org> http://hg.python.org/peps/rev/a62ae7ce5267 changeset: 5412:a62ae7ce5267 user: Georg Brandl date: Mon Mar 17 07:30:58 2014 +0100 summary: Add version switcher update instructions. files: pep-0101.txt | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -275,6 +275,10 @@ pushed. Login to hg.python.org and edit (as the "hg" user) ``/data/hg/repos/cpython/.hg/hgrc`` to that effect. + ___ For a final major release, Doc/tools/sphinxext/static/version_switch.js + must be updated in all maintained branches, so that the new maintenance + branch is not "dev" anymore and there is a new "dev" version. + ___ Push your commits to the remote release clone. $ hg push ssh://hg.python.org/releasing/... -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Mar 17 07:31:15 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:15 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Created_releas?= =?utf-8?q?e_branch_for_3=2E4=2E?= Message-ID: <3fnSMS01ZNz7LjY@mail.python.org> http://hg.python.org/cpython/rev/c9aced9cca53 changeset: 89692:c9aced9cca53 branch: 3.4 parent: 89136:e64ae8b82672 user: Larry Hastings date: Thu Feb 20 23:38:01 2014 -0800 summary: Created release branch for 3.4. files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:17 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:17 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_Use_even_shorter_sleep_in_test=5Ftimeout=5Frounding=28=29_t?= =?utf-8?q?o_make_the?= Message-ID: <3fnSMT1gRfz7LjY@mail.python.org> http://hg.python.org/cpython/rev/4f1df287392c changeset: 89693:4f1df287392c branch: 3.4 user: Victor Stinner date: Mon Feb 10 19:17:46 2014 +0100 summary: Issue #20505: Use even shorter sleep in test_timeout_rounding() to make the test more reliable (= fail more often on Windows with HPET enabled). files: Lib/test/test_asyncio/test_events.py | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1176,13 +1176,15 @@ loop = self.loop yield from asyncio.sleep(1e-2, loop=loop) yield from asyncio.sleep(1e-4, loop=loop) + yield from asyncio.sleep(1e-6, loop=loop) + yield from asyncio.sleep(1e-8, loop=loop) self.loop.run_until_complete(wait()) - # The ideal number of call is 6, but on some platforms, the selector + # The ideal number of call is 10, but on some platforms, the selector # may sleep at little bit less than timeout depending on the resolution - # of the clock used by the kernel. Tolerate 2 useless calls on these + # of the clock used by the kernel. Tolerate 5 useless calls on these # platforms. - self.assertLessEqual(self.loop._run_once_counter, 8, + self.assertLessEqual(self.loop._run_once_counter, 15, {'time_info': time.get_clock_info('time'), 'monotonic_info': time.get_clock_info('monotonic'), 'selector': self.loop._selector.__class__.__name__}) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:18 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:18 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_merge_with_3?= =?utf-8?q?=2E3?= Message-ID: <3fnSMV4PMxz7LjV@mail.python.org> http://hg.python.org/cpython/rev/bce5fc7c081f changeset: 89694:bce5fc7c081f branch: 3.4 user: Georg Brandl date: Mon Feb 10 22:11:21 2014 +0100 summary: merge with 3.3 files: .hgtags | 2 + Doc/tools/sphinxext/susp-ignored.csv | 1 + Lib/idlelib/NEWS.txt | 1 - Lib/smtplib.py | 5 ++- Lib/test/mock_socket.py | 9 +++- Lib/test/test_smtplib.py | 30 +++++++++++++++- 6 files changed, 43 insertions(+), 5 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -118,6 +118,8 @@ fd53c500f8b80f54f3ecedec9da2e8c7e52a6888 v3.3.3rc1 d32442c0e60dfbd71234e807d3d1dedd227495a9 v3.3.3rc2 c3896275c0f61b2510a6c7e6c458a750359a91b8 v3.3.3 +fa92f5f940c6c0d839d7f0611e4b717606504a3c v3.3.4rc1 +7ff62415e4263c432c8acf6e424224209211eadb v3.3.4 46535f65e7f3bcdcf176f36d34bc1fed719ffd2b v3.4.0a1 9265a2168e2cb2a84785d8717792acc661e6b692 v3.4.0a2 dd9cdf90a5073510877e9dd5112f8e6cf20d5e89 v3.4.0a3 diff --git a/Doc/tools/sphinxext/susp-ignored.csv b/Doc/tools/sphinxext/susp-ignored.csv --- a/Doc/tools/sphinxext/susp-ignored.csv +++ b/Doc/tools/sphinxext/susp-ignored.csv @@ -282,3 +282,4 @@ whatsnew/changelog,,::,": Fix FTP tests for IPv6, bind to ""::1"" instead of ""localhost""." whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" whatsnew/changelog,,:password,user:password +whatsnew/changelog,,:gz,w:gz diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -875,4 +875,3 @@ -------------------------------------------------------------------- Refer to HISTORY.txt for additional information on earlier releases. -------------------------------------------------------------------- - diff --git a/Lib/smtplib.py b/Lib/smtplib.py --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -62,6 +62,7 @@ SMTP_SSL_PORT = 465 CRLF = "\r\n" bCRLF = b"\r\n" +_MAXLINE = 8192 # more than 8 times larger than RFC 821, 4.5.3 OLDSTYLE_AUTH = re.compile(r"auth=(.*)", re.I) @@ -365,7 +366,7 @@ self.file = self.sock.makefile('rb') while 1: try: - line = self.file.readline() + line = self.file.readline(_MAXLINE + 1) except OSError as e: self.close() raise SMTPServerDisconnected("Connection unexpectedly closed: " @@ -375,6 +376,8 @@ raise SMTPServerDisconnected("Connection unexpectedly closed") if self.debuglevel > 0: print('reply:', repr(line), file=stderr) + if len(line) > _MAXLINE: + raise SMTPResponseException(500, "Line too long.") resp.append(line[4:].strip(b' \t\r\n')) code = line[:3] # Check that the error code is syntactically correct. diff --git a/Lib/test/mock_socket.py b/Lib/test/mock_socket.py --- a/Lib/test/mock_socket.py +++ b/Lib/test/mock_socket.py @@ -21,8 +21,13 @@ """ def __init__(self, lines): self.lines = lines - def readline(self): - return self.lines.pop(0) + b'\r\n' + def readline(self, limit=-1): + result = self.lines.pop(0) + b'\r\n' + if limit >= 0: + # Re-insert the line, removing the \r\n we added. + self.lines.insert(0, result[limit:-2]) + result = result[:limit] + return result def close(self): pass diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -563,6 +563,33 @@ HOST, self.port, 'localhost', 3) + at unittest.skipUnless(threading, 'Threading required for this test.') +class TooLongLineTests(unittest.TestCase): + respdata = b'250 OK' + (b'.' * smtplib._MAXLINE * 2) + b'\n' + + def setUp(self): + self.old_stdout = sys.stdout + self.output = io.StringIO() + sys.stdout = self.output + + self.evt = threading.Event() + self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.sock.settimeout(15) + self.port = support.bind_port(self.sock) + servargs = (self.evt, self.respdata, self.sock) + threading.Thread(target=server, args=servargs).start() + self.evt.wait() + self.evt.clear() + + def tearDown(self): + self.evt.wait() + sys.stdout = self.old_stdout + + def testLineTooLong(self): + self.assertRaises(smtplib.SMTPResponseException, smtplib.SMTP, + HOST, self.port, 'localhost', 3) + + sim_users = {'Mr.A at somewhere.com':'John A', 'Ms.B at xn--fo-fka.com':'Sally B', 'Mrs.C at somewhereesle.com':'Ruth C', @@ -888,7 +915,8 @@ def test_main(verbose=None): support.run_unittest(GeneralTests, DebuggingServerTests, NonConnectingTests, - BadHELOServerTests, SMTPSimTests) + BadHELOServerTests, SMTPSimTests, + TooLongLineTests) if __name__ == '__main__': test_main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:19 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:19 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_BaseEventLoop_uses_again_the_resolution_of_the_clock_to_dec?= =?utf-8?q?ide_if?= Message-ID: <3fnSMW6tXWz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/6733d9dfffbb changeset: 89695:6733d9dfffbb branch: 3.4 user: Victor Stinner date: Mon Feb 10 23:42:32 2014 +0100 summary: Issue #20505: BaseEventLoop uses again the resolution of the clock to decide if scheduled tasks should be executed or not. files: Lib/asyncio/base_events.py | 5 +++-- Lib/test/test_asyncio/test_events.py | 12 ++++++------ 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -96,6 +96,7 @@ self._default_executor = None self._internal_fds = 0 self._running = False + self._clock_resolution = time.get_clock_info('monotonic').resolution def _make_socket_transport(self, sock, protocol, waiter=None, *, extra=None, server=None): @@ -643,10 +644,10 @@ self._process_events(event_list) # Handle 'later' callbacks that are ready. - now = self.time() + end_time = self.time() + self._clock_resolution while self._scheduled: handle = self._scheduled[0] - if handle._when > now: + if handle._when >= end_time: break handle = heapq.heappop(self._scheduled) self._ready.append(handle) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1178,15 +1178,15 @@ yield from asyncio.sleep(1e-4, loop=loop) yield from asyncio.sleep(1e-6, loop=loop) yield from asyncio.sleep(1e-8, loop=loop) + yield from asyncio.sleep(1e-10, loop=loop) self.loop.run_until_complete(wait()) - # The ideal number of call is 10, but on some platforms, the selector + # The ideal number of call is 12, but on some platforms, the selector # may sleep at little bit less than timeout depending on the resolution - # of the clock used by the kernel. Tolerate 5 useless calls on these - # platforms. - self.assertLessEqual(self.loop._run_once_counter, 15, - {'time_info': time.get_clock_info('time'), - 'monotonic_info': time.get_clock_info('monotonic'), + # of the clock used by the kernel. Tolerate a few useless calls on + # these platforms. + self.assertLessEqual(self.loop._run_once_counter, 20, + {'clock_resolution': self.loop._clock_resolution, 'selector': self.loop._selector.__class__.__name__}) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:21 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:21 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_Fix_TestLoop=2C_set_the_clock_resolution?= Message-ID: <3fnSMY1l4Hz7LkW@mail.python.org> http://hg.python.org/cpython/rev/375e4f9c9732 changeset: 89696:375e4f9c9732 branch: 3.4 user: Victor Stinner date: Tue Feb 11 09:03:47 2014 +0100 summary: Issue #20505: Fix TestLoop, set the clock resolution files: Lib/asyncio/test_utils.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py --- a/Lib/asyncio/test_utils.py +++ b/Lib/asyncio/test_utils.py @@ -191,6 +191,7 @@ self._gen = gen() next(self._gen) self._time = 0 + self._clock_resolution = 1e-9 self._timers = [] self._selector = TestSelector() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:22 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:22 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_Improve_debug_info_in_asyncio_event_loop?= Message-ID: <3fnSMZ3ZHkz7LkF@mail.python.org> http://hg.python.org/cpython/rev/e44ff3b7a497 changeset: 89697:e44ff3b7a497 branch: 3.4 user: Victor Stinner date: Tue Feb 11 10:08:08 2014 +0100 summary: Issue #20505: Improve debug info in asyncio event loop files: Lib/asyncio/base_events.py | 23 ++++++++++++++++++----- 1 files changed, 18 insertions(+), 5 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -634,12 +634,25 @@ else: logger.log(level, 'poll took %.3f seconds', t1-t0) else: - t0 = self.time() + t0_monotonic = time.monotonic() + t0 = time.perf_counter() event_list = self._selector.select(timeout) - dt = self.time() - t0 - if not event_list and timeout and dt < timeout: - print("asyncio: selector.select(%.3f ms) took %.3f ms" - % (timeout*1e3, dt*1e3), + dt = time.perf_counter() - t0 + dt_monotonic = time.monotonic() - t0_monotonic + if not event_list and timeout: # and dt < timeout: + selector = self._selector.__class__.__name__ + if (selector.startswith(("Poll", "Epoll", "Iocp")) + or timeout > 1e-3 or dt > 1e-3): + unit, factor = "ms", 1e3 + else: + unit, factor = "us", 1e6 + print("asyncio: %s.select(%.3f %s) took %.3f %s" + " (monotonic: %.3f %s, clock res: %.3f %s)" + % (self._selector.__class__.__name__, + timeout * factor, unit, + dt * factor, unit, + dt_monotonic * factor, unit, + self._clock_resolution * factor, unit), file=sys.__stderr__, flush=True) self._process_events(event_list) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:23 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:23 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_Oops=2C_only_print_debug_info_if_selector=2Eselect=28timeou?= =?utf-8?q?t=29_took_less?= Message-ID: <3fnSMb5Xpbz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/af840e781700 changeset: 89698:af840e781700 branch: 3.4 user: Victor Stinner date: Tue Feb 11 10:10:41 2014 +0100 summary: Issue #20505: Oops, only print debug info if selector.select(timeout) took less than timeout files: Lib/asyncio/base_events.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -639,7 +639,7 @@ event_list = self._selector.select(timeout) dt = time.perf_counter() - t0 dt_monotonic = time.monotonic() - t0_monotonic - if not event_list and timeout: # and dt < timeout: + if not event_list and timeout and dt < timeout: selector = self._selector.__class__.__name__ if (selector.startswith(("Poll", "Epoll", "Iocp")) or timeout > 1e-3 or dt > 1e-3): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:25 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:25 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_use_also_the_monotonic_time_to_decide_if_asyncio_debug_trac?= =?utf-8?q?es?= Message-ID: <3fnSMd0LcNz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/2faf4b7c52ed changeset: 89699:2faf4b7c52ed branch: 3.4 user: Victor Stinner date: Tue Feb 11 10:26:53 2014 +0100 summary: Issue #20505: use also the monotonic time to decide if asyncio debug traces should be printed files: Lib/asyncio/base_events.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -639,15 +639,16 @@ event_list = self._selector.select(timeout) dt = time.perf_counter() - t0 dt_monotonic = time.monotonic() - t0_monotonic - if not event_list and timeout and dt < timeout: + if (not event_list and timeout + and (dt < timeout or dt_monotonic < timeout)): selector = self._selector.__class__.__name__ if (selector.startswith(("Poll", "Epoll", "Iocp")) or timeout > 1e-3 or dt > 1e-3): unit, factor = "ms", 1e3 else: unit, factor = "us", 1e6 - print("asyncio: %s.select(%.3f %s) took %.3f %s" - " (monotonic: %.3f %s, clock res: %.3f %s)" + print("asyncio: %s.select(%.4f %s) took %.3f %s" + " (monotonic=%.3f %s, clock res=%.3f %s)" % (self._selector.__class__.__name__, timeout * factor, unit, dt * factor, unit, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:26 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:26 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbywgVHVs?= =?utf-8?q?ip_issue_126=3A_call=5Fsoon=28=29=2C_call=5Fsoon=5Fthreadsafe?= =?utf-8?b?KCksIGNhbGxfbGF0ZXIoKSw=?= Message-ID: <3fnSMf2xv3z7LkY@mail.python.org> http://hg.python.org/cpython/rev/0055d537b25e changeset: 89700:0055d537b25e branch: 3.4 user: Victor Stinner date: Tue Feb 11 11:34:30 2014 +0100 summary: asyncio, Tulip issue 126: call_soon(), call_soon_threadsafe(), call_later(), call_at() and run_in_executor() now raise a TypeError if the callback is a coroutine function. files: Lib/asyncio/base_events.py | 6 +++ Lib/asyncio/test_utils.py | 5 ++- Lib/test/test_asyncio/test_base_events.py | 18 ++++++++++ Lib/test/test_asyncio/test_proactor_events.py | 2 +- Lib/test/test_asyncio/test_selector_events.py | 9 ++-- Lib/test/test_asyncio/test_tasks.py | 12 ++--- 6 files changed, 39 insertions(+), 13 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -227,6 +227,8 @@ def call_at(self, when, callback, *args): """Like call_later(), but uses an absolute time.""" + if tasks.iscoroutinefunction(callback): + raise TypeError("coroutines cannot be used with call_at()") timer = events.TimerHandle(when, callback, args) heapq.heappush(self._scheduled, timer) return timer @@ -241,6 +243,8 @@ Any positional arguments after the callback will be passed to the callback when it is called. """ + if tasks.iscoroutinefunction(callback): + raise TypeError("coroutines cannot be used with call_soon()") handle = events.Handle(callback, args) self._ready.append(handle) return handle @@ -252,6 +256,8 @@ return handle def run_in_executor(self, executor, callback, *args): + if tasks.iscoroutinefunction(callback): + raise TypeError("coroutines cannot be used with run_in_executor()") if isinstance(callback, events.Handle): assert not args assert not isinstance(callback, events.TimerHandle) diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py --- a/Lib/asyncio/test_utils.py +++ b/Lib/asyncio/test_utils.py @@ -135,7 +135,7 @@ if name.startswith('__') and name.endswith('__'): # skip magic names continue - dct[name] = unittest.mock.Mock(return_value=None) + dct[name] = MockCallback(return_value=None) return type('TestProtocol', (base,) + base.__bases__, dct)() @@ -274,3 +274,6 @@ def _write_to_self(self): pass + +def MockCallback(**kwargs): + return unittest.mock.Mock(spec=['__call__'], **kwargs) diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -567,6 +567,7 @@ m_socket.getaddrinfo.return_value = [ (2, 1, 6, '', ('127.0.0.1', 10100))] + m_socket.getaddrinfo._is_coroutine = False m_sock = m_socket.socket.return_value = unittest.mock.Mock() m_sock.bind.side_effect = Err @@ -577,6 +578,7 @@ @unittest.mock.patch('asyncio.base_events.socket') def test_create_datagram_endpoint_no_addrinfo(self, m_socket): m_socket.getaddrinfo.return_value = [] + m_socket.getaddrinfo._is_coroutine = False coro = self.loop.create_datagram_endpoint( MyDatagramProto, local_addr=('localhost', 0)) @@ -681,6 +683,22 @@ unittest.mock.ANY, MyProto, sock, None, None) + def test_call_coroutine(self): + @asyncio.coroutine + def coroutine_function(): + pass + + with self.assertRaises(TypeError): + self.loop.call_soon(coroutine_function) + with self.assertRaises(TypeError): + self.loop.call_soon_threadsafe(coroutine_function) + with self.assertRaises(TypeError): + self.loop.call_later(60, coroutine_function) + with self.assertRaises(TypeError): + self.loop.call_at(self.loop.time() + 60, coroutine_function) + with self.assertRaises(TypeError): + self.loop.run_in_executor(None, coroutine_function) + if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -402,7 +402,7 @@ NotImplementedError, BaseProactorEventLoop, self.proactor) def test_make_socket_transport(self): - tr = self.loop._make_socket_transport(self.sock, unittest.mock.Mock()) + tr = self.loop._make_socket_transport(self.sock, asyncio.Protocol()) self.assertIsInstance(tr, _ProactorSocketTransport) def test_loop_self_reading(self): diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -44,8 +44,8 @@ def test_make_socket_transport(self): m = unittest.mock.Mock() self.loop.add_reader = unittest.mock.Mock() - self.assertIsInstance( - self.loop._make_socket_transport(m, m), _SelectorSocketTransport) + transport = self.loop._make_socket_transport(m, asyncio.Protocol()) + self.assertIsInstance(transport, _SelectorSocketTransport) @unittest.skipIf(ssl is None, 'No ssl module') def test_make_ssl_transport(self): @@ -54,8 +54,9 @@ self.loop.add_writer = unittest.mock.Mock() self.loop.remove_reader = unittest.mock.Mock() self.loop.remove_writer = unittest.mock.Mock() - self.assertIsInstance( - self.loop._make_ssl_transport(m, m, m, m), _SelectorSslTransport) + waiter = asyncio.Future(loop=self.loop) + transport = self.loop._make_ssl_transport(m, asyncio.Protocol(), m, waiter) + self.assertIsInstance(transport, _SelectorSslTransport) @unittest.mock.patch('asyncio.selector_events.ssl', None) def test_make_ssl_transport_without_ssl_error(self): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2,8 +2,6 @@ import gc import unittest -import unittest.mock -from unittest.mock import Mock import asyncio from asyncio import test_utils @@ -1358,7 +1356,7 @@ def _check_success(self, **kwargs): a, b, c = [asyncio.Future(loop=self.one_loop) for i in range(3)] fut = asyncio.gather(*self.wrap_futures(a, b, c), **kwargs) - cb = Mock() + cb = test_utils.MockCallback() fut.add_done_callback(cb) b.set_result(1) a.set_result(2) @@ -1380,7 +1378,7 @@ def test_one_exception(self): a, b, c, d, e = [asyncio.Future(loop=self.one_loop) for i in range(5)] fut = asyncio.gather(*self.wrap_futures(a, b, c, d, e)) - cb = Mock() + cb = test_utils.MockCallback() fut.add_done_callback(cb) exc = ZeroDivisionError() a.set_result(1) @@ -1399,7 +1397,7 @@ a, b, c, d = [asyncio.Future(loop=self.one_loop) for i in range(4)] fut = asyncio.gather(*self.wrap_futures(a, b, c, d), return_exceptions=True) - cb = Mock() + cb = test_utils.MockCallback() fut.add_done_callback(cb) exc = ZeroDivisionError() exc2 = RuntimeError() @@ -1460,7 +1458,7 @@ def test_one_cancellation(self): a, b, c, d, e = [asyncio.Future(loop=self.one_loop) for i in range(5)] fut = asyncio.gather(a, b, c, d, e) - cb = Mock() + cb = test_utils.MockCallback() fut.add_done_callback(cb) a.set_result(1) b.cancel() @@ -1479,7 +1477,7 @@ a, b, c, d, e, f = [asyncio.Future(loop=self.one_loop) for i in range(6)] fut = asyncio.gather(a, b, c, d, e, f, return_exceptions=True) - cb = Mock() + cb = test_utils.MockCallback() fut.add_done_callback(cb) a.set_result(1) zde = ZeroDivisionError() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:27 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:27 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbywgVHVs?= =?utf-8?q?ip_issue_130=3A_Add_more_checks_on_subprocess=5Fexec/subprocess?= =?utf-8?q?=5Fshell?= Message-ID: <3fnSMg5Nqlz7Lkb@mail.python.org> http://hg.python.org/cpython/rev/85db10a4a355 changeset: 89701:85db10a4a355 branch: 3.4 user: Victor Stinner date: Tue Feb 11 11:44:56 2014 +0100 summary: asyncio, Tulip issue 130: Add more checks on subprocess_exec/subprocess_shell parameters files: Lib/asyncio/base_events.py | 12 +- Lib/asyncio/subprocess.py | 5 +- Lib/test/test_asyncio/test_base_events.py | 54 ++++++++++- 3 files changed, 64 insertions(+), 7 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -558,7 +558,7 @@ stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False, shell=True, bufsize=0, **kwargs): - if not isinstance(cmd, str): + if not isinstance(cmd, (bytes, str)): raise ValueError("cmd must be a string") if universal_newlines: raise ValueError("universal_newlines must be False") @@ -572,7 +572,7 @@ return transport, protocol @tasks.coroutine - def subprocess_exec(self, protocol_factory, *args, stdin=subprocess.PIPE, + def subprocess_exec(self, protocol_factory, program, *args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False, shell=False, bufsize=0, **kwargs): @@ -582,9 +582,15 @@ raise ValueError("shell must be False") if bufsize != 0: raise ValueError("bufsize must be 0") + popen_args = (program,) + args + for arg in popen_args: + if not isinstance(arg, (str, bytes)): + raise TypeError("program arguments must be " + "a bytes or text string, not %s" + % type(arg).__name__) protocol = protocol_factory() transport = yield from self._make_subprocess_transport( - protocol, args, False, stdin, stdout, stderr, bufsize, **kwargs) + protocol, popen_args, False, stdin, stdout, stderr, bufsize, **kwargs) return transport, protocol def _add_callback(self, handle): diff --git a/Lib/asyncio/subprocess.py b/Lib/asyncio/subprocess.py --- a/Lib/asyncio/subprocess.py +++ b/Lib/asyncio/subprocess.py @@ -180,7 +180,7 @@ return Process(transport, protocol, loop) @tasks.coroutine -def create_subprocess_exec(*args, stdin=None, stdout=None, stderr=None, +def create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, loop=None, limit=streams._DEFAULT_LIMIT, **kwds): if loop is None: loop = events.get_event_loop() @@ -188,7 +188,8 @@ loop=loop) transport, protocol = yield from loop.subprocess_exec( protocol_factory, - *args, stdin=stdin, stdout=stdout, + program, *args, + stdin=stdin, stdout=stdout, stderr=stderr, **kwds) yield from protocol.waiter return Process(transport, protocol, loop) diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -3,6 +3,7 @@ import errno import logging import socket +import sys import time import unittest import unittest.mock @@ -234,8 +235,57 @@ self.assertEqual([handle], list(self.loop._ready)) def test_run_until_complete_type_error(self): - self.assertRaises( - TypeError, self.loop.run_until_complete, 'blah') + self.assertRaises(TypeError, + self.loop.run_until_complete, 'blah') + + def test_subprocess_exec_invalid_args(self): + args = [sys.executable, '-c', 'pass'] + + # missing program parameter (empty args) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol) + + # exepected multiple arguments, not a list + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol, args) + + # program arguments must be strings, not int + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol, sys.executable, 123) + + # universal_newlines, shell, bufsize must not be set + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol, *args, universal_newlines=True) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol, *args, shell=True) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_exec, + asyncio.SubprocessProtocol, *args, bufsize=4096) + + def test_subprocess_shell_invalid_args(self): + # exepected a string, not an int or a list + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_shell, + asyncio.SubprocessProtocol, 123) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_shell, + asyncio.SubprocessProtocol, [sys.executable, '-c', 'pass']) + + # universal_newlines, shell, bufsize must not be set + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_shell, + asyncio.SubprocessProtocol, 'exit 0', universal_newlines=True) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_shell, + asyncio.SubprocessProtocol, 'exit 0', shell=True) + self.assertRaises(TypeError, + self.loop.run_until_complete, self.loop.subprocess_shell, + asyncio.SubprocessProtocol, 'exit 0', bufsize=4096) class MyProto(asyncio.Protocol): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:29 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:29 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbywgVHVs?= =?utf-8?q?ip_issue_131=3A_as=5Fcompleted=28=29_and_wait=28=29_now_raises_?= =?utf-8?q?a_TypeError_if?= Message-ID: <3fnSMj6yClz7LkL@mail.python.org> http://hg.python.org/cpython/rev/5f7a601f2245 changeset: 89702:5f7a601f2245 branch: 3.4 user: Victor Stinner date: Tue Feb 11 11:54:08 2014 +0100 summary: asyncio, Tulip issue 131: as_completed() and wait() now raises a TypeError if the list of futures is not a list but a Future, Task or coroutine object files: Lib/asyncio/tasks.py | 4 ++ Lib/test/test_asyncio/test_tasks.py | 26 +++++++++++++++++ 2 files changed, 30 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -358,6 +358,8 @@ Note: This does not raise TimeoutError! Futures that aren't done when the timeout occurs are returned in the second set. """ + if isinstance(fs, futures.Future) or iscoroutine(fs): + raise TypeError("expect a list of futures, not %s" % type(fs).__name__) if not fs: raise ValueError('Set of coroutines/Futures is empty.') @@ -474,6 +476,8 @@ Note: The futures 'f' are not necessarily members of fs. """ + if isinstance(fs, futures.Future) or iscoroutine(fs): + raise TypeError("expect a list of futures, not %s" % type(fs).__name__) loop = loop if loop is not None else events.get_event_loop() deadline = None if timeout is None else loop.time() + timeout todo = {async(f, loop=loop) for f in set(fs)} diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -7,6 +7,11 @@ from asyncio import test_utils + at asyncio.coroutine +def coroutine_function(): + pass + + class Dummy: def __repr__(self): @@ -1338,6 +1343,27 @@ child2.set_result(2) test_utils.run_briefly(self.loop) + def test_as_completed_invalid_args(self): + fut = asyncio.Future(loop=self.loop) + + # as_completed() expects a list of futures, not a future instance + self.assertRaises(TypeError, self.loop.run_until_complete, + asyncio.as_completed(fut, loop=self.loop)) + self.assertRaises(TypeError, self.loop.run_until_complete, + asyncio.as_completed(coroutine_function(), loop=self.loop)) + + def test_wait_invalid_args(self): + fut = asyncio.Future(loop=self.loop) + + # wait() expects a list of futures, not a future instance + self.assertRaises(TypeError, self.loop.run_until_complete, + asyncio.wait(fut, loop=self.loop)) + self.assertRaises(TypeError, self.loop.run_until_complete, + asyncio.wait(coroutine_function(), loop=self.loop)) + + # wait() expects at least a future + self.assertRaises(ValueError, self.loop.run_until_complete, + asyncio.wait([], loop=self.loop)) class GatherTestsBase: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:31 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:31 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogbWVyZ2UgMy4zICgj?= =?utf-8?q?20594=29?= Message-ID: <3fnSMl1PSdz7LkD@mail.python.org> http://hg.python.org/cpython/rev/fd49c1d2fd6c changeset: 89703:fd49c1d2fd6c branch: 3.4 user: Benjamin Peterson date: Tue Feb 11 10:19:12 2014 -0500 summary: merge 3.3 (#20594) files: Misc/NEWS | 2 ++ Modules/posixmodule.c | 10 +++++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,8 @@ Library ------- +- Issue #20594: Avoid name clash with the libc function posix_close. + What's New in Python 3.4.0 release candidate 1? =============================================== diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7763,8 +7763,12 @@ "close(fd)\n\n\ Close a file descriptor (for low level IO)."); -static PyObject * -posix_close(PyObject *self, PyObject *args) +/* +The underscore at end of function name avoids a name clash with the libc +function posix_close. +*/ +static PyObject * +posix_close_(PyObject *self, PyObject *args) { int fd, res; if (!PyArg_ParseTuple(args, "i:close", &fd)) @@ -11422,7 +11426,7 @@ {"open", (PyCFunction)posix_open,\ METH_VARARGS | METH_KEYWORDS, posix_open__doc__}, - {"close", posix_close, METH_VARARGS, posix_close__doc__}, + {"close", posix_close_, METH_VARARGS, posix_close__doc__}, {"closerange", posix_closerange, METH_VARARGS, posix_closerange__doc__}, {"device_encoding", device_encoding, METH_VARARGS, device_encoding__doc__}, {"dup", posix_dup, METH_VARARGS, posix_dup__doc__}, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:32 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:32 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTA1?= =?utf-8?q?=3A_Remove_debug_code?= Message-ID: <3fnSMm3FYbz7LkM@mail.python.org> http://hg.python.org/cpython/rev/702b20fa7af2 changeset: 89704:702b20fa7af2 branch: 3.4 user: Victor Stinner date: Tue Feb 11 17:53:47 2014 +0100 summary: Issue #20505: Remove debug code files: Lib/asyncio/base_events.py | 20 -------------------- 1 files changed, 0 insertions(+), 20 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -646,27 +646,7 @@ else: logger.log(level, 'poll took %.3f seconds', t1-t0) else: - t0_monotonic = time.monotonic() - t0 = time.perf_counter() event_list = self._selector.select(timeout) - dt = time.perf_counter() - t0 - dt_monotonic = time.monotonic() - t0_monotonic - if (not event_list and timeout - and (dt < timeout or dt_monotonic < timeout)): - selector = self._selector.__class__.__name__ - if (selector.startswith(("Poll", "Epoll", "Iocp")) - or timeout > 1e-3 or dt > 1e-3): - unit, factor = "ms", 1e3 - else: - unit, factor = "us", 1e6 - print("asyncio: %s.select(%.4f %s) took %.3f %s" - " (monotonic=%.3f %s, clock res=%.3f %s)" - % (self._selector.__class__.__name__, - timeout * factor, unit, - dt * factor, unit, - dt_monotonic * factor, unit, - self._clock_resolution * factor, unit), - file=sys.__stderr__, flush=True) self._process_events(event_list) # Handle 'later' callbacks that are ready. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:33 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:33 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNDk1?= =?utf-8?q?=3A_Skip_test=5Fread=5Fpty=5Foutput=28=29_of_test=5Fasyncio_on_?= =?utf-8?q?FreeBSD_older_than?= Message-ID: <3fnSMn4zn1z7Ljh@mail.python.org> http://hg.python.org/cpython/rev/2cf25865fc66 changeset: 89705:2cf25865fc66 branch: 3.4 user: Victor Stinner date: Tue Feb 11 18:40:56 2014 +0100 summary: Issue #20495: Skip test_read_pty_output() of test_asyncio on FreeBSD older than FreeBSD 8 files: Lib/test/test_asyncio/test_events.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -956,6 +956,8 @@ # select, poll and kqueue don't support character devices (PTY) on Mac OS X # older than 10.6 (Snow Leopard) @support.requires_mac_ver(10, 6) + # Issue #20495: The test hangs on FreeBSD 7.2 but pass on FreeBSD 9 + @support.requires_freebsd_version(8) def test_read_pty_output(self): proto = None -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:34 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:34 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpby5ldmVu?= =?utf-8?q?ts=3A_Use_=5F=5Fslots=5F=5F_in_Handle_and_TimerHandle?= Message-ID: <3fnSMp6Rbmz7LjY@mail.python.org> http://hg.python.org/cpython/rev/dcaa7a26eed3 changeset: 89706:dcaa7a26eed3 branch: 3.4 user: Yury Selivanov date: Wed Feb 12 17:01:52 2014 -0500 summary: asyncio.events: Use __slots__ in Handle and TimerHandle files: Lib/asyncio/events.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -19,6 +19,8 @@ class Handle: """Object returned by callback registration methods.""" + __slots__ = ['_callback', '_args', '_cancelled'] + def __init__(self, callback, args): assert not isinstance(callback, Handle), 'A Handle is not a callback' self._callback = callback @@ -46,6 +48,8 @@ class TimerHandle(Handle): """Object returned by timed callback registration methods.""" + __slots__ = ['_when'] + def __init__(self, when, callback, args): assert when is not None super().__init__(callback, args) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:36 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:36 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbzogQ2hh?= =?utf-8?q?nge_as=5Fcompleted=28=29_to_use_a_Queue=2C_to_avoid_O=28N**2=29?= =?utf-8?q?_behavior=2E_Fixes?= Message-ID: <3fnSMr1rNmz7Lkg@mail.python.org> http://hg.python.org/cpython/rev/b52113fb58a5 changeset: 89707:b52113fb58a5 branch: 3.4 user: Guido van Rossum date: Wed Feb 12 17:58:19 2014 -0800 summary: asyncio: Change as_completed() to use a Queue, to avoid O(N**2) behavior. Fixes issue #20566. files: Lib/asyncio/tasks.py | 53 ++++++++++------ Lib/test/test_asyncio/test_tasks.py | 23 +++++++- 2 files changed, 55 insertions(+), 21 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -463,7 +463,11 @@ # This is *not* a @coroutine! It is just an iterator (yielding Futures). def as_completed(fs, *, loop=None, timeout=None): - """Return an iterator whose values, when waited for, are Futures. + """Return an iterator whose values are coroutines. + + When waiting for the yielded coroutines you'll get the results (or + exceptions!) of the original Futures (or coroutines), in the order + in which and as soon as they complete. This differs from PEP 3148; the proper way to use this is: @@ -471,8 +475,8 @@ result = yield from f # The 'yield from' may raise. # Use result. - Raises TimeoutError if the timeout occurs before all Futures are - done. + If a timeout is specified, the 'yield from' will raise + TimeoutError when the timeout occurs before all Futures are done. Note: The futures 'f' are not necessarily members of fs. """ @@ -481,27 +485,36 @@ loop = loop if loop is not None else events.get_event_loop() deadline = None if timeout is None else loop.time() + timeout todo = {async(f, loop=loop) for f in set(fs)} - completed = collections.deque() + from .queues import Queue # Import here to avoid circular import problem. + done = Queue(loop=loop) + timeout_handle = None + + def _on_timeout(): + for f in todo: + f.remove_done_callback(_on_completion) + done.put_nowait(None) # Queue a dummy value for _wait_for_one(). + todo.clear() # Can't do todo.remove(f) in the loop. + + def _on_completion(f): + if not todo: + return # _on_timeout() was here first. + todo.remove(f) + done.put_nowait(f) + if not todo and timeout_handle is not None: + timeout_handle.cancel() @coroutine def _wait_for_one(): - while not completed: - timeout = None - if deadline is not None: - timeout = deadline - loop.time() - if timeout < 0: - raise futures.TimeoutError() - done, pending = yield from _wait( - todo, timeout, FIRST_COMPLETED, loop) - # Multiple callers might be waiting for the same events - # and getting the same outcome. Dedupe by updating todo. - for f in done: - if f in todo: - todo.remove(f) - completed.append(f) - f = completed.popleft() - return f.result() # May raise. + f = yield from done.get() + if f is None: + # Dummy value from _on_timeout(). + raise futures.TimeoutError + return f.result() # May raise f.exception(). + for f in todo: + f.add_done_callback(_on_completion) + if todo and timeout is not None: + timeout_handle = loop.call_later(timeout, _on_timeout) for _ in range(len(todo)): yield _wait_for_one() diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -779,7 +779,6 @@ yield 0 yield 0 yield 0.1 - yield 0.02 loop = test_utils.TestLoop(gen) self.addCleanup(loop.close) @@ -791,6 +790,8 @@ def foo(): values = [] for f in asyncio.as_completed([a, b], timeout=0.12, loop=loop): + if values: + loop.advance_time(0.02) try: v = yield from f values.append((1, v)) @@ -809,6 +810,26 @@ loop.advance_time(10) loop.run_until_complete(asyncio.wait([a, b], loop=loop)) + def test_as_completed_with_unused_timeout(self): + + def gen(): + yield + yield 0 + yield 0.01 + + loop = test_utils.TestLoop(gen) + self.addCleanup(loop.close) + + a = asyncio.sleep(0.01, 'a', loop=loop) + + @asyncio.coroutine + def foo(): + for f in asyncio.as_completed([a], timeout=1, loop=loop): + v = yield from f + self.assertEqual(v, 'a') + + res = loop.run_until_complete(asyncio.Task(foo(), loop=loop)) + def test_as_completed_reverse_wait(self): def gen(): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:37 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:37 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXluY2lvLCBUdWxp?= =?utf-8?q?p_issue_129=3A_BaseEventLoop=2Esock=5Fconnect=28=29_now_raises_?= =?utf-8?q?an_error_if?= Message-ID: <3fnSMs4mRQz7LkL@mail.python.org> http://hg.python.org/cpython/rev/b4c75a9f1541 changeset: 89708:b4c75a9f1541 branch: 3.4 user: Victor Stinner date: Thu Feb 13 09:24:37 2014 +0100 summary: ayncio, Tulip issue 129: BaseEventLoop.sock_connect() now raises an error if the address is not resolved (hostname instead of an IP address) for AF_INET and AF_INET6 address families. files: Doc/library/asyncio-eventloop.rst | 6 +++ Lib/asyncio/base_events.py | 25 ++++++++++++++++ Lib/asyncio/proactor_events.py | 9 +++++- Lib/asyncio/selector_events.py | 20 +++++------- Lib/test/test_asyncio/test_events.py | 12 +++++++ 5 files changed, 59 insertions(+), 13 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -366,6 +366,12 @@ Connect to a remote socket at *address*. + The *address* must be already resolved to avoid the trap of hanging the + entire event loop when the address requires doing a DNS lookup. For + example, it must be an IP address, not an hostname, for + :py:data:`~socket.AF_INET` and :py:data:`~socket.AF_INET6` address families. + Use :meth:`getaddrinfo` to resolve the hostname asynchronously. + This method returns a :ref:`coroutine object `. .. seealso:: diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -41,6 +41,31 @@ """Raised to stop the event loop.""" +def _check_resolved_address(sock, address): + # Ensure that the address is already resolved to avoid the trap of hanging + # the entire event loop when the address requires doing a DNS lookup. + family = sock.family + if family not in (socket.AF_INET, socket.AF_INET6): + return + + host, port = address + type_mask = 0 + if hasattr(socket, 'SOCK_NONBLOCK'): + type_mask |= socket.SOCK_NONBLOCK + if hasattr(socket, 'SOCK_CLOEXEC'): + type_mask |= socket.SOCK_CLOEXEC + # Use getaddrinfo(AI_NUMERICHOST) to ensure that the address is + # already resolved. + try: + socket.getaddrinfo(host, port, + family=family, + type=(sock.type & ~type_mask), + proto=sock.proto, + flags=socket.AI_NUMERICHOST) + except socket.gaierror as err: + raise ValueError("address must be resolved (IP address), got %r: %s" + % (address, err)) + def _raise_stop_error(*args): raise _StopError diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -404,7 +404,14 @@ return self._proactor.send(sock, data) def sock_connect(self, sock, address): - return self._proactor.connect(sock, address) + try: + base_events._check_resolved_address(sock, address) + except ValueError as err: + fut = futures.Future(loop=self) + fut.set_exception(err) + return fut + else: + return self._proactor.connect(sock, address) def sock_accept(self, sock): return self._proactor.accept(sock) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -208,6 +208,8 @@ return fut def _sock_recv(self, fut, registered, sock, n): + # _sock_recv() can add itself as an I/O callback if the operation can't + # be done immediatly. Don't use it directly, call sock_recv(). fd = sock.fileno() if registered: # Remove the callback early. It should be rare that the @@ -260,22 +262,16 @@ def sock_connect(self, sock, address): """XXX""" - # That address better not require a lookup! We're not calling - # self.getaddrinfo() for you here. But verifying this is - # complicated; the socket module doesn't have a pattern for - # IPv6 addresses (there are too many forms, apparently). fut = futures.Future(loop=self) - self._sock_connect(fut, False, sock, address) + try: + base_events._check_resolved_address(sock, address) + except ValueError as err: + fut.set_exception(err) + else: + self._sock_connect(fut, False, sock, address) return fut def _sock_connect(self, fut, registered, sock, address): - # TODO: Use getaddrinfo() to look up the address, to avoid the - # trap of hanging the entire event loop when the address - # requires doing a DNS lookup. (OTOH, the caller should - # already have done this, so it would be nice if we could - # easily tell whether the address needs looking up or not. I - # know how to do this for IPv4, but IPv6 addresses have many - # syntaxes.) fd = sock.fileno() if registered: self.remove_writer(fd) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1191,6 +1191,18 @@ {'clock_resolution': self.loop._clock_resolution, 'selector': self.loop._selector.__class__.__name__}) + def test_sock_connect_address(self): + address = ('www.python.org', 80) + for family in (socket.AF_INET, socket.AF_INET6): + for sock_type in (socket.SOCK_STREAM, socket.SOCK_DGRAM): + sock = socket.socket(family, sock_type) + with sock: + connect = self.loop.sock_connect(sock, address) + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(connect) + self.assertIn('address must be resolved', + str(cm.exception)) + class SubprocessTestsMixin: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:38 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:38 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRml4IHRlc3RfYXN5?= =?utf-8?q?ncio/test=5Fevents=2Epy=3A_skip_IPv6_if_IPv6_is_disabled_on_the?= =?utf-8?q?_host?= Message-ID: <3fnSMt6Wtmz7LjV@mail.python.org> http://hg.python.org/cpython/rev/778c5dd644d1 changeset: 89709:778c5dd644d1 branch: 3.4 user: Victor Stinner date: Thu Feb 13 10:46:05 2014 +0100 summary: Fix test_asyncio/test_events.py: skip IPv6 if IPv6 is disabled on the host files: Lib/test/test_asyncio/test_events.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -1192,8 +1192,12 @@ 'selector': self.loop._selector.__class__.__name__}) def test_sock_connect_address(self): + families = [socket.AF_INET] + if support.IPV6_ENABLED: + families.append(socket.AF_INET6) + address = ('www.python.org', 80) - for family in (socket.AF_INET, socket.AF_INET6): + for family in families: for sock_type in (socket.SOCK_STREAM, socket.SOCK_DGRAM): sock = socket.socket(family, sock_type) with sock: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:40 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:40 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNTI2?= =?utf-8?q?=2C_=2319466=3A_Revert_changes_of_issue_=2319466_which_introduc?= =?utf-8?q?es_a?= Message-ID: <3fnSMw2QtZz7LkM@mail.python.org> http://hg.python.org/cpython/rev/9ce58a73b6b5 changeset: 89710:9ce58a73b6b5 branch: 3.4 user: Victor Stinner date: Thu Feb 13 12:48:54 2014 +0100 summary: Issue #20526, #19466: Revert changes of issue #19466 which introduces a regression: don't clear anymore the state of Python threads early during the Python shutdown. files: Lib/test/test_threading.py | 49 -------------------------- Misc/NEWS | 4 ++ Python/pythonrun.c | 20 ++------- 3 files changed, 9 insertions(+), 64 deletions(-) diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -616,51 +616,6 @@ t.join() self.assertRaises(ValueError, bs.release) - def test_locals_at_exit(self): - # Issue #19466: thread locals must not be deleted before destructors - # are called - rc, out, err = assert_python_ok("-c", """if 1: - import threading - - class Atexit: - def __del__(self): - print("thread_dict.atexit = %r" % thread_dict.atexit) - - thread_dict = threading.local() - thread_dict.atexit = "atexit" - - atexit = Atexit() - """) - self.assertEqual(out.rstrip(), b"thread_dict.atexit = 'atexit'") - - def test_warnings_at_exit(self): - # Issue #19466: try to call most destructors at Python shutdown before - # destroying Python thread states - filename = __file__ - rc, out, err = assert_python_ok("-Wd", "-c", """if 1: - import time - import threading - - def open_sleep(): - # a warning will be emitted when the open file will be - # destroyed (without being explicitly closed) while the daemon - # thread is destroyed - fileobj = open(%a, 'rb') - start_event.set() - time.sleep(60.0) - - start_event = threading.Event() - - thread = threading.Thread(target=open_sleep) - thread.daemon = True - thread.start() - - # wait until the thread started - start_event.wait() - """ % filename) - self.assertRegex(err.rstrip(), - b"^sys:1: ResourceWarning: unclosed file ") - @cpython_only def test_frame_tstate_tracing(self): # Issue #14432: Crash when a generator is created in a C thread that is @@ -786,10 +741,6 @@ import sys import time import threading - import warnings - - # ignore "unclosed file ..." warnings - warnings.filterwarnings('ignore', '', ResourceWarning) thread_has_run = set() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #20526: Revert changes of issue #19466 which introduces a regression: + don't clear anymore the state of Python threads early during the Python + shutdown. + Library ------- diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -582,13 +582,11 @@ _Py_Finalizing = tstate; initialized = 0; - /* Destroy the state of all threads except of the current thread: in - practice, only daemon threads should still be alive. Clear frames of - other threads to call objects destructor. Destructors will be called in - the current Python thread. Since _Py_Finalizing has been set, no other - Python threads can lock the GIL at this point (if they try, they will - exit immediately). */ - _PyThreadState_DeleteExcept(tstate); + /* Flush stdout+stderr */ + flush_std_files(); + + /* Disable signal handling */ + PyOS_FiniInterrupts(); /* Collect garbage. This may call finalizers; it's nice to call these * before all modules are destroyed. @@ -603,7 +601,6 @@ * XXX I haven't seen a real-life report of either of these. */ PyGC_Collect(); - #ifdef COUNT_ALLOCS /* With COUNT_ALLOCS, it helps to run GC multiple times: each collection might release some types from the type @@ -611,13 +608,6 @@ while (PyGC_Collect() > 0) /* nothing */; #endif - - /* Flush stdout+stderr */ - flush_std_files(); - - /* Disable signal handling */ - PyOS_FiniInterrupts(); - /* Destroy all modules */ PyImport_Cleanup(); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:41 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:41 +0100 (CET) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_set_line_and_c?= =?utf-8?q?olumn_numbers_for_keyword-only_arg_nodes_=28closes_=2320619=29?= Message-ID: <3fnSMx4Njvz7Lkb@mail.python.org> http://hg.python.org/cpython/rev/3c01209ab697 changeset: 89711:3c01209ab697 branch: 3.4 user: Benjamin Peterson date: Thu Feb 13 19:22:14 2014 -0500 summary: set line and column numbers for keyword-only arg nodes (closes #20619) files: Lib/test/test_ast.py | 4 ++-- Misc/NEWS | 3 +++ Python/ast.c | 2 ++ 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -37,7 +37,7 @@ # FunctionDef with kwargs "def f(**kwargs): pass", # FunctionDef with all kind of args - "def f(a, b=1, c=None, d=[], e={}, *args, **kwargs): pass", + "def f(a, b=1, c=None, d=[], e={}, *args, f=42, **kwargs): pass", # ClassDef "class C:pass", # ClassDef, new style class @@ -973,7 +973,7 @@ ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None)], None, [], [], None, [('Num', (1, 8), 0)]), [('Pass', (1, 12))], [], None)]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], ('arg', (1, 7), 'args', None), [], [], None, []), [('Pass', (1, 14))], [], None)]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], ('arg', (1, 8), 'kwargs', None), []), [('Pass', (1, 17))], [], None)]), -('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None), ('arg', (1, 9), 'b', None), ('arg', (1, 14), 'c', None), ('arg', (1, 22), 'd', None), ('arg', (1, 28), 'e', None)], ('arg', (1, 35), 'args', None), [], [], ('arg', (1, 43), 'kwargs', None), [('Num', (1, 11), 1), ('NameConstant', (1, 16), None), ('List', (1, 24), [], ('Load',)), ('Dict', (1, 30), [], [])]), [('Pass', (1, 52))], [], None)]), + ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6), 'a', None), ('arg', (1, 9), 'b', None), ('arg', (1, 14), 'c', None), ('arg', (1, 22), 'd', None), ('arg', (1, 28), 'e', None)], ('arg', (1, 35), 'args', None), [('arg', (1, 41), 'f', None)], [('Num', (1, 43), 42)], ('arg', (1, 49), 'kwargs', None), [('Num', (1, 11), 1), ('NameConstant', (1, 16), None), ('List', (1, 24), [], ('Load',)), ('Dict', (1, 30), [], [])]), [('Pass', (1, 58))], [], None)]), ('Module', [('ClassDef', (1, 0), 'C', [], [], None, None, [('Pass', (1, 8))], [])]), ('Module', [('ClassDef', (1, 0), 'C', [('Name', (1, 8), 'object', ('Load',))], [], None, None, [('Pass', (1, 17))], [])]), ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [], None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [], None)]), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #20619: Give the AST nodes of keyword-only arguments a column and line + number. + - Issue #20526: Revert changes of issue #19466 which introduces a regression: don't clear anymore the state of Python threads early during the Python shutdown. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -1203,6 +1203,8 @@ arg = arg(argname, annotation, c->c_arena); if (!arg) goto error; + arg->lineno = LINENO(ch); + arg->col_offset = ch->n_col_offset; asdl_seq_SET(kwonlyargs, j++, arg); i += 2; /* the name and the comma */ break; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Mar 17 07:31:42 2014 From: python-checkins at python.org (larry.hastings) Date: Mon, 17 Mar 2014 07:31:42 +0100 (CET) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE5NzQ0?= =?utf-8?q?=3A_Handle_missing_SSL/TLS_in_ensurepip?= Message-ID: <3fnSMy62Mjz7LjX@mail.python.org> http://hg.python.org/c