From python-checkins at python.org  Thu Dec  1 00:22:04 2011
From: python-checkins at python.org (nadeem.vawda)
Date: Thu, 01 Dec 2011 00:22:04 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Credit_Per_=C3=98yvind_Karl?=
 =?utf8?q?sen_for_the_initial_implementation_of_the_lzma_module?=
Message-ID: <E1RVtTY-0006TF-Ns@dinsdale.python.org>

http://hg.python.org/cpython/rev/6cde416ef03b
changeset:   73798:6cde416ef03b
user:        Nadeem Vawda <nadeem.vawda at gmail.com>
date:        Thu Dec 01 01:18:27 2011 +0200
summary:
  Credit Per ?yvind Karlsen for the initial implementation of the lzma module (issue #6715).

files:
  Misc/ACKS             |  1 +
  Misc/NEWS             |  1 +
  Modules/_lzmamodule.c |  7 ++++++-
  3 files changed, 8 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -502,6 +502,7 @@
 Peter van Kampen
 Rafe Kaplan
 Jacob Kaplan-Moss
+Per ?yvind Karlsen
 Lou Kates
 Hiroaki Kawai
 Sebastien Keim
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -400,6 +400,7 @@
 -------
 
 - Issue #6715: Add a module 'lzma' for compression using the LZMA algorithm.
+  Thanks to Per ?yvind Karlsen for the initial implementation.
 
 - Issue #13487: Make inspect.getmodule robust against changes done to
   sys.modules while it is iterating over it.
diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c
--- a/Modules/_lzmamodule.c
+++ b/Modules/_lzmamodule.c
@@ -1,4 +1,9 @@
-/* _lzma - Low-level Python interface to liblzma. */
+/* _lzma - Low-level Python interface to liblzma.
+
+   Initial implementation by Per ?yvind Karlsen.
+   Rewritten by Nadeem Vawda.
+
+*/
 
 #define PY_SSIZE_T_CLEAN
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 02:52:11 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 02:52:11 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_PyObject=5FRepr=28=29_ensur?=
 =?utf8?q?es_that_the_result_is_a_ready_Unicode_string?=
Message-ID: <E1RVvop-0000cV-Gx@dinsdale.python.org>

http://hg.python.org/cpython/rev/c5d2ce38b0d3
changeset:   73799:c5d2ce38b0d3
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 02:15:00 2011 +0100
summary:
  PyObject_Repr() ensures that the result is a ready Unicode string

And PyObject_Str() and PyObject_Repr() don't make strings ready in debug
mode to ensure that the caller makes the string ready before using it.

files:
  Objects/object.c |  8 ++++++++
  1 files changed, 8 insertions(+), 0 deletions(-)


diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -385,6 +385,10 @@
         Py_DECREF(res);
         return NULL;
     }
+#ifndef Py_DEBUG
+    if (PyUnicode_READY(res) < 0)
+        return NULL;
+#endif
     return res;
 }
 
@@ -403,8 +407,10 @@
     if (v == NULL)
         return PyUnicode_FromString("<NULL>");
     if (PyUnicode_CheckExact(v)) {
+#ifndef Py_DEBUG
         if (PyUnicode_READY(v) < 0)
             return NULL;
+#endif
         Py_INCREF(v);
         return v;
     }
@@ -426,8 +432,10 @@
         Py_DECREF(res);
         return NULL;
     }
+#ifndef Py_DEBUG
     if (PyUnicode_READY(res) < 0)
         return NULL;
+#endif
     assert(_PyUnicode_CheckConsistency(res, 1));
     return res;
 }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 02:52:12 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 02:52:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_PyCodec=5FIgnoreErrors=28?=
 =?utf8?q?=29_avoids_the_deprecated_=22u=23=22_format?=
Message-ID: <E1RVvoq-0000cf-7g@dinsdale.python.org>

http://hg.python.org/cpython/rev/6407294eb3c6
changeset:   73800:6407294eb3c6
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 02:52:11 2011 +0100
summary:
  PyCodec_IgnoreErrors() avoids the deprecated "u#" format

files:
  Python/codecs.c |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Python/codecs.c b/Python/codecs.c
--- a/Python/codecs.c
+++ b/Python/codecs.c
@@ -510,8 +510,7 @@
         wrong_exception_type(exc);
         return NULL;
     }
-    /* ouch: passing NULL, 0, pos gives None instead of u'' */
-    return Py_BuildValue("(u#n)", &end, 0, end);
+    return Py_BuildValue("(Nn)", PyUnicode_New(0, 0), end);
 }
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 02:52:12 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 02:52:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_c-api=3A_Replace_PyUnicodeO?=
 =?utf8?q?bject*_by_PyObject*_in_=22U=22_format_doc?=
Message-ID: <E1RVvoq-0000cj-U7@dinsdale.python.org>

http://hg.python.org/cpython/rev/ba8e7886fdd7
changeset:   73801:ba8e7886fdd7
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 02:52:55 2011 +0100
summary:
  c-api: Replace PyUnicodeObject* by PyObject* in "U" format doc

files:
  Doc/c-api/arg.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst
--- a/Doc/c-api/arg.rst
+++ b/Doc/c-api/arg.rst
@@ -146,7 +146,7 @@
    Like ``u#``, but the Python object may also be ``None``, in which case the
    :c:type:`Py_UNICODE` pointer is set to *NULL*.
 
-``U`` (:class:`str`) [PyUnicodeObject \*]
+``U`` (:class:`str`) [PyObject \*]
    Requires that the Python object is a Unicode object, without attempting
    any conversion.  Raises :exc:`TypeError` if the object is not a Unicode
    object.  The C variable may also be declared as :c:type:`PyObject\*`.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 03:16:37 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 03:16:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_MultibyteCodec=5FDecode=28?=
 =?utf8?q?=29_catchs_PyUnicode=5FAS=5FUNICODE=28=29_failures?=
Message-ID: <E1RVwCT-0002Rp-AQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/7216cf767993
changeset:   73802:7216cf767993
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 03:18:30 2011 +0100
summary:
  MultibyteCodec_Decode() catchs PyUnicode_AS_UNICODE() failures

files:
  Modules/cjkcodecs/multibytecodec.c |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c
--- a/Modules/cjkcodecs/multibytecodec.c
+++ b/Modules/cjkcodecs/multibytecodec.c
@@ -643,6 +643,8 @@
     if (buf.outobj == NULL)
         goto errorexit;
     buf.outbuf = PyUnicode_AS_UNICODE(buf.outobj);
+    if (buf.outbuf == NULL)
+        goto errorexit;
     buf.outbuf_end = buf.outbuf + PyUnicode_GET_SIZE(buf.outobj);
 
     if (self->codec->decinit != NULL &&

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 03:16:38 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 03:16:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Replace_PyUnicode=5FFromUni?=
 =?utf8?b?Y29kZShOVUxMLCAwKSBieSBQeVVuaWNvZGVfTmV3KDAsIDAp?=
Message-ID: <E1RVwCU-0002Rz-2W@dinsdale.python.org>

http://hg.python.org/cpython/rev/d855329d0f72
changeset:   73803:d855329d0f72
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 03:18:59 2011 +0100
summary:
  Replace PyUnicode_FromUnicode(NULL, 0) by PyUnicode_New(0, 0)

Create an empty string with the new Unicode API.

files:
  Modules/cjkcodecs/multibytecodec.c |  4 ++--
  Objects/stringlib/unicode_format.h |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c
--- a/Modules/cjkcodecs/multibytecodec.c
+++ b/Modules/cjkcodecs/multibytecodec.c
@@ -633,7 +633,7 @@
     if (datalen == 0) {
         PyBuffer_Release(&pdata);
         ERROR_DECREF(errorcb);
-        return make_tuple(PyUnicode_FromUnicode(NULL, 0), 0);
+        return make_tuple(PyUnicode_New(0, 0), 0);
     }
 
     buf.excobj = NULL;
@@ -1265,7 +1265,7 @@
     Py_ssize_t rsize, finalsize = 0;
 
     if (sizehint == 0)
-        return PyUnicode_FromUnicode(NULL, 0);
+        return PyUnicode_New(0, 0);
 
     buf.outobj = buf.excobj = NULL;
     cres = NULL;
diff --git a/Objects/stringlib/unicode_format.h b/Objects/stringlib/unicode_format.h
--- a/Objects/stringlib/unicode_format.h
+++ b/Objects/stringlib/unicode_format.h
@@ -79,7 +79,7 @@
 SubString_new_object_or_empty(SubString *str)
 {
     if (str->str == NULL) {
-        return PyUnicode_FromUnicode(NULL, 0);
+        return PyUnicode_New(0, 0);
     }
     return SubString_new_object(str);
 }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  1 03:20:10 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 01 Dec 2011 03:20:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_PyObject=5FRepr=28=29?=
 =?utf8?q?=3A_don=27t_call_PyUnicode=5FREADY=28=29_if_res_is_NULL?=
Message-ID: <E1RVwFu-0002gv-KL@dinsdale.python.org>

http://hg.python.org/cpython/rev/0d1536ec44e8
changeset:   73804:0d1536ec44e8
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 01 03:22:44 2011 +0100
summary:
  Fix PyObject_Repr(): don't call PyUnicode_READY() if res is NULL

files:
  Objects/object.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -378,7 +378,9 @@
         return PyUnicode_FromFormat("<%s object at %p>",
                                     v->ob_type->tp_name, v);
     res = (*v->ob_type->tp_repr)(v);
-    if (res != NULL && !PyUnicode_Check(res)) {
+    if (res == NULL)
+        return NULL;
+    if (!PyUnicode_Check(res)) {
         PyErr_Format(PyExc_TypeError,
                      "__repr__ returned non-string (type %.200s)",
                      res->ob_type->tp_name);

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Dec  1 05:36:16 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 01 Dec 2011 05:36:16 +0100
Subject: [Python-checkins] Daily reference leaks (0d1536ec44e8): sum=0
Message-ID: <E1RVyNc-0001Ks-EU@ap.vmr.nerim.net>

results for 0d1536ec44e8 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogonBh4G', '-x']

From python-checkins at python.org  Thu Dec  1 16:27:09 2011
From: python-checkins at python.org (mark.dickinson)
Date: Thu, 01 Dec 2011 16:27:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=239530=3A_Fix_undefi?=
 =?utf8?q?ned_behaviour_due_to_signed_overflow_in?=
Message-ID: <E1RW8XV-0004RD-LX@dinsdale.python.org>

http://hg.python.org/cpython/rev/7e37598a25a6
changeset:   73805:7e37598a25a6
user:        Mark Dickinson <mdickinson at enthought.com>
date:        Thu Dec 01 15:27:04 2011 +0000
summary:
  Issue #9530: Fix undefined behaviour due to signed overflow in Python/formatter_unicode.c.

files:
  Python/formatter_unicode.c |  16 +++++++---------
  1 files changed, 7 insertions(+), 9 deletions(-)


diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -51,7 +51,7 @@
 get_integer(PyObject *str, Py_ssize_t *pos, Py_ssize_t end,
                   Py_ssize_t *result)
 {
-    Py_ssize_t accumulator, digitval, oldaccumulator;
+    Py_ssize_t accumulator, digitval;
     int numdigits;
     accumulator = numdigits = 0;
     for (;;(*pos)++, numdigits++) {
@@ -61,19 +61,17 @@
         if (digitval < 0)
             break;
         /*
-           This trick was copied from old Unicode format code.  It's cute,
-           but would really suck on an old machine with a slow divide
-           implementation.  Fortunately, in the normal case we do not
-           expect too many digits.
+           Detect possible overflow before it happens:
+
+              accumulator * 10 + digitval > PY_SSIZE_T_MAX if and only if
+              accumulator > (PY_SSIZE_T_MAX - digitval) / 10.
         */
-        oldaccumulator = accumulator;
-        accumulator *= 10;
-        if ((accumulator+10)/10 != oldaccumulator+1) {
+        if (accumulator > (PY_SSIZE_T_MAX - digitval) / 10) {
             PyErr_Format(PyExc_ValueError,
                          "Too many decimal digits in format string");
             return -1;
         }
-        accumulator += digitval;
+        accumulator = accumulator * 10 + digitval;
     }
     *result = accumulator;
     return numdigits;

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Dec  2 05:37:16 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 02 Dec 2011 05:37:16 +0100
Subject: [Python-checkins] Daily reference leaks (7e37598a25a6): sum=0
Message-ID: <E1RWKs8-000115-1D@ap.vmr.nerim.net>

results for 7e37598a25a6 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloglWXr6j', '-x']

From python-checkins at python.org  Fri Dec  2 17:24:09 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 17:24:09 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzg0MTQ6IGFkZCBt?=
 =?utf8?q?ore_tests_for_=22assert=22=2E__Initial_patch_by_Gregory_Nofi=2E?=
Message-ID: <E1RWVuD-0007XZ-JS@dinsdale.python.org>

http://hg.python.org/cpython/rev/bcfb499338c1
changeset:   73806:bcfb499338c1
branch:      2.7
parent:      73783:3ecddf168f1f
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 18:17:30 2011 +0200
summary:
  #8414: add more tests for "assert".  Initial patch by Gregory Nofi.

files:
  Lib/test/test_grammar.py |  26 ++++++++++++++++++++++++--
  1 files changed, 24 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -551,13 +551,35 @@
         assert 1, 1
         assert lambda x:x
         assert 1, lambda x:x+1
+
+        try:
+            assert True
+        except AssertionError as e:
+            self.fail("'assert True' should not have raised an AssertionError")
+
+        try:
+            assert True, 'this should always pass'
+        except AssertionError as e:
+            self.fail("'assert True, msg' should not have "
+                      "raised an AssertionError")
+
+    # these tests fail if python is run with -O, so check __debug__
+    @unittest.skipUnless(__debug__, "Won't work if __debug__ is False")
+    def testAssert2(self):
         try:
             assert 0, "msg"
         except AssertionError, e:
             self.assertEqual(e.args[0], "msg")
         else:
-            if __debug__:
-                self.fail("AssertionError not raised by assert 0")
+            self.fail("AssertionError not raised by assert 0")
+
+        try:
+            assert False
+        except AssertionError as e:
+            self.assertEqual(len(e.args), 0)
+        else:
+            self.fail("AssertionError not raised by 'assert False'")
+
 
     ### compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
     # Tested below

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 17:24:11 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 17:24:11 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzg0MTQ6IGFkZCBt?=
 =?utf8?q?ore_tests_for_=22assert=22=2E__Initial_patch_by_Gregory_Nofi=2E?=
Message-ID: <E1RWVuF-0007Xg-Qp@dinsdale.python.org>

http://hg.python.org/cpython/rev/1efefeda00a7
changeset:   73807:1efefeda00a7
branch:      3.2
parent:      73796:2c05b8a6cdd1
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 18:22:52 2011 +0200
summary:
  #8414: add more tests for "assert".  Initial patch by Gregory Nofi.

files:
  Lib/test/test_grammar.py |  26 ++++++++++++++++++++++++--
  1 files changed, 24 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -493,13 +493,35 @@
         assert 1, 1
         assert lambda x:x
         assert 1, lambda x:x+1
+
+        try:
+            assert True
+        except AssertionError as e:
+            self.fail("'assert True' should not have raised an AssertionError")
+
+        try:
+            assert True, 'this should always pass'
+        except AssertionError as e:
+            self.fail("'assert True, msg' should not have "
+                      "raised an AssertionError")
+
+    # these tests fail if python is run with -O, so check __debug__
+    @unittest.skipUnless(__debug__, "Won't work if __debug__ is False")
+    def testAssert2(self):
         try:
             assert 0, "msg"
         except AssertionError as e:
             self.assertEqual(e.args[0], "msg")
         else:
-            if __debug__:
-                self.fail("AssertionError not raised by assert 0")
+            self.fail("AssertionError not raised by assert 0")
+
+        try:
+            assert False
+        except AssertionError as e:
+            self.assertEqual(len(e.args), 0)
+        else:
+            self.fail("AssertionError not raised by 'assert False'")
+
 
     ### compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
     # Tested below

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 17:24:12 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 17:24:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?b?OiAjODQxNDogbWVyZ2Ugd2l0aCAzLjIu?=
Message-ID: <E1RWVuG-0007Xk-W4@dinsdale.python.org>

http://hg.python.org/cpython/rev/47afbb2033aa
changeset:   73808:47afbb2033aa
parent:      73805:7e37598a25a6
parent:      73807:1efefeda00a7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 18:23:54 2011 +0200
summary:
  #8414: merge with 3.2.

files:
  Lib/test/test_grammar.py |  26 ++++++++++++++++++++++++--
  1 files changed, 24 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -500,13 +500,35 @@
         assert 1, 1
         assert lambda x:x
         assert 1, lambda x:x+1
+
+        try:
+            assert True
+        except AssertionError as e:
+            self.fail("'assert True' should not have raised an AssertionError")
+
+        try:
+            assert True, 'this should always pass'
+        except AssertionError as e:
+            self.fail("'assert True, msg' should not have "
+                      "raised an AssertionError")
+
+    # these tests fail if python is run with -O, so check __debug__
+    @unittest.skipUnless(__debug__, "Won't work if __debug__ is False")
+    def testAssert2(self):
         try:
             assert 0, "msg"
         except AssertionError as e:
             self.assertEqual(e.args[0], "msg")
         else:
-            if __debug__:
-                self.fail("AssertionError not raised by assert 0")
+            self.fail("AssertionError not raised by assert 0")
+
+        try:
+            assert False
+        except AssertionError as e:
+            self.assertEqual(len(e.args), 0)
+        else:
+            self.fail("AssertionError not raised by 'assert False'")
+
 
     ### compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
     # Tested below

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:29:17 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:29:17 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNDk5OiBmaXgg?=
 =?utf8?q?example_adding_=3E=3E=3E_before_the_comments=2E?=
Message-ID: <E1RWWvF-0005J2-RA@dinsdale.python.org>

http://hg.python.org/cpython/rev/d9e918c8d9d6
changeset:   73809:d9e918c8d9d6
branch:      2.7
parent:      73806:bcfb499338c1
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:26:48 2011 +0200
summary:
  #13499: fix example adding >>> before the comments.

files:
  Doc/library/uuid.rst |  16 ++++++++--------
  1 files changed, 8 insertions(+), 8 deletions(-)


diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst
--- a/Doc/library/uuid.rst
+++ b/Doc/library/uuid.rst
@@ -225,34 +225,34 @@
 
    >>> import uuid
 
-   # make a UUID based on the host ID and current time
+   >>> # make a UUID based on the host ID and current time
    >>> uuid.uuid1()
    UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')
 
-   # make a UUID using an MD5 hash of a namespace UUID and a name
+   >>> # make a UUID using an MD5 hash of a namespace UUID and a name
    >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
    UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')
 
-   # make a random UUID
+   >>> # make a random UUID
    >>> uuid.uuid4()
    UUID('16fd2706-8baf-433b-82eb-8c7fada847da')
 
-   # make a UUID using a SHA-1 hash of a namespace UUID and a name
+   >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
    >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
    UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')
 
-   # make a UUID from a string of hex digits (braces and hyphens ignored)
+   >>> # make a UUID from a string of hex digits (braces and hyphens ignored)
    >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')
 
-   # convert a UUID to a string of hex digits in standard form
+   >>> # convert a UUID to a string of hex digits in standard form
    >>> str(x)
    '00010203-0405-0607-0809-0a0b0c0d0e0f'
 
-   # get the raw 16 bytes of the UUID
+   >>> # get the raw 16 bytes of the UUID
    >>> x.bytes
    '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'
 
-   # make a UUID from a 16-byte string
+   >>> # make a UUID from a 16-byte string
    >>> uuid.UUID(bytes=x.bytes)
    UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:29:19 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:29:19 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNDk5OiBmaXgg?=
 =?utf8?q?example_adding_=3E=3E=3E_before_the_comments=2E?=
Message-ID: <E1RWWvH-0005JY-1C@dinsdale.python.org>

http://hg.python.org/cpython/rev/9e7728dc35e7
changeset:   73810:9e7728dc35e7
branch:      3.2
parent:      73807:1efefeda00a7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:28:36 2011 +0200
summary:
  #13499: fix example adding >>> before the comments.

files:
  Doc/library/uuid.rst |  16 ++++++++--------
  1 files changed, 8 insertions(+), 8 deletions(-)


diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst
--- a/Doc/library/uuid.rst
+++ b/Doc/library/uuid.rst
@@ -222,34 +222,34 @@
 
    >>> import uuid
 
-   # make a UUID based on the host ID and current time
+   >>> # make a UUID based on the host ID and current time
    >>> uuid.uuid1()
    UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')
 
-   # make a UUID using an MD5 hash of a namespace UUID and a name
+   >>> # make a UUID using an MD5 hash of a namespace UUID and a name
    >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
    UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')
 
-   # make a random UUID
+   >>> # make a random UUID
    >>> uuid.uuid4()
    UUID('16fd2706-8baf-433b-82eb-8c7fada847da')
 
-   # make a UUID using a SHA-1 hash of a namespace UUID and a name
+   >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
    >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
    UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')
 
-   # make a UUID from a string of hex digits (braces and hyphens ignored)
+   >>> # make a UUID from a string of hex digits (braces and hyphens ignored)
    >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')
 
-   # convert a UUID to a string of hex digits in standard form
+   >>> # convert a UUID to a string of hex digits in standard form
    >>> str(x)
    '00010203-0405-0607-0809-0a0b0c0d0e0f'
 
-   # get the raw 16 bytes of the UUID
+   >>> # get the raw 16 bytes of the UUID
    >>> x.bytes
    b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'
 
-   # make a UUID from a 16-byte string
+   >>> # make a UUID from a 16-byte string
    >>> uuid.UUID(bytes=x.bytes)
    UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:29:22 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:29:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313499=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RWWvK-0005K4-1S@dinsdale.python.org>

http://hg.python.org/cpython/rev/060c7093a81f
changeset:   73811:060c7093a81f
parent:      73808:47afbb2033aa
parent:      73810:9e7728dc35e7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:29:10 2011 +0200
summary:
  #13499: merge with 3.2.

files:
  Doc/library/uuid.rst |  16 ++++++++--------
  1 files changed, 8 insertions(+), 8 deletions(-)


diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst
--- a/Doc/library/uuid.rst
+++ b/Doc/library/uuid.rst
@@ -222,34 +222,34 @@
 
    >>> import uuid
 
-   # make a UUID based on the host ID and current time
+   >>> # make a UUID based on the host ID and current time
    >>> uuid.uuid1()
    UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')
 
-   # make a UUID using an MD5 hash of a namespace UUID and a name
+   >>> # make a UUID using an MD5 hash of a namespace UUID and a name
    >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
    UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')
 
-   # make a random UUID
+   >>> # make a random UUID
    >>> uuid.uuid4()
    UUID('16fd2706-8baf-433b-82eb-8c7fada847da')
 
-   # make a UUID using a SHA-1 hash of a namespace UUID and a name
+   >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
    >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
    UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')
 
-   # make a UUID from a string of hex digits (braces and hyphens ignored)
+   >>> # make a UUID from a string of hex digits (braces and hyphens ignored)
    >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')
 
-   # convert a UUID to a string of hex digits in standard form
+   >>> # convert a UUID to a string of hex digits in standard form
    >>> str(x)
    '00010203-0405-0607-0809-0a0b0c0d0e0f'
 
-   # get the raw 16 bytes of the UUID
+   >>> # get the raw 16 bytes of the UUID
    >>> x.bytes
    b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'
 
-   # make a UUID from a 16-byte string
+   >>> # make a UUID from a 16-byte string
    >>> uuid.UUID(bytes=x.bytes)
    UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:49:12 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:49:12 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNDk0OiBzL2Nh?=
 =?utf8?q?st/convert/=2E__Also_add_a_link=2E?=
Message-ID: <E1RWXEW-00077P-QP@dinsdale.python.org>

http://hg.python.org/cpython/rev/2f9c986b46cd
changeset:   73812:2f9c986b46cd
branch:      2.7
parent:      73809:d9e918c8d9d6
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:47:24 2011 +0200
summary:
  #13494: s/cast/convert/.  Also add a link.

files:
  Doc/library/stdtypes.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -2955,9 +2955,9 @@
 used to represent truth values (although other values can also be considered
 false or true).  In numeric contexts (for example when used as the argument to
 an arithmetic operator), they behave like the integers 0 and 1, respectively.
-The built-in function :func:`bool` can be used to cast any value to a Boolean,
-if the value can be interpreted as a truth value (see section Truth Value
-Testing above).
+The built-in function :func:`bool` can be used to convert any value to a
+Boolean, if the value can be interpreted as a truth value (see section
+:ref:`truth` above).
 
 .. index::
    single: False

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:49:13 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:49:13 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNDk0OiBzL2Nh?=
 =?utf8?q?st/convert/=2E__Also_add_a_link=2E?=
Message-ID: <E1RWXEX-00077T-MX@dinsdale.python.org>

http://hg.python.org/cpython/rev/69369fd3514b
changeset:   73813:69369fd3514b
branch:      3.2
parent:      73810:9e7728dc35e7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:47:24 2011 +0200
summary:
  #13494: s/cast/convert/.  Also add a link.

files:
  Doc/library/stdtypes.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -2736,9 +2736,9 @@
 used to represent truth values (although other values can also be considered
 false or true).  In numeric contexts (for example when used as the argument to
 an arithmetic operator), they behave like the integers 0 and 1, respectively.
-The built-in function :func:`bool` can be used to cast any value to a Boolean,
-if the value can be interpreted as a truth value (see section Truth Value
-Testing above).
+The built-in function :func:`bool` can be used to convert any value to a
+Boolean, if the value can be interpreted as a truth value (see section
+:ref:`truth` above).
 
 .. index::
    single: False

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 18:49:14 2011
From: python-checkins at python.org (ezio.melotti)
Date: Fri, 02 Dec 2011 18:49:14 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313494=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RWXEY-00077X-IS@dinsdale.python.org>

http://hg.python.org/cpython/rev/454b97887c5a
changeset:   73814:454b97887c5a
parent:      73811:060c7093a81f
parent:      73813:69369fd3514b
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Fri Dec 02 19:49:02 2011 +0200
summary:
  #13494: merge with 3.2.

files:
  Doc/library/stdtypes.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -2772,9 +2772,9 @@
 used to represent truth values (although other values can also be considered
 false or true).  In numeric contexts (for example when used as the argument to
 an arithmetic operator), they behave like the integers 0 and 1, respectively.
-The built-in function :func:`bool` can be used to cast any value to a Boolean,
-if the value can be interpreted as a truth value (see section Truth Value
-Testing above).
+The built-in function :func:`bool` can be used to convert any value to a
+Boolean, if the value can be interpreted as a truth value (see section
+:ref:`truth` above).
 
 .. index::
    single: False

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 20:25:16 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 02 Dec 2011 20:25:16 +0100
Subject: [Python-checkins] =?utf8?q?peps=3A_Update_PEP_3154_after_PEP_3155?=
	=?utf8?q?_has_been_accepted=2E?=
Message-ID: <E1RWYjU-0007QM-Jn@dinsdale.python.org>

http://hg.python.org/peps/rev/d54015aaa32b
changeset:   4006:d54015aaa32b
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 02 20:19:29 2011 +0100
summary:
  Update PEP 3154 after PEP 3155 has been accepted.

files:
  pep-3154.txt |  37 ++++++++++++++++---------------------
  1 files changed, 16 insertions(+), 21 deletions(-)


diff --git a/pep-3154.txt b/pep-3154.txt
--- a/pep-3154.txt
+++ b/pep-3154.txt
@@ -71,27 +71,20 @@
 special method (``__getnewargs_ex__`` ?) and a new opcode (NEWOBJEX ?)
 are needed.
 
-Serializing more callable objects
----------------------------------
+Serializing more "lookupable" objects
+-------------------------------------
 
-Currently, only module-global functions are serializable.
-Multiprocessing has custom support for pickling other callables such
-as bound methods [4]_.  This support could be folded in the protocol,
-and made more efficient through a new GETATTR opcode.
+For some kinds of objects, it only makes sense to serialize them by name
+(for example classes and functions).  By default, pickle is only able to
+serialize module-global functions and classes by name.  Supporting other
+kinds of objects, such as unbound methods [4]_, is a common request.
+Actually, third-party support for some of them, such as bound methods,
+is implemented in the multiprocessing module [5]_.
 
-Serializing "pseudo-global" objects
------------------------------------
-
-Objects which are not module-global, but should be treated in a
-similar fashion -- such as unbound methods [5]_ or nested classes --
-cannot currently be pickled (or, rather, unpickled) because the pickle
-protocol does not correctly specify how to retrieve them.  One
-solution would be through the adjunction of a ``__namespace__`` (or
-``__qualname__``) to all class and function objects, specifying the
-full "path" by which they can be retrieved.  For globals, this would
-generally be ``"{}.{}".format(obj.__module__, obj.__name__)``.  Then a
-new opcode can resolve that path and push the object on the stack,
-similarly to the GLOBAL opcode.
+:pep:`3155` now makes it possible to lookup many more objects by name.
+Generalizing the GLOBAL opcode to accept dot-separated names, or adding
+a special GETATTR opcode, would allow the standard pickle implementation
+to support, in an efficient way, all those kinds of objects.
 
 Binary encoding for all opcodes
 -------------------------------
@@ -131,12 +124,12 @@
 .. [3] "pickle/copyreg doesn't support keyword only arguments in __new__":
    http://bugs.python.org/issue4727
 
-.. [4] Lib/multiprocessing/forking.py:
+.. [4] "pickle should support methods":
+   http://bugs.python.org/issue9276
+
+.. [5] Lib/multiprocessing/forking.py:
    http://hg.python.org/cpython/file/baea9f5f973c/Lib/multiprocessing/forking.py#l54
 
-.. [5] "pickle should support methods":
-   http://bugs.python.org/issue9276
-
 
 Copyright
 =========

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Fri Dec  2 20:25:17 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 02 Dec 2011 20:25:17 +0100
Subject: [Python-checkins] =?utf8?q?peps=3A_Mark_PEP_3155_final=2E?=
Message-ID: <E1RWYjV-0007QZ-Ad@dinsdale.python.org>

http://hg.python.org/peps/rev/f41beb5dcdaa
changeset:   4007:f41beb5dcdaa
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 02 20:20:06 2011 +0100
summary:
  Mark PEP 3155 final.

files:
  pep-3155.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-3155.txt b/pep-3155.txt
--- a/pep-3155.txt
+++ b/pep-3155.txt
@@ -3,7 +3,7 @@
 Version: $Revision$
 Last-Modified: $Date$
 Author: Antoine Pitrou <solipsis at pitrou.net>
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 2011-10-29

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Fri Dec  2 20:28:35 2011
From: python-checkins at python.org (petri.lehtinen)
Date: Fri, 02 Dec 2011 20:28:35 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNDM5?=
 =?utf8?q?=3A_Fix_many_errors_in_turtle_docstrings=2E?=
Message-ID: <E1RWYmh-0007lw-SG@dinsdale.python.org>

http://hg.python.org/cpython/rev/6e03ab9950f6
changeset:   73815:6e03ab9950f6
branch:      2.7
parent:      73812:2f9c986b46cd
user:        Petri Lehtinen <petri at digip.org>
date:        Fri Dec 02 21:09:30 2011 +0200
summary:
  Issue #13439: Fix many errors in turtle docstrings.

files:
  Lib/lib-tk/turtle.py |  104 ++++++++++++++----------------
  Misc/NEWS            |    2 +
  2 files changed, 52 insertions(+), 54 deletions(-)


diff --git a/Lib/lib-tk/turtle.py b/Lib/lib-tk/turtle.py
--- a/Lib/lib-tk/turtle.py
+++ b/Lib/lib-tk/turtle.py
@@ -859,7 +859,7 @@
         >>> poly = ((0,0),(10,-5),(0,10),(-10,-5))
         >>> s = Shape("compound")
         >>> s.addcomponent(poly, "red", "blue")
-        ### .. add more components and then use register_shape()
+        >>> # .. add more components and then use register_shape()
         """
         if self._type != "compound":
             raise TurtleGraphicsError("Cannot add component to %s Shape"
@@ -958,7 +958,7 @@
         No argument.
 
         Example (for a TurtleScreen instance named screen):
-        screen.clear()
+        >>> screen.clear()
 
         Note: this method is not available as function.
         """
@@ -1030,8 +1030,8 @@
         Example (for a TurtleScreen instance named screen):
         >>> screen.setworldcoordinates(-10,-0.5,50,1.5)
         >>> for _ in range(36):
-                left(10)
-                forward(0.5)
+        ...     left(10)
+        ...     forward(0.5)
         """
         if self.mode() != "world":
             self.mode("world")
@@ -1136,7 +1136,7 @@
         >>> screen.colormode()
         1.0
         >>> screen.colormode(255)
-        >>> turtle.pencolor(240,160,80)
+        >>> pencolor(240,160,80)
         """
         if cmode is None:
             return self._colormode
@@ -1204,9 +1204,9 @@
         >>> screen.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                fd(dist)
-                rt(90)
-                dist += 2
+        ...     fd(dist)
+        ...     rt(90)
+        ...     dist += 2
         """
         if n is None:
             return self._tracing
@@ -1233,7 +1233,7 @@
         self._delayvalue = int(delay)
 
     def _incrementudc(self):
-        "Increment upadate counter."""
+        """Increment upadate counter."""
         if not TurtleScreen._RUNNING:
             TurtleScreen._RUNNNING = True
             raise Terminator
@@ -1304,13 +1304,10 @@
         Example (for a TurtleScreen instance named screen
         and a Turtle instance named turtle):
 
-        >>> screen.onclick(turtle.goto)
-
-        ### Subsequently clicking into the TurtleScreen will
-        ### make the turtle move to the clicked point.
+        >>> screen.onclick(goto)
+        >>> # Subsequently clicking into the TurtleScreen will
+        >>> # make the turtle move to the clicked point.
         >>> screen.onclick(None)
-
-        ### event-binding will be removed
         """
         self._onscreenclick(fun, btn, add)
 
@@ -1324,20 +1321,18 @@
         In order to be able to register key-events, TurtleScreen
         must have focus. (See method listen.)
 
-        Example (for a TurtleScreen instance named screen
-        and a Turtle instance named turtle):
+        Example (for a TurtleScreen instance named screen):
 
         >>> def f():
-                fd(50)
-                lt(60)
-
-
+        ...     fd(50)
+        ...     lt(60)
+        ...
         >>> screen.onkey(f, "Up")
         >>> screen.listen()
 
-        ### Subsequently the turtle can be moved by
-        ### repeatedly pressing the up-arrow key,
-        ### consequently drawing a hexagon
+        Subsequently the turtle can be moved by repeatedly pressing
+        the up-arrow key, consequently drawing a hexagon
+
         """
         if fun is None:
             if key in self._keys:
@@ -1369,12 +1364,12 @@
 
         >>> running = True
         >>> def f():
-                if running:
-                        fd(50)
-                        lt(60)
-                        screen.ontimer(f, 250)
-
-        >>> f()   ### makes the turtle marching around
+        ...     if running:
+        ...             fd(50)
+        ...             lt(60)
+        ...             screen.ontimer(f, 250)
+        ...
+        >>> f()   # makes the turtle marching around
         >>> running = False
         """
         self._ontimer(fun, t)
@@ -1418,7 +1413,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> turtle.screensize(2000,1500)
-            ### e. g. to search for an erroneously escaped turtle ;-)
+        >>> # e. g. to search for an erroneously escaped turtle ;-)
         """
         return self._resize(canvwidth, canvheight, bg)
 
@@ -2004,7 +1999,7 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.pensize()
         1
-        turtle.pensize(10)   # from here on lines of width 10 are drawn
+        >>> turtle.pensize(10)   # from here on lines of width 10 are drawn
         """
         if width is None:
             return self._pensize
@@ -2516,7 +2511,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> while undobufferentries():
-                undo()
+        ...     undo()
         """
         if self.undobuffer is None:
             return 0
@@ -2592,9 +2587,9 @@
         >>> turtle.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                turtle.fd(dist)
-                turtle.rt(90)
-                dist += 2
+        ...     turtle.fd(dist)
+        ...     turtle.rt(90)
+        ...     dist += 2
         """
         return self.screen.tracer(flag, delay)
 
@@ -2763,7 +2758,6 @@
         >>> turtle.shapesize(5,2)
         >>> turtle.tilt(45)
         >>> turtle.tiltangle()
-        >>>
         """
         tilt = -self._tilt * (180.0/math.pi) * self._angleOrient
         return (tilt / self._degreesPerAU) % self._fullcircle
@@ -2963,7 +2957,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(8):
-                turtle.stamp(); turtle.fd(30)
+        ...     turtle.stamp(); turtle.fd(30)
         ...
         >>> turtle.clearstamps(2)
         >>> turtle.clearstamps(-2)
@@ -3430,9 +3424,9 @@
         Example for the anonymous turtle, i. e. the procedural way:
 
         >>> def turn(x, y):
-                left(360)
-
-        >>> onclick(turn) # Now clicking into the turtle will turn it.
+        ...     left(360)
+        ...
+        >>> onclick(turn)  # Now clicking into the turtle will turn it.
         >>> onclick(None)  # event-binding will be removed
         """
         self.screen._onclick(self.turtle._item, fun, btn, add)
@@ -3448,16 +3442,17 @@
 
         Example (for a MyTurtle instance named joe):
         >>> class MyTurtle(Turtle):
-                def glow(self,x,y):
-                        self.fillcolor("red")
-                def unglow(self,x,y):
-                        self.fillcolor("")
-
+        ...     def glow(self,x,y):
+        ...             self.fillcolor("red")
+        ...     def unglow(self,x,y):
+        ...             self.fillcolor("")
+        ...
         >>> joe = MyTurtle()
         >>> joe.onclick(joe.glow)
         >>> joe.onrelease(joe.unglow)
-        ### clicking on joe turns fillcolor red,
-        ### unclicking turns it to transparent.
+
+        Clicking on joe turns fillcolor red, unclicking turns it to
+        transparent.
         """
         self.screen._onrelease(self.turtle._item, fun, btn, add)
         self._update()
@@ -3476,9 +3471,9 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.ondrag(turtle.goto)
 
-        ### Subsequently clicking and dragging a Turtle will
-        ### move it across the screen thereby producing handdrawings
-        ### (if pen is down).
+        Subsequently clicking and dragging a Turtle will move it
+        across the screen thereby producing handdrawings (if pen is
+        down).
         """
         self.screen._ondrag(self.turtle._item, fun, btn, add)
 
@@ -3525,10 +3520,11 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(4):
-                turtle.fd(50); turtle.lt(80)
-
+        ...     turtle.fd(50); turtle.lt(80)
+        ...
         >>> for i in range(8):
-                turtle.undo()
+        ...     turtle.undo()
+        ...
         """
         if self.undobuffer is None:
             return
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -79,6 +79,8 @@
 Library
 -------
 
+- Issue #13439: Fix many errors in turtle docstrings.
+
 - Issue #12856: Ensure child processes do not inherit the parent's random
   seed for filename generation in the tempfile module.  Patch by Brian
   Harring.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 20:28:36 2011
From: python-checkins at python.org (petri.lehtinen)
Date: Fri, 02 Dec 2011 20:28:36 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNDM5?=
 =?utf8?q?=3A_Fix_many_errors_in_turtle_docstrings=2E?=
Message-ID: <E1RWYmi-0007m0-KQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/cc559e1e3bd8
changeset:   73816:cc559e1e3bd8
branch:      3.2
parent:      73813:69369fd3514b
user:        Petri Lehtinen <petri at digip.org>
date:        Fri Dec 02 21:24:14 2011 +0200
summary:
  Issue #13439: Fix many errors in turtle docstrings.

files:
  Lib/turtle.py |  134 ++++++++++++++++++-------------------
  Misc/NEWS     |    2 +
  2 files changed, 66 insertions(+), 70 deletions(-)


diff --git a/Lib/turtle.py b/Lib/turtle.py
--- a/Lib/turtle.py
+++ b/Lib/turtle.py
@@ -905,7 +905,7 @@
         >>> poly = ((0,0),(10,-5),(0,10),(-10,-5))
         >>> s = Shape("compound")
         >>> s.addcomponent(poly, "red", "blue")
-        ### .. add more components and then use register_shape()
+        >>> # .. add more components and then use register_shape()
         """
         if self._type != "compound":
             raise TurtleGraphicsError("Cannot add component to %s Shape"
@@ -1003,7 +1003,7 @@
         no backgroundimage, no eventbindings and tracing on.
 
         Example (for a TurtleScreen instance named screen):
-        screen.clear()
+        >>> screen.clear()
 
         Note: this method is not available as function.
         """
@@ -1077,8 +1077,8 @@
         Example (for a TurtleScreen instance named screen):
         >>> screen.setworldcoordinates(-10,-0.5,50,1.5)
         >>> for _ in range(36):
-                left(10)
-                forward(0.5)
+        ...     left(10)
+        ...     forward(0.5)
         """
         if self.mode() != "world":
             self.mode("world")
@@ -1182,7 +1182,7 @@
         >>> screen.colormode()
         1.0
         >>> screen.colormode(255)
-        >>> turtle.pencolor(240,160,80)
+        >>> pencolor(240,160,80)
         """
         if cmode is None:
             return self._colormode
@@ -1250,9 +1250,9 @@
         >>> screen.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                fd(dist)
-                rt(90)
-                dist += 2
+        ...     fd(dist)
+        ...     rt(90)
+        ...     dist += 2
         """
         if n is None:
             return self._tracing
@@ -1279,7 +1279,7 @@
         self._delayvalue = int(delay)
 
     def _incrementudc(self):
-        "Increment upadate counter."""
+        """Increment upadate counter."""
         if not TurtleScreen._RUNNING:
             TurtleScreen._RUNNNING = True
             raise Terminator
@@ -1347,16 +1347,12 @@
                clicked point on the canvas.
         num -- the number of the mouse-button, defaults to 1
 
-        Example (for a TurtleScreen instance named screen
-        and a Turtle instance named turtle):
-
-        >>> screen.onclick(turtle.goto)
-
-        ### Subsequently clicking into the TurtleScreen will
-        ### make the turtle move to the clicked point.
+        Example (for a TurtleScreen instance named screen)
+
+        >>> screen.onclick(goto)
+        >>> # Subsequently clicking into the TurtleScreen will
+        >>> # make the turtle move to the clicked point.
         >>> screen.onclick(None)
-
-        ### event-binding will be removed
         """
         self._onscreenclick(fun, btn, add)
 
@@ -1370,20 +1366,18 @@
         In order to be able to register key-events, TurtleScreen
         must have focus. (See method listen.)
 
-        Example (for a TurtleScreen instance named screen
-        and a Turtle instance named turtle):
+        Example (for a TurtleScreen instance named screen):
 
         >>> def f():
-                fd(50)
-                lt(60)
-
-
+        ...     fd(50)
+        ...     lt(60)
+        ...
         >>> screen.onkey(f, "Up")
         >>> screen.listen()
 
-        ### Subsequently the turtle can be moved by
-        ### repeatedly pressing the up-arrow key,
-        ### consequently drawing a hexagon
+        Subsequently the turtle can be moved by repeatedly pressing
+        the up-arrow key, consequently drawing a hexagon
+
         """
         if fun is None:
             if key in self._keys:
@@ -1407,16 +1401,15 @@
         and a Turtle instance named turtle):
 
         >>> def f():
-                fd(50)
-
-
-        >>> screen.onkey(f, "Up")
+        ...     fd(50)
+        ...     lt(60)
+        ...
+        >>> screen.onkeypress(f, "Up")
         >>> screen.listen()
 
-        ### Subsequently the turtle can be moved by
-        ### repeatedly pressing the up-arrow key,
-        ### or by keeping pressed the up-arrow key.
-        ### consequently drawing a hexagon.
+        Subsequently the turtle can be moved by repeatedly pressing
+        the up-arrow key, or by keeping pressed the up-arrow key.
+        consequently drawing a hexagon.
         """
         if fun is None:
             if key in self._keys:
@@ -1448,12 +1441,12 @@
 
         >>> running = True
         >>> def f():
-                if running:
-                        fd(50)
-                        lt(60)
-                        screen.ontimer(f, 250)
-
-        >>> f()   ### makes the turtle marching around
+        ...     if running:
+        ...             fd(50)
+        ...             lt(60)
+        ...             screen.ontimer(f, 250)
+        ...
+        >>> f()   # makes the turtle marching around
         >>> running = False
         """
         self._ontimer(fun, t)
@@ -1497,7 +1490,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> turtle.screensize(2000,1500)
-            ### e. g. to search for an erroneously escaped turtle ;-)
+        >>> # e.g. to search for an erroneously escaped turtle ;-)
         """
         return self._resize(canvwidth, canvheight, bg)
 
@@ -2085,7 +2078,7 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.pensize()
         1
-        turtle.pensize(10)   # from here on lines of width 10 are drawn
+        >>> turtle.pensize(10)   # from here on lines of width 10 are drawn
         """
         if width is None:
             return self._pensize
@@ -2560,7 +2553,7 @@
         """Delete the turtle's drawings and restore its default values.
 
         No argument.
-,
+
         Delete the turtle's drawings from the screen, re-center the turtle
         and set variables to the default values.
 
@@ -2607,7 +2600,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> while undobufferentries():
-                undo()
+        ...     undo()
         """
         if self.undobuffer is None:
             return 0
@@ -2683,9 +2676,9 @@
         >>> turtle.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                turtle.fd(dist)
-                turtle.rt(90)
-                dist += 2
+        ...     turtle.fd(dist)
+        ...     turtle.rt(90)
+        ...     dist += 2
         """
         return self.screen.tracer(flag, delay)
 
@@ -2883,7 +2876,6 @@
         >>> turtle.shapesize(5,2)
         >>> turtle.tilt(45)
         >>> turtle.tiltangle()
-        >>>
         """
         if angle is None:
             tilt = -self._tilt * (180.0/math.pi) * self._angleOrient
@@ -2928,7 +2920,7 @@
         >>> turtle.shapesize(4,2)
         >>> turtle.shearfactor(-0.5)
         >>> turtle.shapetransform()
-        >>> (4.0, -1.0, -0.0, 2.0)
+        (4.0, -1.0, -0.0, 2.0)
         """
         if t11 is t12 is t21 is t22 is None:
             return self._shapetrafo
@@ -3126,7 +3118,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(8):
-                turtle.stamp(); turtle.fd(30)
+        ...     turtle.stamp(); turtle.fd(30)
         ...
         >>> turtle.clearstamps(2)
         >>> turtle.clearstamps(-2)
@@ -3302,9 +3294,9 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.begin_fill()
         >>> if turtle.filling():
-                turtle.pensize(5)
-        else:
-                turtle.pensize(3)
+        ...     turtle.pensize(5)
+        ... else:
+        ...     turtle.pensize(3)
         """
         return isinstance(self._fillpath, list)
 
@@ -3534,9 +3526,9 @@
         Example for the anonymous turtle, i. e. the procedural way:
 
         >>> def turn(x, y):
-                left(360)
-
-        >>> onclick(turn) # Now clicking into the turtle will turn it.
+        ...     left(360)
+        ...
+        >>> onclick(turn)  # Now clicking into the turtle will turn it.
         >>> onclick(None)  # event-binding will be removed
         """
         self.screen._onclick(self.turtle._item, fun, btn, add)
@@ -3552,16 +3544,17 @@
 
         Example (for a MyTurtle instance named joe):
         >>> class MyTurtle(Turtle):
-                def glow(self,x,y):
-                        self.fillcolor("red")
-                def unglow(self,x,y):
-                        self.fillcolor("")
-
+        ...     def glow(self,x,y):
+        ...             self.fillcolor("red")
+        ...     def unglow(self,x,y):
+        ...             self.fillcolor("")
+        ...
         >>> joe = MyTurtle()
         >>> joe.onclick(joe.glow)
         >>> joe.onrelease(joe.unglow)
-        ### clicking on joe turns fillcolor red,
-        ### unclicking turns it to transparent.
+
+        Clicking on joe turns fillcolor red, unclicking turns it to
+        transparent.
         """
         self.screen._onrelease(self.turtle._item, fun, btn, add)
         self._update()
@@ -3580,9 +3573,9 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.ondrag(turtle.goto)
 
-        ### Subsequently clicking and dragging a Turtle will
-        ### move it across the screen thereby producing handdrawings
-        ### (if pen is down).
+        Subsequently clicking and dragging a Turtle will move it
+        across the screen thereby producing handdrawings (if pen is
+        down).
         """
         self.screen._ondrag(self.turtle._item, fun, btn, add)
 
@@ -3630,10 +3623,11 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(4):
-                turtle.fd(50); turtle.lt(80)
-
+        ...     turtle.fd(50); turtle.lt(80)
+        ...
         >>> for i in range(8):
-                turtle.undo()
+        ...     turtle.undo()
+        ...
         """
         if self.undobuffer is None:
             return
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -87,6 +87,8 @@
 Library
 -------
 
+- Issue #13439: Fix many errors in turtle docstrings.
+
 - Issue #13487: Make inspect.getmodule robust against changes done to
   sys.modules while it is iterating over it.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  2 20:28:37 2011
From: python-checkins at python.org (petri.lehtinen)
Date: Fri, 02 Dec 2011 20:28:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313439=3A_Merge_branch_3=2E2?=
Message-ID: <E1RWYmj-0007mY-C8@dinsdale.python.org>

http://hg.python.org/cpython/rev/8d60c1c89105
changeset:   73817:8d60c1c89105
parent:      73814:454b97887c5a
parent:      73816:cc559e1e3bd8
user:        Petri Lehtinen <petri at digip.org>
date:        Fri Dec 02 21:25:39 2011 +0200
summary:
  Issue #13439: Merge branch 3.2

files:
  Lib/turtle.py |  134 ++++++++++++++++++-------------------
  Misc/NEWS     |    2 +
  2 files changed, 66 insertions(+), 70 deletions(-)


diff --git a/Lib/turtle.py b/Lib/turtle.py
--- a/Lib/turtle.py
+++ b/Lib/turtle.py
@@ -904,7 +904,7 @@
         >>> poly = ((0,0),(10,-5),(0,10),(-10,-5))
         >>> s = Shape("compound")
         >>> s.addcomponent(poly, "red", "blue")
-        ### .. add more components and then use register_shape()
+        >>> # .. add more components and then use register_shape()
         """
         if self._type != "compound":
             raise TurtleGraphicsError("Cannot add component to %s Shape"
@@ -1002,7 +1002,7 @@
         no backgroundimage, no eventbindings and tracing on.
 
         Example (for a TurtleScreen instance named screen):
-        screen.clear()
+        >>> screen.clear()
 
         Note: this method is not available as function.
         """
@@ -1076,8 +1076,8 @@
         Example (for a TurtleScreen instance named screen):
         >>> screen.setworldcoordinates(-10,-0.5,50,1.5)
         >>> for _ in range(36):
-                left(10)
-                forward(0.5)
+        ...     left(10)
+        ...     forward(0.5)
         """
         if self.mode() != "world":
             self.mode("world")
@@ -1181,7 +1181,7 @@
         >>> screen.colormode()
         1.0
         >>> screen.colormode(255)
-        >>> turtle.pencolor(240,160,80)
+        >>> pencolor(240,160,80)
         """
         if cmode is None:
             return self._colormode
@@ -1249,9 +1249,9 @@
         >>> screen.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                fd(dist)
-                rt(90)
-                dist += 2
+        ...     fd(dist)
+        ...     rt(90)
+        ...     dist += 2
         """
         if n is None:
             return self._tracing
@@ -1278,7 +1278,7 @@
         self._delayvalue = int(delay)
 
     def _incrementudc(self):
-        "Increment upadate counter."""
+        """Increment upadate counter."""
         if not TurtleScreen._RUNNING:
             TurtleScreen._RUNNNING = True
             raise Terminator
@@ -1346,16 +1346,12 @@
                clicked point on the canvas.
         num -- the number of the mouse-button, defaults to 1
 
-        Example (for a TurtleScreen instance named screen
-        and a Turtle instance named turtle):
-
-        >>> screen.onclick(turtle.goto)
-
-        ### Subsequently clicking into the TurtleScreen will
-        ### make the turtle move to the clicked point.
+        Example (for a TurtleScreen instance named screen)
+
+        >>> screen.onclick(goto)
+        >>> # Subsequently clicking into the TurtleScreen will
+        >>> # make the turtle move to the clicked point.
         >>> screen.onclick(None)
-
-        ### event-binding will be removed
         """
         self._onscreenclick(fun, btn, add)
 
@@ -1369,20 +1365,18 @@
         In order to be able to register key-events, TurtleScreen
         must have focus. (See method listen.)
 
-        Example (for a TurtleScreen instance named screen
-        and a Turtle instance named turtle):
+        Example (for a TurtleScreen instance named screen):
 
         >>> def f():
-                fd(50)
-                lt(60)
-
-
+        ...     fd(50)
+        ...     lt(60)
+        ...
         >>> screen.onkey(f, "Up")
         >>> screen.listen()
 
-        ### Subsequently the turtle can be moved by
-        ### repeatedly pressing the up-arrow key,
-        ### consequently drawing a hexagon
+        Subsequently the turtle can be moved by repeatedly pressing
+        the up-arrow key, consequently drawing a hexagon
+
         """
         if fun is None:
             if key in self._keys:
@@ -1406,16 +1400,15 @@
         and a Turtle instance named turtle):
 
         >>> def f():
-                fd(50)
-
-
-        >>> screen.onkey(f, "Up")
+        ...     fd(50)
+        ...     lt(60)
+        ...
+        >>> screen.onkeypress(f, "Up")
         >>> screen.listen()
 
-        ### Subsequently the turtle can be moved by
-        ### repeatedly pressing the up-arrow key,
-        ### or by keeping pressed the up-arrow key.
-        ### consequently drawing a hexagon.
+        Subsequently the turtle can be moved by repeatedly pressing
+        the up-arrow key, or by keeping pressed the up-arrow key.
+        consequently drawing a hexagon.
         """
         if fun is None:
             if key in self._keys:
@@ -1447,12 +1440,12 @@
 
         >>> running = True
         >>> def f():
-                if running:
-                        fd(50)
-                        lt(60)
-                        screen.ontimer(f, 250)
-
-        >>> f()   ### makes the turtle marching around
+        ...     if running:
+        ...             fd(50)
+        ...             lt(60)
+        ...             screen.ontimer(f, 250)
+        ...
+        >>> f()   # makes the turtle marching around
         >>> running = False
         """
         self._ontimer(fun, t)
@@ -1496,7 +1489,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> turtle.screensize(2000,1500)
-            ### e. g. to search for an erroneously escaped turtle ;-)
+        >>> # e.g. to search for an erroneously escaped turtle ;-)
         """
         return self._resize(canvwidth, canvheight, bg)
 
@@ -2084,7 +2077,7 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.pensize()
         1
-        turtle.pensize(10)   # from here on lines of width 10 are drawn
+        >>> turtle.pensize(10)   # from here on lines of width 10 are drawn
         """
         if width is None:
             return self._pensize
@@ -2559,7 +2552,7 @@
         """Delete the turtle's drawings and restore its default values.
 
         No argument.
-,
+
         Delete the turtle's drawings from the screen, re-center the turtle
         and set variables to the default values.
 
@@ -2606,7 +2599,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> while undobufferentries():
-                undo()
+        ...     undo()
         """
         if self.undobuffer is None:
             return 0
@@ -2682,9 +2675,9 @@
         >>> turtle.tracer(8, 25)
         >>> dist = 2
         >>> for i in range(200):
-                turtle.fd(dist)
-                turtle.rt(90)
-                dist += 2
+        ...     turtle.fd(dist)
+        ...     turtle.rt(90)
+        ...     dist += 2
         """
         return self.screen.tracer(flag, delay)
 
@@ -2882,7 +2875,6 @@
         >>> turtle.shapesize(5,2)
         >>> turtle.tilt(45)
         >>> turtle.tiltangle()
-        >>>
         """
         if angle is None:
             tilt = -self._tilt * (180.0/math.pi) * self._angleOrient
@@ -2927,7 +2919,7 @@
         >>> turtle.shapesize(4,2)
         >>> turtle.shearfactor(-0.5)
         >>> turtle.shapetransform()
-        >>> (4.0, -1.0, -0.0, 2.0)
+        (4.0, -1.0, -0.0, 2.0)
         """
         if t11 is t12 is t21 is t22 is None:
             return self._shapetrafo
@@ -3125,7 +3117,7 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(8):
-                turtle.stamp(); turtle.fd(30)
+        ...     turtle.stamp(); turtle.fd(30)
         ...
         >>> turtle.clearstamps(2)
         >>> turtle.clearstamps(-2)
@@ -3301,9 +3293,9 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.begin_fill()
         >>> if turtle.filling():
-                turtle.pensize(5)
-        else:
-                turtle.pensize(3)
+        ...     turtle.pensize(5)
+        ... else:
+        ...     turtle.pensize(3)
         """
         return isinstance(self._fillpath, list)
 
@@ -3533,9 +3525,9 @@
         Example for the anonymous turtle, i. e. the procedural way:
 
         >>> def turn(x, y):
-                left(360)
-
-        >>> onclick(turn) # Now clicking into the turtle will turn it.
+        ...     left(360)
+        ...
+        >>> onclick(turn)  # Now clicking into the turtle will turn it.
         >>> onclick(None)  # event-binding will be removed
         """
         self.screen._onclick(self.turtle._item, fun, btn, add)
@@ -3551,16 +3543,17 @@
 
         Example (for a MyTurtle instance named joe):
         >>> class MyTurtle(Turtle):
-                def glow(self,x,y):
-                        self.fillcolor("red")
-                def unglow(self,x,y):
-                        self.fillcolor("")
-
+        ...     def glow(self,x,y):
+        ...             self.fillcolor("red")
+        ...     def unglow(self,x,y):
+        ...             self.fillcolor("")
+        ...
         >>> joe = MyTurtle()
         >>> joe.onclick(joe.glow)
         >>> joe.onrelease(joe.unglow)
-        ### clicking on joe turns fillcolor red,
-        ### unclicking turns it to transparent.
+
+        Clicking on joe turns fillcolor red, unclicking turns it to
+        transparent.
         """
         self.screen._onrelease(self.turtle._item, fun, btn, add)
         self._update()
@@ -3579,9 +3572,9 @@
         Example (for a Turtle instance named turtle):
         >>> turtle.ondrag(turtle.goto)
 
-        ### Subsequently clicking and dragging a Turtle will
-        ### move it across the screen thereby producing handdrawings
-        ### (if pen is down).
+        Subsequently clicking and dragging a Turtle will move it
+        across the screen thereby producing handdrawings (if pen is
+        down).
         """
         self.screen._ondrag(self.turtle._item, fun, btn, add)
 
@@ -3629,10 +3622,11 @@
 
         Example (for a Turtle instance named turtle):
         >>> for i in range(4):
-                turtle.fd(50); turtle.lt(80)
-
+        ...     turtle.fd(50); turtle.lt(80)
+        ...
         >>> for i in range(8):
-                turtle.undo()
+        ...     turtle.undo()
+        ...
         """
         if self.undobuffer is None:
             return
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -399,6 +399,8 @@
 Library
 -------
 
+- Issue #13439: Fix many errors in turtle docstrings.
+
 - Issue #6715: Add a module 'lzma' for compression using the LZMA algorithm.
   Thanks to Per ?yvind Karlsen for the initial implementation.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 00:31:18 2011
From: python-checkins at python.org (brett.cannon)
Date: Sat, 03 Dec 2011 00:31:18 +0100
Subject: [Python-checkins] =?utf8?q?peps=3A_Fix_a_spelling_error=2E?=
Message-ID: <E1RWcZa-0004Lu-Ir@dinsdale.python.org>

http://hg.python.org/peps/rev/22c2c36d0072
changeset:   4008:22c2c36d0072
user:        Brett Cannon <brett at python.org>
date:        Fri Dec 02 18:31:13 2011 -0500
summary:
  Fix a spelling error.

files:
  pep-0362.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-0362.txt b/pep-0362.txt
--- a/pep-0362.txt
+++ b/pep-0362.txt
@@ -44,7 +44,7 @@
 representation affecting the function it represents (but this is an
 `Open Issues`_).
 
-Indirecation of signature introspection can also occur.  If a
+Indirection of signature introspection can also occur.  If a
 decorator took a decorated function's signature object and set it on
 the decorating function then introspection could be redirected to what
 is actually expected instead of the typical ``*args, **kwargs``

-- 
Repository URL: http://hg.python.org/peps

From solipsis at pitrou.net  Sat Dec  3 05:37:27 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 03 Dec 2011 05:37:27 +0100
Subject: [Python-checkins] Daily reference leaks (8d60c1c89105): sum=0
Message-ID: <E1RWhLr-0000s0-Up@ap.vmr.nerim.net>

results for 8d60c1c89105 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogxX2Nwm', '-x']

From python-checkins at python.org  Sat Dec  3 14:44:39 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sat, 03 Dec 2011 14:44:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2312612=3A_Add_some_?=
 =?utf8?q?Valgrind_suppressions_for_64-bit_machines=2E_Patch_by_Paul?=
Message-ID: <E1RWptP-0005HS-Pe@dinsdale.python.org>

http://hg.python.org/cpython/rev/3eb73f45a614
changeset:   73818:3eb73f45a614
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sat Dec 03 14:43:57 2011 +0100
summary:
  Issue #12612: Add some Valgrind suppressions for 64-bit machines. Patch by Paul
Price.

files:
  Misc/ACKS                 |   1 +
  Misc/valgrind-python.supp |  24 ++++++++++++++++++++++++
  2 files changed, 25 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -777,6 +777,7 @@
 Amrit Prem
 Paul Prescod
 Donovan Preston
+Paul Price
 Jyrki Pulliainen
 Steve Purcell
 Eduardo P?rez
diff --git a/Misc/valgrind-python.supp b/Misc/valgrind-python.supp
--- a/Misc/valgrind-python.supp
+++ b/Misc/valgrind-python.supp
@@ -137,6 +137,18 @@
 ###}
 ###
 ###{
+###   ADDRESS_IN_RANGE/Use of uninitialised value of size 8
+###   Memcheck:Addr8
+###   fun:PyObject_Free
+###}
+###
+###{
+###   ADDRESS_IN_RANGE/Use of uninitialised value of size 8
+###   Memcheck:Value8
+###   fun:PyObject_Free
+###}
+###
+###{
 ###   ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
 ###   Memcheck:Cond
 ###   fun:PyObject_Free
@@ -155,6 +167,18 @@
 ###}
 ###
 ###{
+###   ADDRESS_IN_RANGE/Use of uninitialised value of size 8
+###   Memcheck:Addr8
+###   fun:PyObject_Realloc
+###}
+###
+###{
+###   ADDRESS_IN_RANGE/Use of uninitialised value of size 8
+###   Memcheck:Value8
+###   fun:PyObject_Realloc
+###}
+###
+###{
 ###   ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
 ###   Memcheck:Cond
 ###   fun:PyObject_Realloc

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:00:32 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:00:32 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEyNjY2?=
 =?utf8?q?=3A_Clarifying_changes_in_map_for_Python_3?=
Message-ID: <E1RWq8m-0006Nw-Ex@dinsdale.python.org>

http://hg.python.org/cpython/rev/3b505df38fd8
changeset:   73819:3b505df38fd8
branch:      3.2
parent:      73816:cc559e1e3bd8
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Mon Aug 01 17:51:34 2011 -0400
summary:
  Issue #12666: Clarifying changes in map for Python 3

files:
  Doc/whatsnew/3.0.rst |  10 +++++++++-
  1 files changed, 9 insertions(+), 1 deletions(-)


diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst
--- a/Doc/whatsnew/3.0.rst
+++ b/Doc/whatsnew/3.0.rst
@@ -154,7 +154,9 @@
   :meth:`dict.itervalues` methods are no longer supported.
 
 * :func:`map` and :func:`filter` return iterators.  If you really need
-  a list, a quick fix is e.g.  ``list(map(...))``, but a better fix is
+  a list and the input sequences are all of equal length, a quick
+  fix is to wrap :func:`map` in :func:`list`, e.g. ``list(map(...))``,
+  but a better fix is
   often to use a list comprehension (especially when the original code
   uses :keyword:`lambda`), or rewriting the code so it doesn't need a
   list at all.  Particularly tricky is :func:`map` invoked for the
@@ -162,6 +164,12 @@
   regular :keyword:`for` loop (since creating a list would just be
   wasteful).
 
+  If the input sequences are not of equal length, :func:`map` will
+  stop at the termination of the shortest of the sequences. For full
+  compatibility with `map` from Python 2.x, also wrap the sequences in
+  :func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes
+  ``list(map(func, itertools.zip_longest(*sequences)))``.
+
 * :func:`range` now behaves like :func:`xrange` used to behave, except
   it works with values of arbitrary size.  The latter no longer
   exists.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:00:33 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:00:33 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEyNjY2?=
 =?utf8?q?=3A_Added_section_about_map_changes=2E?=
Message-ID: <E1RWq8n-0006O0-LQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/0e2812b16f5f
changeset:   73820:0e2812b16f5f
branch:      3.2
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sat Dec 03 08:24:21 2011 -0500
summary:
  Issue #12666: Added section about map changes.

files:
  Doc/howto/pyporting.rst |  12 ++++++++++++
  1 files changed, 12 insertions(+), 0 deletions(-)


diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst
--- a/Doc/howto/pyporting.rst
+++ b/Doc/howto/pyporting.rst
@@ -505,6 +505,18 @@
 to :mod:`unittest`.
 
 
+Update `map` for imbalanced input sequences
+'''''''''''''''''''''''''''''''''''''''''''
+
+With Python 2, `map` would pad input sequences of unequal length with
+`None` values, returning a sequence as long as the longest input sequence.
+
+With Python 3, if the input sequences to `map` are of unequal length, `map`
+will stop at the termination of the shortest of the sequences. For full
+compatibility with `map` from Python 2.x, also wrap the sequences in
+:func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes
+``list(map(func, itertools.zip_longest(*sequences)))``.
+
 Eliminate ``-3`` Warnings
 -------------------------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:00:36 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:00:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_fix_for_Issue_=2312666_from_3=2E2?=
Message-ID: <E1RWq8q-0006OW-FD@dinsdale.python.org>

http://hg.python.org/cpython/rev/51af35bd46f7
changeset:   73821:51af35bd46f7
parent:      73818:3eb73f45a614
parent:      73820:0e2812b16f5f
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sat Dec 03 08:59:53 2011 -0500
summary:
  Merge fix for Issue #12666 from 3.2

files:
  Doc/howto/pyporting.rst |  12 ++++++++++++
  Doc/whatsnew/3.0.rst    |  10 +++++++++-
  2 files changed, 21 insertions(+), 1 deletions(-)


diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst
--- a/Doc/howto/pyporting.rst
+++ b/Doc/howto/pyporting.rst
@@ -505,6 +505,18 @@
 to :mod:`unittest`.
 
 
+Update `map` for imbalanced input sequences
+'''''''''''''''''''''''''''''''''''''''''''
+
+With Python 2, `map` would pad input sequences of unequal length with
+`None` values, returning a sequence as long as the longest input sequence.
+
+With Python 3, if the input sequences to `map` are of unequal length, `map`
+will stop at the termination of the shortest of the sequences. For full
+compatibility with `map` from Python 2.x, also wrap the sequences in
+:func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes
+``list(map(func, itertools.zip_longest(*sequences)))``.
+
 Eliminate ``-3`` Warnings
 -------------------------
 
diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst
--- a/Doc/whatsnew/3.0.rst
+++ b/Doc/whatsnew/3.0.rst
@@ -154,7 +154,9 @@
   :meth:`dict.itervalues` methods are no longer supported.
 
 * :func:`map` and :func:`filter` return iterators.  If you really need
-  a list, a quick fix is e.g.  ``list(map(...))``, but a better fix is
+  a list and the input sequences are all of equal length, a quick
+  fix is to wrap :func:`map` in :func:`list`, e.g. ``list(map(...))``,
+  but a better fix is
   often to use a list comprehension (especially when the original code
   uses :keyword:`lambda`), or rewriting the code so it doesn't need a
   list at all.  Particularly tricky is :func:`map` invoked for the
@@ -162,6 +164,12 @@
   regular :keyword:`for` loop (since creating a list would just be
   wasteful).
 
+  If the input sequences are not of equal length, :func:`map` will
+  stop at the termination of the shortest of the sequences. For full
+  compatibility with `map` from Python 2.x, also wrap the sequences in
+  :func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes
+  ``list(map(func, itertools.zip_longest(*sequences)))``.
+
 * :func:`range` now behaves like :func:`xrange` used to behave, except
   it works with values of arbitrary size.  The latter no longer
   exists.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:46:11 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:46:11 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzMjEx?=
 =?utf8?q?=3A_Add_=2Ereason_attribute_to_HTTPError_to_implement_parent_cla?=
 =?utf8?q?ss?=
Message-ID: <E1RWqqx-0000qL-UU@dinsdale.python.org>

http://hg.python.org/cpython/rev/ee94b89f65ab
changeset:   73822:ee94b89f65ab
branch:      2.7
parent:      73815:6e03ab9950f6
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Mon Nov 07 10:44:25 2011 -0500
summary:
  Issue #13211: Add .reason attribute to HTTPError to implement parent class (URLError) interface.

files:
  Lib/test/test_urllib2.py |  11 +++++++++++
  Lib/urllib2.py           |   6 ++++++
  2 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -1318,6 +1318,17 @@
         req = Request(url)
         self.assertEqual(req.get_full_url(), url)
 
+def test_HTTPError_interface():
+    """
+    Issue 13211 reveals that HTTPError didn't implement the URLError
+    interface even though HTTPError is a subclass of URLError.
+
+    >>> err = urllib2.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None)
+    >>> assert hasattr(err, 'reason')
+    >>> err.reason
+    'something bad happened'
+    """
+
 def test_main(verbose=None):
     from test import test_urllib2
     test_support.run_doctest(test_urllib2, verbose)
diff --git a/Lib/urllib2.py b/Lib/urllib2.py
--- a/Lib/urllib2.py
+++ b/Lib/urllib2.py
@@ -166,6 +166,12 @@
     def __str__(self):
         return 'HTTP Error %s: %s' % (self.code, self.msg)
 
+    # since URLError specifies a .reason attribute, HTTPError should also
+    #  provide this attribute. See issue13211 fo discussion.
+    @property
+    def reason(self):
+        return self.msg
+
 # copied from cookielib.py
 _cut_port_re = re.compile(r":\d+$")
 def request_host(request):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:46:15 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:46:15 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzMjEx?=
 =?utf8?q?=3A_Add_=2Ereason_attribute_to_HTTPError_to_implement_parent_cla?=
 =?utf8?q?ss?=
Message-ID: <E1RWqr1-0000qP-2f@dinsdale.python.org>

http://hg.python.org/cpython/rev/abfe76a19f63
changeset:   73823:abfe76a19f63
branch:      3.2
parent:      73820:0e2812b16f5f
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Mon Nov 07 10:50:32 2011 -0500
summary:
  Issue #13211: Add .reason attribute to HTTPError to implement parent class (URLError) interface.

files:
  Lib/test/test_urllib2.py |  11 +++++++++++
  Lib/urllib/error.py      |   6 ++++++
  2 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -1409,6 +1409,17 @@
         req = Request(url)
         self.assertEqual(req.get_full_url(), url)
 
+def test_HTTPError_interface():
+    """
+    Issue 13211 reveals that HTTPError didn't implement the URLError
+    interface even though HTTPError is a subclass of URLError.
+
+    >>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None)
+    >>> assert hasattr(err, 'reason')
+    >>> err.reason
+    'something bad happened'
+    """
+
 def test_main(verbose=None):
     from test import test_urllib2
     support.run_doctest(test_urllib2, verbose)
diff --git a/Lib/urllib/error.py b/Lib/urllib/error.py
--- a/Lib/urllib/error.py
+++ b/Lib/urllib/error.py
@@ -52,6 +52,12 @@
     def __str__(self):
         return 'HTTP Error %s: %s' % (self.code, self.msg)
 
+    # since URLError specifies a .reason attribute, HTTPError should also
+    #  provide this attribute. See issue13211 for discussion.
+    @property
+    def reason(self):
+        return self.msg
+
 # exception raised when downloaded size does not match content-length
 class ContentTooShortError(URLError):
     def __init__(self, message, content):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 15:46:18 2011
From: python-checkins at python.org (jason.coombs)
Date: Sat, 03 Dec 2011 15:46:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merged_fix_for_=2313211_from_3=2E2?=
Message-ID: <E1RWqr4-0000qy-Vh@dinsdale.python.org>

http://hg.python.org/cpython/rev/deb60efd32eb
changeset:   73824:deb60efd32eb
parent:      73821:51af35bd46f7
parent:      73823:abfe76a19f63
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sat Dec 03 09:39:58 2011 -0500
summary:
  Merged fix for #13211 from 3.2

files:
  Lib/test/test_urllib2.py |  11 +++++++++++
  Lib/urllib/error.py      |   6 ++++++
  2 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -1449,6 +1449,17 @@
         req = Request(url)
         self.assertEqual(req.get_full_url(), url)
 
+def test_HTTPError_interface():
+    """
+    Issue 13211 reveals that HTTPError didn't implement the URLError
+    interface even though HTTPError is a subclass of URLError.
+
+    >>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None)
+    >>> assert hasattr(err, 'reason')
+    >>> err.reason
+    'something bad happened'
+    """
+
 def test_main(verbose=None):
     from test import test_urllib2
     support.run_doctest(test_urllib2, verbose)
diff --git a/Lib/urllib/error.py b/Lib/urllib/error.py
--- a/Lib/urllib/error.py
+++ b/Lib/urllib/error.py
@@ -55,6 +55,12 @@
     def __str__(self):
         return 'HTTP Error %s: %s' % (self.code, self.msg)
 
+    # since URLError specifies a .reason attribute, HTTPError should also
+    #  provide this attribute. See issue13211 for discussion.
+    @property
+    def reason(self):
+        return self.msg
+
 # exception raised when downloaded size does not match content-length
 class ContentTooShortError(URLError):
     def __init__(self, message, content):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 16:01:29 2011
From: python-checkins at python.org (eric.araujo)
Date: Sat, 03 Dec 2011 16:01:29 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_glitches_in_email=2Epol?=
	=?utf8?q?icy_docs_=28=2312208=29?=
Message-ID: <E1RWr5l-0001xc-G9@dinsdale.python.org>

http://hg.python.org/cpython/rev/9ffb00748a47
changeset:   73825:9ffb00748a47
user:        ?ric Araujo <merwok at netwok.org>
date:        Sat Dec 03 16:00:56 2011 +0100
summary:
  Fix glitches in email.policy docs (#12208)

files:
  Doc/library/email.policy.rst |  6 ++++--
  1 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst
--- a/Doc/library/email.policy.rst
+++ b/Doc/library/email.policy.rst
@@ -48,16 +48,18 @@
    >>> import email.policy
    >>> from subprocess import Popen, PIPE
    >>> with open('mymsg.txt', 'b') as f:
-   ...     Msg = msg_from_binary_file(f, policy=email.policy.mbox)
+   ...     msg = msg_from_binary_file(f, policy=email.policy.mbox)
    >>> p = Popen(['sendmail', msg['To'][0].address], stdin=PIPE)
    >>> g = BytesGenerator(p.stdin, policy=email.policy.SMTP)
    >>> g.flatten(msg)
    >>> p.stdin.close()
    >>> rc = p.wait()
 
+.. XXX email.policy.mbox/MBOX does not exist yet
+
 Some email package methods accept a *policy* keyword argument, allowing the
 policy to be overridden for that method.  For example, the following code uses
-the :meth:`email.message.Message.as_string` method of the *msg* object from the
+the :meth:`~email.message.Message.as_string` method of the *msg* object from the
 previous example and re-write it to a file using the native line separators for
 the platform on which it is running::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 19:51:40 2011
From: python-checkins at python.org (meador.inge)
Date: Sat, 03 Dec 2011 19:51:40 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTEz?=
 =?utf8?q?=3A_IOBase_docs_incorrectly_link_to_the_readline_module?=
Message-ID: <E1RWugW-0001Ss-Up@dinsdale.python.org>

http://hg.python.org/cpython/rev/fb8b6d310fb8
changeset:   73826:fb8b6d310fb8
branch:      2.7
parent:      73822:ee94b89f65ab
user:        Meador Inge <meadori at gmail.com>
date:        Sat Dec 03 12:13:42 2011 -0600
summary:
  Issue #13513: IOBase docs incorrectly link to the readline module

files:
  Doc/library/io.rst |  2 +-
  Misc/NEWS          |  3 +++
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Doc/library/io.rst b/Doc/library/io.rst
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -233,7 +233,7 @@
    :class:`IOBase` object can be iterated over yielding the lines in a stream.
    Lines are defined slightly differently depending on whether the stream is
    a binary stream (yielding :class:`bytes`), or a text stream (yielding
-   :class:`unicode` strings).  See :meth:`readline` below.
+   :class:`unicode` strings).  See :meth:`~IOBase.readline` below.
 
    IOBase is also a context manager and therefore supports the
    :keyword:`with` statement.  In this example, *file* is closed after the
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -430,6 +430,9 @@
 Documentation
 -------------
 
+- Issue #13513: Fix io.IOBase documentation to correctly link to the
+  io.IOBase.readline method instead of the readline module.
+
 - Issue #13237: Reorganise subprocess documentation to emphasise convenience
   functions and the most commonly needed arguments to Popen.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 19:51:45 2011
From: python-checkins at python.org (meador.inge)
Date: Sat, 03 Dec 2011 19:51:45 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTEz?=
 =?utf8?q?=3A_IOBase_docs_incorrectly_link_to_the_readline_module?=
Message-ID: <E1RWugb-0001Sx-0I@dinsdale.python.org>

http://hg.python.org/cpython/rev/9792e812198f
changeset:   73827:9792e812198f
branch:      3.2
parent:      73823:abfe76a19f63
user:        Meador Inge <meadori at gmail.com>
date:        Sat Dec 03 12:29:54 2011 -0600
summary:
  Issue #13513: IOBase docs incorrectly link to the readline module

files:
  Doc/library/io.rst |  2 +-
  Misc/NEWS          |  3 +++
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Doc/library/io.rst b/Doc/library/io.rst
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -217,7 +217,7 @@
    :class:`IOBase` object can be iterated over yielding the lines in a stream.
    Lines are defined slightly differently depending on whether the stream is
    a binary stream (yielding bytes), or a text stream (yielding character
-   strings).  See :meth:`readline` below.
+   strings).  See :meth:`~IOBase.readline` below.
 
    IOBase is also a context manager and therefore supports the
    :keyword:`with` statement.  In this example, *file* is closed after the
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -316,6 +316,9 @@
 Documentation
 -------------
 
+- Issue #13513: Fix io.IOBase documentation to correctly link to the
+  io.IOBase.readline method instead of the readline module.
+
 - Issue #13237: Reorganise subprocess documentation to emphasise convenience
   functions and the most commonly needed arguments to Popen.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 19:51:48 2011
From: python-checkins at python.org (meador.inge)
Date: Sat, 03 Dec 2011 19:51:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313513=3A_IOBase_docs_incorrectly_link_to_the_readli?=
 =?utf8?q?ne_module?=
Message-ID: <E1RWuge-0001Ti-4z@dinsdale.python.org>

http://hg.python.org/cpython/rev/ab5bc05ac223
changeset:   73828:ab5bc05ac223
parent:      73825:9ffb00748a47
parent:      73827:9792e812198f
user:        Meador Inge <meadori at gmail.com>
date:        Sat Dec 03 12:50:18 2011 -0600
summary:
  Issue #13513: IOBase docs incorrectly link to the readline module

files:
  Doc/library/io.rst |  2 +-
  Misc/NEWS          |  3 +++
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Doc/library/io.rst b/Doc/library/io.rst
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -213,7 +213,7 @@
    :class:`IOBase` object can be iterated over yielding the lines in a stream.
    Lines are defined slightly differently depending on whether the stream is
    a binary stream (yielding bytes), or a text stream (yielding character
-   strings).  See :meth:`readline` below.
+   strings).  See :meth:`~IOBase.readline` below.
 
    IOBase is also a context manager and therefore supports the
    :keyword:`with` statement.  In this example, *file* is closed after the
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1885,6 +1885,9 @@
 Documentation
 -------------
 
+- Issue #13513: Fix io.IOBase documentation to correctly link to the
+  io.IOBase.readline method instead of the readline module.
+
 - Issue #13237: Reorganise subprocess documentation to emphasise convenience
   functions and the most commonly needed arguments to Popen.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 21:13:06 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 21:13:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Mention_the_new?=
	=?utf8?q?_GTK+_3_bindings=2E?=
Message-ID: <E1RWvxK-0006ki-9c@dinsdale.python.org>

http://hg.python.org/cpython/rev/1e3f09a896da
changeset:   73829:1e3f09a896da
branch:      3.2
parent:      73827:9792e812198f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 20:59:24 2011 +0100
summary:
  Mention the new GTK+ 3 bindings.

files:
  Doc/faq/gui.rst          |   8 ++++++--
  Doc/library/othergui.rst |  16 +++++++++++-----
  2 files changed, 17 insertions(+), 7 deletions(-)


diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst
--- a/Doc/faq/gui.rst
+++ b/Doc/faq/gui.rst
@@ -68,8 +68,12 @@
 Gtk+
 ----
 
-PyGtk bindings for the `Gtk+ toolkit <http://www.gtk.org>`_ have been
-implemented by James Henstridge; see <http://www.pygtk.org>.
+The `GObject introspection bindings <https://live.gnome.org/PyGObject>`_
+for Python allow you to write GTK+ 3 applications.  There is also a
+`Python GTK+ 3 Tutorial <http://python-gtk-3-tutorial.readthedocs.org/en/latest/>`_.
+
+The older PyGtk bindings for the `Gtk+ 2 toolkit <http://www.gtk.org>`_ have
+been implemented by James Henstridge; see <http://www.pygtk.org>.
 
 FLTK
 ----
diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -34,11 +34,17 @@
 
 .. seealso::
 
-   `PyGTK <http://www.pygtk.org/>`_
-      is a set of bindings for the `GTK <http://www.gtk.org/>`_ widget set. It
-      provides an object oriented interface that is slightly higher level than
-      the C one. It comes with many more widgets than Tkinter provides, and has
-      good Python-specific reference documentation. There are also bindings to
+   `PyGObject <https://live.gnome.org/PyGObject>`_
+      provides introspection bindings for C libraries using
+      `GObject <http://developer.gnome.org/gobject/stable/>`_.  One of
+      these libraries is the `GTK+ 3 <http://www.gtk.org/>`_ widget set.
+      GTK+ comes with many more widgets than Tkinter provides.  An online
+      `Python GTK+ 3 Tutorial <http://python-gtk-3-tutorial.readthedocs.org/en/latest/>`_
+      is available.
+
+      `PyGTK <http://www.pygtk.org/>`_ provides bindings for an older version
+      of the library, GTK+ 2.  It provides an object oriented interface that
+      is slightly higher level than the C one.  There are also bindings to
       `GNOME <http://www.gnome.org>`_.  One well known PyGTK application is
       `PythonCAD <http://www.pythoncad.org/>`_. An online `tutorial
       <http://www.pygtk.org/pygtk2tutorial/index.html>`_ is available.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 21:13:07 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 21:13:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Mention_PySide_?=
 =?utf8?q?in_the_=22other_GUIs=22_page=2E?=
Message-ID: <E1RWvxL-0006ks-1R@dinsdale.python.org>

http://hg.python.org/cpython/rev/bc3f1629d825
changeset:   73830:bc3f1629d825
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 21:07:01 2011 +0100
summary:
  Mention PySide in the "other GUIs" page.

files:
  Doc/library/othergui.rst |  5 +++++
  1 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -61,6 +61,11 @@
       with Python and Qt <http://www.qtrac.eu/pyqtbook.html>`_, by Mark
       Summerfield.
 
+   `PySide <http://www.pyside.org/>`_
+      is a newer binding to the Qt toolkit, provided by Nokia.
+      Compared to PyQt, its licensing scheme is friendlier to non-open source
+      applications.
+
    `wxPython <http://www.wxpython.org>`_
       wxPython is a cross-platform GUI toolkit for Python that is built around
       the popular `wxWidgets <http://www.wxwidgets.org/>`_ (formerly wxWindows)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 21:13:07 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 21:13:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_doc_fixes_from_3=2E2?=
Message-ID: <E1RWvxL-0006kw-QK@dinsdale.python.org>

http://hg.python.org/cpython/rev/309f12301de2
changeset:   73831:309f12301de2
parent:      73828:ab5bc05ac223
parent:      73830:bc3f1629d825
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 21:07:50 2011 +0100
summary:
  Merge doc fixes from 3.2

files:
  Doc/faq/gui.rst          |   8 ++++++--
  Doc/library/othergui.rst |  21 ++++++++++++++++-----
  2 files changed, 22 insertions(+), 7 deletions(-)


diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst
--- a/Doc/faq/gui.rst
+++ b/Doc/faq/gui.rst
@@ -68,8 +68,12 @@
 Gtk+
 ----
 
-PyGtk bindings for the `Gtk+ toolkit <http://www.gtk.org>`_ have been
-implemented by James Henstridge; see <http://www.pygtk.org>.
+The `GObject introspection bindings <https://live.gnome.org/PyGObject>`_
+for Python allow you to write GTK+ 3 applications.  There is also a
+`Python GTK+ 3 Tutorial <http://python-gtk-3-tutorial.readthedocs.org/en/latest/>`_.
+
+The older PyGtk bindings for the `Gtk+ 2 toolkit <http://www.gtk.org>`_ have
+been implemented by James Henstridge; see <http://www.pygtk.org>.
 
 FLTK
 ----
diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -34,11 +34,17 @@
 
 .. seealso::
 
-   `PyGTK <http://www.pygtk.org/>`_
-      is a set of bindings for the `GTK <http://www.gtk.org/>`_ widget set. It
-      provides an object oriented interface that is slightly higher level than
-      the C one. It comes with many more widgets than Tkinter provides, and has
-      good Python-specific reference documentation. There are also bindings to
+   `PyGObject <https://live.gnome.org/PyGObject>`_
+      provides introspection bindings for C libraries using
+      `GObject <http://developer.gnome.org/gobject/stable/>`_.  One of
+      these libraries is the `GTK+ 3 <http://www.gtk.org/>`_ widget set.
+      GTK+ comes with many more widgets than Tkinter provides.  An online
+      `Python GTK+ 3 Tutorial <http://python-gtk-3-tutorial.readthedocs.org/en/latest/>`_
+      is available.
+
+      `PyGTK <http://www.pygtk.org/>`_ provides bindings for an older version
+      of the library, GTK+ 2.  It provides an object oriented interface that
+      is slightly higher level than the C one.  There are also bindings to
       `GNOME <http://www.gnome.org>`_.  One well known PyGTK application is
       `PythonCAD <http://www.pythoncad.org/>`_. An online `tutorial
       <http://www.pygtk.org/pygtk2tutorial/index.html>`_ is available.
@@ -55,6 +61,11 @@
       with Python and Qt <http://www.qtrac.eu/pyqtbook.html>`_, by Mark
       Summerfield.
 
+   `PySide <http://www.pyside.org/>`_
+      is a newer binding to the Qt toolkit, provided by Nokia.
+      Compared to PyQt, its licensing scheme is friendlier to non-open source
+      applications.
+
    `wxPython <http://www.wxpython.org>`_
       wxPython is a cross-platform GUI toolkit for Python that is built around
       the popular `wxWidgets <http://www.wxwidgets.org/>`_ (formerly wxWindows)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 21:30:38 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 21:30:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_referenc?=
 =?utf8?q?es_to_psyco=2C_which_is_mostly_unmaintained_and_doesn=27t_work_w?=
 =?utf8?q?ith?=
Message-ID: <E1RWwEI-0000Ks-C1@dinsdale.python.org>

http://hg.python.org/cpython/rev/48a723092e1e
changeset:   73832:48a723092e1e
branch:      3.2
parent:      73830:bc3f1629d825
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 21:21:36 2011 +0100
summary:
  Remove references to psyco, which is mostly unmaintained and doesn't work with Python 3.

files:
  Doc/faq/extending.rst   |   8 +-------
  Doc/faq/programming.rst |  14 +++-----------
  2 files changed, 4 insertions(+), 18 deletions(-)


diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
--- a/Doc/faq/extending.rst
+++ b/Doc/faq/extending.rst
@@ -37,13 +37,7 @@
 There are a number of alternatives to writing your own C extensions, depending
 on what you're trying to do.
 
-.. XXX make sure these all work; mention Cython
-
-If you need more speed, `Psyco <http://psyco.sourceforge.net/>`_ generates x86
-assembly code from Python bytecode.  You can use Psyco to compile the most
-time-critical functions in your code, and gain a significant improvement with
-very little effort, as long as you're running on a machine with an
-x86-compatible processor.
+.. XXX make sure these all work
 
 `Cython <http://cython.org>`_ and its relative `Pyrex
 <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ are compilers
diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -121,19 +121,11 @@
 That's a tough one, in general.  There are many tricks to speed up Python code;
 consider rewriting parts in C as a last resort.
 
-In some cases it's possible to automatically translate Python to C or x86
-assembly language, meaning that you don't have to modify your code to gain
-increased speed.
-
-.. XXX seems to have overlap with other questions!
-
 `Cython <http://cython.org>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
 can compile a slightly modified version of Python code into a C extension, and
-can be used on many different platforms.
-
-`Psyco <http://psyco.sourceforge.net>`_ is a just-in-time compiler that
-translates Python code into x86 assembly language.  If you can use it, Psyco can
-provide dramatic speedups for critical functions.
+can be used on many different platforms.  Depending on your code, Cython
+may be able to make it significantly faster than when run by the Python
+interpreter.
 
 The rest of this answer will discuss various tricks for squeezing a bit more
 speed out of Python code.  *Never* apply any optimization tricks unless you know

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 21:30:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 21:30:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Remove_references_to_psyco=2C_which_is_mostly_unmaintained_a?=
 =?utf8?q?nd_doesn=27t_work_with?=
Message-ID: <E1RWwEJ-0000L8-2g@dinsdale.python.org>

http://hg.python.org/cpython/rev/06087f6890af
changeset:   73833:06087f6890af
parent:      73831:309f12301de2
parent:      73832:48a723092e1e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 21:25:17 2011 +0100
summary:
  Remove references to psyco, which is mostly unmaintained and doesn't work with Python 3.

files:
  Doc/faq/extending.rst   |   8 +-------
  Doc/faq/programming.rst |  14 +++-----------
  2 files changed, 4 insertions(+), 18 deletions(-)


diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
--- a/Doc/faq/extending.rst
+++ b/Doc/faq/extending.rst
@@ -37,13 +37,7 @@
 There are a number of alternatives to writing your own C extensions, depending
 on what you're trying to do.
 
-.. XXX make sure these all work; mention Cython
-
-If you need more speed, `Psyco <http://psyco.sourceforge.net/>`_ generates x86
-assembly code from Python bytecode.  You can use Psyco to compile the most
-time-critical functions in your code, and gain a significant improvement with
-very little effort, as long as you're running on a machine with an
-x86-compatible processor.
+.. XXX make sure these all work
 
 `Cython <http://cython.org>`_ and its relative `Pyrex
 <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ are compilers
diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -121,19 +121,11 @@
 That's a tough one, in general.  There are many tricks to speed up Python code;
 consider rewriting parts in C as a last resort.
 
-In some cases it's possible to automatically translate Python to C or x86
-assembly language, meaning that you don't have to modify your code to gain
-increased speed.
-
-.. XXX seems to have overlap with other questions!
-
 `Cython <http://cython.org>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
 can compile a slightly modified version of Python code into a C extension, and
-can be used on many different platforms.
-
-`Psyco <http://psyco.sourceforge.net>`_ is a just-in-time compiler that
-translates Python code into x86 assembly language.  If you can use it, Psyco can
-provide dramatic speedups for critical functions.
+can be used on many different platforms.  Depending on your code, Cython
+may be able to make it significantly faster than when run by the Python
+interpreter.
 
 The rest of this answer will discuss various tricks for squeezing a bit more
 speed out of Python code.  *Never* apply any optimization tricks unless you know

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:17:02 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:17:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Streamline_FAQ_?=
 =?utf8?q?entry_about_the_ternary_operator=2C_and_suggest_using_io=2EStrin?=
 =?utf8?q?gIO?=
Message-ID: <E1RWwxC-0003It-5J@dinsdale.python.org>

http://hg.python.org/cpython/rev/140b8c98c7b9
changeset:   73834:140b8c98c7b9
branch:      3.2
parent:      73832:48a723092e1e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:11:11 2011 +0100
summary:
  Streamline FAQ entry about the ternary operator, and suggest using io.StringIO for a mutable unicode container.

files:
  Doc/faq/programming.rst |  74 +++++++---------------------
  1 files changed, 20 insertions(+), 54 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -679,61 +679,21 @@
 Is there an equivalent of C's "?:" ternary operator?
 ----------------------------------------------------
 
-Yes, this feature was added in Python 2.5. The syntax would be as follows::
+Yes, there is. The syntax is as follows::
 
    [on_true] if [expression] else [on_false]
 
    x, y = 50, 25
-
    small = x if x < y else y
 
-For versions previous to 2.5 the answer would be 'No'.
+Before this syntax was introduced in Python 2.5, a common idiom was to use
+logical operators::
 
-.. XXX remove rest?
+   [expression] and [on_true] or [on_false]
 
-In many cases you can mimic ``a ? b : c`` with ``a and b or c``, but there's a
-flaw: if *b* is zero (or empty, or ``None`` -- anything that tests false) then
-*c* will be selected instead.  In many cases you can prove by looking at the
-code that this can't happen (e.g. because *b* is a constant or has a type that
-can never be false), but in general this can be a problem.
-
-Tim Peters (who wishes it was Steve Majewski) suggested the following solution:
-``(a and [b] or [c])[0]``.  Because ``[b]`` is a singleton list it is never
-false, so the wrong path is never taken; then applying ``[0]`` to the whole
-thing gets the *b* or *c* that you really wanted.  Ugly, but it gets you there
-in the rare cases where it is really inconvenient to rewrite your code using
-'if'.
-
-The best course is usually to write a simple ``if...else`` statement.  Another
-solution is to implement the ``?:`` operator as a function::
-
-   def q(cond, on_true, on_false):
-       if cond:
-           if not isfunction(on_true):
-               return on_true
-           else:
-               return on_true()
-       else:
-           if not isfunction(on_false):
-               return on_false
-           else:
-               return on_false()
-
-In most cases you'll pass b and c directly: ``q(a, b, c)``.  To avoid evaluating
-b or c when they shouldn't be, encapsulate them within a lambda function, e.g.:
-``q(a, lambda: b, lambda: c)``.
-
-It has been asked *why* Python has no if-then-else expression.  There are
-several answers: many languages do just fine without one; it can easily lead to
-less readable code; no sufficiently "Pythonic" syntax has been discovered; a
-search of the standard library found remarkably few places where using an
-if-then-else expression would make the code more understandable.
-
-In 2002, :pep:`308` was written proposing several possible syntaxes and the
-community was asked to vote on the issue.  The vote was inconclusive.  Most
-people liked one of the syntaxes, but also hated other syntaxes; many votes
-implied that people preferred no ternary operator rather than having a syntax
-they hated.
+However, this idiom is unsafe, as it can give wrong results when *on_true*
+has a false boolean value.  Therefore, it is always better to use
+the ``... if ... else ...`` form.
 
 
 Is it possible to write obfuscated one-liners in Python?
@@ -852,15 +812,21 @@
 How do I modify a string in place?
 ----------------------------------
 
-You can't, because strings are immutable.  If you need an object with this
-ability, try converting the string to a list or use the array module::
+You can't, because strings are immutable.  In most situations, you should
+simply construct a new string from the various parts you want to assemble
+it from.  However, if you need an object with the ability to modify in-place
+unicode data, try using a :class:`io.StringIO` object or the :mod:`array`
+module::
 
    >>> s = "Hello, world"
-   >>> a = list(s)
-   >>> print(a)
-   ['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']
-   >>> a[7:] = list("there!")
-   >>> ''.join(a)
+   >>> sio = io.StringIO(s)
+   >>> sio.getvalue()
+   'Hello, world'
+   >>> sio.seek(7)
+   7
+   >>> sio.write("there!")
+   6
+   >>> sio.getvalue()
    'Hello, there!'
 
    >>> import array

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:17:02 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:17:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Streamline_FAQ_entry_about_the_ternary_operator=2C_and_sugge?=
 =?utf8?q?st_using_io=2EStringIO?=
Message-ID: <E1RWwxC-0003J3-Sl@dinsdale.python.org>

http://hg.python.org/cpython/rev/10440e132cfb
changeset:   73835:10440e132cfb
parent:      73833:06087f6890af
parent:      73834:140b8c98c7b9
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:11:45 2011 +0100
summary:
  Streamline FAQ entry about the ternary operator, and suggest using io.StringIO for a mutable unicode container.

files:
  Doc/faq/programming.rst |  74 +++++++---------------------
  1 files changed, 20 insertions(+), 54 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -679,61 +679,21 @@
 Is there an equivalent of C's "?:" ternary operator?
 ----------------------------------------------------
 
-Yes, this feature was added in Python 2.5. The syntax would be as follows::
+Yes, there is. The syntax is as follows::
 
    [on_true] if [expression] else [on_false]
 
    x, y = 50, 25
-
    small = x if x < y else y
 
-For versions previous to 2.5 the answer would be 'No'.
+Before this syntax was introduced in Python 2.5, a common idiom was to use
+logical operators::
 
-.. XXX remove rest?
+   [expression] and [on_true] or [on_false]
 
-In many cases you can mimic ``a ? b : c`` with ``a and b or c``, but there's a
-flaw: if *b* is zero (or empty, or ``None`` -- anything that tests false) then
-*c* will be selected instead.  In many cases you can prove by looking at the
-code that this can't happen (e.g. because *b* is a constant or has a type that
-can never be false), but in general this can be a problem.
-
-Tim Peters (who wishes it was Steve Majewski) suggested the following solution:
-``(a and [b] or [c])[0]``.  Because ``[b]`` is a singleton list it is never
-false, so the wrong path is never taken; then applying ``[0]`` to the whole
-thing gets the *b* or *c* that you really wanted.  Ugly, but it gets you there
-in the rare cases where it is really inconvenient to rewrite your code using
-'if'.
-
-The best course is usually to write a simple ``if...else`` statement.  Another
-solution is to implement the ``?:`` operator as a function::
-
-   def q(cond, on_true, on_false):
-       if cond:
-           if not isfunction(on_true):
-               return on_true
-           else:
-               return on_true()
-       else:
-           if not isfunction(on_false):
-               return on_false
-           else:
-               return on_false()
-
-In most cases you'll pass b and c directly: ``q(a, b, c)``.  To avoid evaluating
-b or c when they shouldn't be, encapsulate them within a lambda function, e.g.:
-``q(a, lambda: b, lambda: c)``.
-
-It has been asked *why* Python has no if-then-else expression.  There are
-several answers: many languages do just fine without one; it can easily lead to
-less readable code; no sufficiently "Pythonic" syntax has been discovered; a
-search of the standard library found remarkably few places where using an
-if-then-else expression would make the code more understandable.
-
-In 2002, :pep:`308` was written proposing several possible syntaxes and the
-community was asked to vote on the issue.  The vote was inconclusive.  Most
-people liked one of the syntaxes, but also hated other syntaxes; many votes
-implied that people preferred no ternary operator rather than having a syntax
-they hated.
+However, this idiom is unsafe, as it can give wrong results when *on_true*
+has a false boolean value.  Therefore, it is always better to use
+the ``... if ... else ...`` form.
 
 
 Is it possible to write obfuscated one-liners in Python?
@@ -852,15 +812,21 @@
 How do I modify a string in place?
 ----------------------------------
 
-You can't, because strings are immutable.  If you need an object with this
-ability, try converting the string to a list or use the array module::
+You can't, because strings are immutable.  In most situations, you should
+simply construct a new string from the various parts you want to assemble
+it from.  However, if you need an object with the ability to modify in-place
+unicode data, try using a :class:`io.StringIO` object or the :mod:`array`
+module::
 
    >>> s = "Hello, world"
-   >>> a = list(s)
-   >>> print(a)
-   ['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']
-   >>> a[7:] = list("there!")
-   >>> ''.join(a)
+   >>> sio = io.StringIO(s)
+   >>> sio.getvalue()
+   'Hello, world'
+   >>> sio.seek(7)
+   7
+   >>> sio.write("there!")
+   6
+   >>> sio.getvalue()
    'Hello, there!'
 
    >>> import array

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:31:15 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:31:15 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_outdate_?=
	=?utf8?q?FAQ_content?=
Message-ID: <E1RWxAx-0004RA-Bq@dinsdale.python.org>

http://hg.python.org/cpython/rev/93f64ae51fff
changeset:   73836:93f64ae51fff
branch:      3.2
parent:      73834:140b8c98c7b9
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:19:55 2011 +0100
summary:
  Remove outdate FAQ content

files:
  Doc/faq/programming.rst |  40 +++++-----------------------
  1 files changed, 8 insertions(+), 32 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -901,11 +901,11 @@
 Is there an equivalent to Perl's chomp() for removing trailing newlines from strings?
 -------------------------------------------------------------------------------------
 
-Starting with Python 2.2, you can use ``S.rstrip("\r\n")`` to remove all
-occurrences of any line terminator from the end of the string ``S`` without
-removing other trailing whitespace.  If the string ``S`` represents more than
-one line, with several empty lines at the end, the line terminators for all the
-blank lines will be removed::
+You can use ``S.rstrip("\r\n")`` to remove all occurrences of any line
+terminator from the end of the string ``S`` without removing other trailing
+whitespace.  If the string ``S`` represents more than one line, with several
+empty lines at the end, the line terminators for all the blank lines will
+be removed::
 
    >>> lines = ("line 1 \r\n"
    ...          "\r\n"
@@ -916,15 +916,6 @@
 Since this is typically only desired when reading text one line at a time, using
 ``S.rstrip()`` this way works well.
 
-For older versions of Python, there are two partial substitutes:
-
-- If you want to remove all trailing whitespace, use the ``rstrip()`` method of
-  string objects.  This removes all trailing whitespace, not just a single
-  newline.
-
-- Otherwise, if there is only one line in the string ``S``, use
-  ``S.splitlines()[0]``.
-
 
 Is there a scanf() or sscanf() equivalent?
 ------------------------------------------
@@ -1042,15 +1033,8 @@
            else:
                last = mylist[i]
 
-If all elements of the list may be used as dictionary keys (i.e. they are all
-hashable) this is often faster ::
-
-   d = {}
-   for x in mylist:
-       d[x] = 1
-   mylist = list(d.keys())
-
-In Python 2.5 and later, the following is possible instead::
+If all elements of the list may be used as set keys (i.e. they are all
+:term:`hashable`) this is often faster ::
 
    mylist = list(set(mylist))
 
@@ -1420,15 +1404,7 @@
 
    C.count = 314
 
-Static methods are possible since Python 2.2::
-
-   class C:
-       def static(arg1, arg2, arg3):
-           # No 'self' parameter!
-           ...
-       static = staticmethod(static)
-
-With Python 2.4's decorators, this can also be written as ::
+Static methods are possible::
 
    class C:
        @staticmethod

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:31:16 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:31:16 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Remove_outdated_FAQ_content?=
Message-ID: <E1RWxAy-0004SE-2a@dinsdale.python.org>

http://hg.python.org/cpython/rev/3d96c2dbec13
changeset:   73837:3d96c2dbec13
parent:      73835:10440e132cfb
parent:      73836:93f64ae51fff
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:26:01 2011 +0100
summary:
  Remove outdated FAQ content

files:
  Doc/faq/programming.rst |  40 +++++-----------------------
  1 files changed, 8 insertions(+), 32 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -901,11 +901,11 @@
 Is there an equivalent to Perl's chomp() for removing trailing newlines from strings?
 -------------------------------------------------------------------------------------
 
-Starting with Python 2.2, you can use ``S.rstrip("\r\n")`` to remove all
-occurrences of any line terminator from the end of the string ``S`` without
-removing other trailing whitespace.  If the string ``S`` represents more than
-one line, with several empty lines at the end, the line terminators for all the
-blank lines will be removed::
+You can use ``S.rstrip("\r\n")`` to remove all occurrences of any line
+terminator from the end of the string ``S`` without removing other trailing
+whitespace.  If the string ``S`` represents more than one line, with several
+empty lines at the end, the line terminators for all the blank lines will
+be removed::
 
    >>> lines = ("line 1 \r\n"
    ...          "\r\n"
@@ -916,15 +916,6 @@
 Since this is typically only desired when reading text one line at a time, using
 ``S.rstrip()`` this way works well.
 
-For older versions of Python, there are two partial substitutes:
-
-- If you want to remove all trailing whitespace, use the ``rstrip()`` method of
-  string objects.  This removes all trailing whitespace, not just a single
-  newline.
-
-- Otherwise, if there is only one line in the string ``S``, use
-  ``S.splitlines()[0]``.
-
 
 Is there a scanf() or sscanf() equivalent?
 ------------------------------------------
@@ -1042,15 +1033,8 @@
            else:
                last = mylist[i]
 
-If all elements of the list may be used as dictionary keys (i.e. they are all
-hashable) this is often faster ::
-
-   d = {}
-   for x in mylist:
-       d[x] = 1
-   mylist = list(d.keys())
-
-In Python 2.5 and later, the following is possible instead::
+If all elements of the list may be used as set keys (i.e. they are all
+:term:`hashable`) this is often faster ::
 
    mylist = list(set(mylist))
 
@@ -1420,15 +1404,7 @@
 
    C.count = 314
 
-Static methods are possible since Python 2.2::
-
-   class C:
-       def static(arg1, arg2, arg3):
-           # No 'self' parameter!
-           ...
-       static = staticmethod(static)
-
-With Python 2.4's decorators, this can also be written as ::
+Static methods are possible::
 
    class C:
        @staticmethod

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:45:37 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:45:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Advocate_PyTupl?=
 =?utf8?q?e=5FPack_instead_of_manual_building_of_tuples?=
Message-ID: <E1RWxOr-0005Rl-6n@dinsdale.python.org>

http://hg.python.org/cpython/rev/8267af6f370d
changeset:   73838:8267af6f370d
branch:      3.2
parent:      73836:93f64ae51fff
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:30:19 2011 +0100
summary:
  Advocate PyTuple_Pack instead of manual building of tuples

files:
  Doc/faq/extending.rst |  7 +------
  1 files changed, 1 insertions(+), 6 deletions(-)


diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
--- a/Doc/faq/extending.rst
+++ b/Doc/faq/extending.rst
@@ -99,12 +99,7 @@
 How do I use Py_BuildValue() to create a tuple of arbitrary length?
 -------------------------------------------------------------------
 
-You can't.  Use ``t = PyTuple_New(n)`` instead, and fill it with objects using
-``PyTuple_SetItem(t, i, o)`` -- note that this "eats" a reference count of
-``o``, so you have to :c:func:`Py_INCREF` it.  Lists have similar functions
-``PyList_New(n)`` and ``PyList_SetItem(l, i, o)``.  Note that you *must* set all
-the tuple items to some value before you pass the tuple to Python code --
-``PyTuple_New(n)`` initializes them to NULL, which isn't a valid Python value.
+You can't.  Use :c:func:`PyTuple_Pack` instead.
 
 
 How do I call an object's method from C?

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:45:37 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:45:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Refine_FAQ_entr?=
	=?utf8?q?y_for_catching_stdout?=
Message-ID: <E1RWxOr-0005SN-TT@dinsdale.python.org>

http://hg.python.org/cpython/rev/c82492fc9943
changeset:   73839:c82492fc9943
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:35:31 2011 +0100
summary:
  Refine FAQ entry for catching stdout

files:
  Doc/faq/extending.rst |  21 +++++++++++++++------
  1 files changed, 15 insertions(+), 6 deletions(-)


diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
--- a/Doc/faq/extending.rst
+++ b/Doc/faq/extending.rst
@@ -142,21 +142,30 @@
 just allow the standard traceback mechanism to work. Then, the output will go
 wherever your ``write()`` method sends it.
 
-The easiest way to do this is to use the StringIO class in the standard library.
+The easiest way to do this is to use the :class:`io.StringIO` class::
 
-Sample code and use for catching stdout:
+   >>> import io, sys
+   >>> sys.stdout = io.StringIO()
+   >>> print('foo')
+   >>> print('hello world!')
+   >>> sys.stderr.write(sys.stdout.getvalue())
+   foo
+   hello world!
 
-   >>> class StdoutCatcher:
+A custom object to do the same would look like this::
+
+   >>> import io, sys
+   >>> class StdoutCatcher(io.TextIOBase):
    ...     def __init__(self):
-   ...         self.data = ''
+   ...         self.data = []
    ...     def write(self, stuff):
-   ...         self.data = self.data + stuff
+   ...         self.data.append(stuff)
    ...
    >>> import sys
    >>> sys.stdout = StdoutCatcher()
    >>> print('foo')
    >>> print('hello world!')
-   >>> sys.stderr.write(sys.stdout.data)
+   >>> sys.stderr.write(''.join(sys.stdout.data))
    foo
    hello world!
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:45:38 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:45:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_outdated?=
 =?utf8?q?_question_=28the_bsddb_module_isn=27t_included_anymore=29?=
Message-ID: <E1RWxOs-0005SR-Jc@dinsdale.python.org>

http://hg.python.org/cpython/rev/040c751e5497
changeset:   73840:040c751e5497
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:37:14 2011 +0100
summary:
  Remove outdated question (the bsddb module isn't included anymore)

files:
  Doc/faq/general.rst |  34 ---------------------------------
  1 files changed, 0 insertions(+), 34 deletions(-)


diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -469,38 +469,3 @@
 If you want to discuss Python's use in education, you may be interested in
 joining `the edu-sig mailing list
 <http://python.org/community/sigs/current/edu-sig>`_.
-
-
-Upgrading Python
-================
-
-What is this bsddb185 module my application keeps complaining about?
---------------------------------------------------------------------
-
-.. XXX remove this question?
-
-Starting with Python2.3, the distribution includes the `PyBSDDB package
-<http://pybsddb.sf.net/>` as a replacement for the old bsddb module.  It
-includes functions which provide backward compatibility at the API level, but
-requires a newer version of the underlying `Berkeley DB
-<http://www.sleepycat.com>`_ library.  Files created with the older bsddb module
-can't be opened directly using the new module.
-
-Using your old version of Python and a pair of scripts which are part of Python
-2.3 (db2pickle.py and pickle2db.py, in the Tools/scripts directory) you can
-convert your old database files to the new format.  Using your old Python
-version, run the db2pickle.py script to convert it to a pickle, e.g.::
-
-   python2.2 <pathto>/db2pickley.py database.db database.pck
-
-Rename your database file::
-
-   mv database.db olddatabase.db
-
-Now convert the pickle file to a new format database::
-
-   python <pathto>/pickle2db.py database.db database.pck
-
-The precise commands you use will vary depending on the particulars of your
-installation.  For full details about operation of these two scripts check the
-doc string at the start of each one.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:45:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:45:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_compatibili?=
	=?utf8?q?ty_statement=2E?=
Message-ID: <E1RWxOt-0005SV-AD@dinsdale.python.org>

http://hg.python.org/cpython/rev/01292dd6dadd
changeset:   73841:01292dd6dadd
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:39:13 2011 +0100
summary:
  Fix compatibility statement.

files:
  Doc/faq/general.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -19,7 +19,7 @@
 window systems, and is extensible in C or C++.  It is also usable as an
 extension language for applications that need a programmable interface.
 Finally, Python is portable: it runs on many Unix variants, on the Mac, and on
-PCs under MS-DOS, Windows, Windows NT, and OS/2.
+Windows 2000 and later.
 
 To find out more, start with :ref:`tutorial-index`.  The `Beginner's Guide to
 Python <http://wiki.python.org/moin/BeginnersGuide>`_ links to other

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 22:45:40 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 22:45:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_from_3=2E2?=
Message-ID: <E1RWxOu-0005SZ-0S@dinsdale.python.org>

http://hg.python.org/cpython/rev/abd2adbfceb7
changeset:   73842:abd2adbfceb7
parent:      73837:3d96c2dbec13
parent:      73841:01292dd6dadd
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:40:23 2011 +0100
summary:
  Merge from 3.2

files:
  Doc/faq/extending.rst |  28 +++++++++++++----------
  Doc/faq/general.rst   |  36 +------------------------------
  2 files changed, 17 insertions(+), 47 deletions(-)


diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
--- a/Doc/faq/extending.rst
+++ b/Doc/faq/extending.rst
@@ -99,12 +99,7 @@
 How do I use Py_BuildValue() to create a tuple of arbitrary length?
 -------------------------------------------------------------------
 
-You can't.  Use ``t = PyTuple_New(n)`` instead, and fill it with objects using
-``PyTuple_SetItem(t, i, o)`` -- note that this "eats" a reference count of
-``o``, so you have to :c:func:`Py_INCREF` it.  Lists have similar functions
-``PyList_New(n)`` and ``PyList_SetItem(l, i, o)``.  Note that you *must* set all
-the tuple items to some value before you pass the tuple to Python code --
-``PyTuple_New(n)`` initializes them to NULL, which isn't a valid Python value.
+You can't.  Use :c:func:`PyTuple_Pack` instead.
 
 
 How do I call an object's method from C?
@@ -147,21 +142,30 @@
 just allow the standard traceback mechanism to work. Then, the output will go
 wherever your ``write()`` method sends it.
 
-The easiest way to do this is to use the StringIO class in the standard library.
+The easiest way to do this is to use the :class:`io.StringIO` class::
 
-Sample code and use for catching stdout:
+   >>> import io, sys
+   >>> sys.stdout = io.StringIO()
+   >>> print('foo')
+   >>> print('hello world!')
+   >>> sys.stderr.write(sys.stdout.getvalue())
+   foo
+   hello world!
 
-   >>> class StdoutCatcher:
+A custom object to do the same would look like this::
+
+   >>> import io, sys
+   >>> class StdoutCatcher(io.TextIOBase):
    ...     def __init__(self):
-   ...         self.data = ''
+   ...         self.data = []
    ...     def write(self, stuff):
-   ...         self.data = self.data + stuff
+   ...         self.data.append(stuff)
    ...
    >>> import sys
    >>> sys.stdout = StdoutCatcher()
    >>> print('foo')
    >>> print('hello world!')
-   >>> sys.stderr.write(sys.stdout.data)
+   >>> sys.stderr.write(''.join(sys.stdout.data))
    foo
    hello world!
 
diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -19,7 +19,7 @@
 window systems, and is extensible in C or C++.  It is also usable as an
 extension language for applications that need a programmable interface.
 Finally, Python is portable: it runs on many Unix variants, on the Mac, and on
-PCs under MS-DOS, Windows, Windows NT, and OS/2.
+Windows 2000 and later.
 
 To find out more, start with :ref:`tutorial-index`.  The `Beginner's Guide to
 Python <http://wiki.python.org/moin/BeginnersGuide>`_ links to other
@@ -469,38 +469,3 @@
 If you want to discuss Python's use in education, you may be interested in
 joining `the edu-sig mailing list
 <http://python.org/community/sigs/current/edu-sig>`_.
-
-
-Upgrading Python
-================
-
-What is this bsddb185 module my application keeps complaining about?
---------------------------------------------------------------------
-
-.. XXX remove this question?
-
-Starting with Python2.3, the distribution includes the `PyBSDDB package
-<http://pybsddb.sf.net/>` as a replacement for the old bsddb module.  It
-includes functions which provide backward compatibility at the API level, but
-requires a newer version of the underlying `Berkeley DB
-<http://www.sleepycat.com>`_ library.  Files created with the older bsddb module
-can't be opened directly using the new module.
-
-Using your old version of Python and a pair of scripts which are part of Python
-2.3 (db2pickle.py and pickle2db.py, in the Tools/scripts directory) you can
-convert your old database files to the new format.  Using your old Python
-version, run the db2pickle.py script to convert it to a pickle, e.g.::
-
-   python2.2 <pathto>/db2pickley.py database.db database.pck
-
-Rename your database file::
-
-   mv database.db olddatabase.db
-
-Now convert the pickle file to a new format database::
-
-   python <pathto>/pickle2db.py database.db database.pck
-
-The precise commands you use will vary depending on the particulars of your
-installation.  For full details about operation of these two scripts check the
-doc string at the start of each one.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 23:15:25 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 23:15:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Reformulate_ans?=
 =?utf8?q?wer=2C_and_remove_off-topic_discussion_of_bytecode_in_a_question?=
Message-ID: <E1RWxrh-0007R7-D3@dinsdale.python.org>

http://hg.python.org/cpython/rev/00fdd8b1e17b
changeset:   73843:00fdd8b1e17b
branch:      3.2
parent:      73841:01292dd6dadd
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 22:56:02 2011 +0100
summary:
  Reformulate answer, and remove off-topic discussion of bytecode in a question about compiling to C.

files:
  Doc/faq/design.rst |  49 +++++++++++----------------------
  1 files changed, 17 insertions(+), 32 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -380,11 +380,24 @@
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-Not easily.  Python's high level data types, dynamic typing of objects and
+Practical answer:
+
+`Cython <http://cython.org/>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
+compile a modified version of Python with optional annotations into C
+extensions.  `Weave <http://www.scipy.org/Weave>`_ makes it easy to
+intermingle Python and C code in various ways to increase performance.
+`Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python
+into C++ code, aiming to support the full Python language.
+
+Theoretical answer:
+
+   .. XXX not sure what to make of this
+
+Not trivially.  Python's high level data types, dynamic typing of objects and
 run-time invocation of the interpreter (using :func:`eval` or :func:`exec`)
-together mean that a "compiled" Python program would probably consist mostly of
-calls into the Python run-time system, even for seemingly simple operations like
-``x+1``.
+together mean that a na?vely "compiled" Python program would probably consist
+mostly of calls into the Python run-time system, even for seemingly simple
+operations like ``x+1``.
 
 Several projects described in the Python newsgroup or at past `Python
 conferences <http://python.org/community/workshops/>`_ have shown that this
@@ -395,34 +408,6 @@
 from the `1997 Python conference
 <http://python.org/workshops/1997-10/proceedings/>`_ for more information.)
 
-Internally, Python source code is always translated into a bytecode
-representation, and this bytecode is then executed by the Python virtual
-machine.  In order to avoid the overhead of repeatedly parsing and translating
-modules that rarely change, this byte code is written into a file whose name
-ends in ".pyc" whenever a module is parsed.  When the corresponding .py file is
-changed, it is parsed and translated again and the .pyc file is rewritten.
-
-There is no performance difference once the .pyc file has been loaded, as the
-bytecode read from the .pyc file is exactly the same as the bytecode created by
-direct translation.  The only difference is that loading code from a .pyc file
-is faster than parsing and translating a .py file, so the presence of
-precompiled .pyc files improves the start-up time of Python scripts.  If
-desired, the Lib/compileall.py module can be used to create valid .pyc files for
-a given set of modules.
-
-Note that the main script executed by Python, even if its filename ends in .py,
-is not compiled to a .pyc file.  It is compiled to bytecode, but the bytecode is
-not saved to a file.  Usually main scripts are quite short, so this doesn't cost
-much speed.
-
-.. XXX check which of these projects are still alive
-
-There are also several programs which make it easier to intermingle Python and C
-code in various ways to increase performance.  See, for example, `Cython
-<http://cython.org/>`_, `Pyrex
-<http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ and `Weave
-<http://www.scipy.org/Weave>`_.
-
 
 How does Python manage memory?
 ------------------------------

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 23:15:26 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 23:15:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Break_down_and_?=
 =?utf8?q?refine_memory_management_question?=
Message-ID: <E1RWxri-0007RH-3G@dinsdale.python.org>

http://hg.python.org/cpython/rev/29fc3c23c73d
changeset:   73844:29fc3c23c73d
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 23:06:50 2011 +0100
summary:
  Break down and refine memory management question

files:
  Doc/faq/design.rst |  74 +++++++++++++++------------------
  1 files changed, 34 insertions(+), 40 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -413,66 +413,59 @@
 ------------------------------
 
 The details of Python memory management depend on the implementation.  The
-standard C implementation of Python uses reference counting to detect
-inaccessible objects, and another mechanism to collect reference cycles,
+standard implementation of Python, :term:`CPython`, uses reference counting to
+detect inaccessible objects, and another mechanism to collect reference cycles,
 periodically executing a cycle detection algorithm which looks for inaccessible
 cycles and deletes the objects involved. The :mod:`gc` module provides functions
 to perform a garbage collection, obtain debugging statistics, and tune the
 collector's parameters.
 
-Jython relies on the Java runtime so the JVM's garbage collector is used.  This
-difference can cause some subtle porting problems if your Python code depends on
-the behavior of the reference counting implementation.
+Other implementations (such as `Jython <http://www.jython.org>`_ or
+`PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism
+such as a full-blown garbage collector.  This difference can cause some
+subtle porting problems if your Python code depends on the behavior of the
+reference counting implementation.
 
-.. XXX relevant for Python 3?
+In some Python implementations, the following code (which is fine in CPython)
+will probably run out of file descriptors::
 
-   Sometimes objects get stuck in traceback temporarily and hence are not
-   deallocated when you might expect.  Clear the traceback with::
+   for file in very_long_list_of_files:
+       f = open(file)
+       c = f.read(1)
 
-     import sys
-     sys.last_traceback = None
+Indeed, using CPython's reference counting and destructor scheme, each new
+assignment to *f* closes the previous file.  With a traditional GC, however,
+those file objects will only get collected (and closed) at varying and possibly
+long intervals.
 
-   Tracebacks are used for reporting errors, implementing debuggers and related
-   things.  They contain a portion of the program state extracted during the
-   handling of an exception (usually the most recent exception).
+If you want to write code that will work with any Python implementation,
+you should explicitly close the file or use the :keyword:`with` statement;
+this will work regardless of memory management scheme::
 
-In the absence of circularities, Python programs do not need to manage memory
-explicitly.
+   for file in very_long_list_of_files:
+       with open(file) as f:
+           c = f.read(1)
 
-Why doesn't Python use a more traditional garbage collection scheme?  For one
-thing, this is not a C standard feature and hence it's not portable.  (Yes, we
-know about the Boehm GC library.  It has bits of assembler code for *most*
-common platforms, not for all of them, and although it is mostly transparent, it
-isn't completely transparent; patches are required to get Python to work with
-it.)
+
+Why doesn't CPython use a more traditional garbage collection scheme?
+---------------------------------------------------------------------
+
+For one thing, this is not a C standard feature and hence it's not portable.
+(Yes, we know about the Boehm GC library.  It has bits of assembler code for
+*most* common platforms, not for all of them, and although it is mostly
+transparent, it isn't completely transparent; patches are required to get
+Python to work with it.)
 
 Traditional GC also becomes a problem when Python is embedded into other
 applications.  While in a standalone Python it's fine to replace the standard
 malloc() and free() with versions provided by the GC library, an application
 embedding Python may want to have its *own* substitute for malloc() and free(),
-and may not want Python's.  Right now, Python works with anything that
+and may not want Python's.  Right now, CPython works with anything that
 implements malloc() and free() properly.
 
-In Jython, the following code (which is fine in CPython) will probably run out
-of file descriptors long before it runs out of memory::
 
-   for file in very_long_list_of_files:
-       f = open(file)
-       c = f.read(1)
-
-Using the current reference counting and destructor scheme, each new assignment
-to f closes the previous file.  Using GC, this is not guaranteed.  If you want
-to write code that will work with any Python implementation, you should
-explicitly close the file or use the :keyword:`with` statement; this will work
-regardless of GC::
-
-   for file in very_long_list_of_files:
-       with open(file) as f:
-           c = f.read(1)
-
-
-Why isn't all memory freed when Python exits?
----------------------------------------------
+Why isn't all memory freed when CPython exits?
+----------------------------------------------
 
 Objects referenced from the global namespaces of Python modules are not always
 deallocated when Python exits.  This may happen if there are circular

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 23:15:26 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 23:15:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Streamline_ment?=
	=?utf8?q?ion_of_sorted=28=29?=
Message-ID: <E1RWxri-0007RL-Pe@dinsdale.python.org>

http://hg.python.org/cpython/rev/bf0c5cd303e5
changeset:   73845:bf0c5cd303e5
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 23:08:57 2011 +0100
summary:
  Streamline mention of sorted()

files:
  Doc/faq/design.rst |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -625,10 +625,10 @@
 you won't be fooled into accidentally overwriting a list when you need a sorted
 copy but also need to keep the unsorted version around.
 
-In Python 2.4 a new built-in function -- :func:`sorted` -- has been added.
-This function creates a new list from a provided iterable, sorts it and returns
-it.  For example, here's how to iterate over the keys of a dictionary in sorted
-order::
+If you want to return a new list, use the built-in :func:`sorted` function
+instead.  This function creates a new list from a provided iterable, sorts
+it and returns it.  For example, here's how to iterate over the keys of a
+dictionary in sorted order::
 
    for key in sorted(mydict):
        ... # do whatever with mydict[key]...

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec  3 23:15:27 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 03 Dec 2011 23:15:27 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_from_3=2E2?=
Message-ID: <E1RWxrj-0007RP-GC@dinsdale.python.org>

http://hg.python.org/cpython/rev/f09477693434
changeset:   73846:f09477693434
parent:      73842:abd2adbfceb7
parent:      73845:bf0c5cd303e5
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Dec 03 23:10:12 2011 +0100
summary:
  Merge from 3.2

files:
  Doc/faq/design.rst |  131 +++++++++++++-------------------
  1 files changed, 55 insertions(+), 76 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -380,11 +380,24 @@
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-Not easily.  Python's high level data types, dynamic typing of objects and
+Practical answer:
+
+`Cython <http://cython.org/>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
+compile a modified version of Python with optional annotations into C
+extensions.  `Weave <http://www.scipy.org/Weave>`_ makes it easy to
+intermingle Python and C code in various ways to increase performance.
+`Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python
+into C++ code, aiming to support the full Python language.
+
+Theoretical answer:
+
+   .. XXX not sure what to make of this
+
+Not trivially.  Python's high level data types, dynamic typing of objects and
 run-time invocation of the interpreter (using :func:`eval` or :func:`exec`)
-together mean that a "compiled" Python program would probably consist mostly of
-calls into the Python run-time system, even for seemingly simple operations like
-``x+1``.
+together mean that a na?vely "compiled" Python program would probably consist
+mostly of calls into the Python run-time system, even for seemingly simple
+operations like ``x+1``.
 
 Several projects described in the Python newsgroup or at past `Python
 conferences <http://python.org/community/workshops/>`_ have shown that this
@@ -395,99 +408,64 @@
 from the `1997 Python conference
 <http://python.org/workshops/1997-10/proceedings/>`_ for more information.)
 
-Internally, Python source code is always translated into a bytecode
-representation, and this bytecode is then executed by the Python virtual
-machine.  In order to avoid the overhead of repeatedly parsing and translating
-modules that rarely change, this byte code is written into a file whose name
-ends in ".pyc" whenever a module is parsed.  When the corresponding .py file is
-changed, it is parsed and translated again and the .pyc file is rewritten.
-
-There is no performance difference once the .pyc file has been loaded, as the
-bytecode read from the .pyc file is exactly the same as the bytecode created by
-direct translation.  The only difference is that loading code from a .pyc file
-is faster than parsing and translating a .py file, so the presence of
-precompiled .pyc files improves the start-up time of Python scripts.  If
-desired, the Lib/compileall.py module can be used to create valid .pyc files for
-a given set of modules.
-
-Note that the main script executed by Python, even if its filename ends in .py,
-is not compiled to a .pyc file.  It is compiled to bytecode, but the bytecode is
-not saved to a file.  Usually main scripts are quite short, so this doesn't cost
-much speed.
-
-.. XXX check which of these projects are still alive
-
-There are also several programs which make it easier to intermingle Python and C
-code in various ways to increase performance.  See, for example, `Cython
-<http://cython.org/>`_, `Pyrex
-<http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ and `Weave
-<http://www.scipy.org/Weave>`_.
-
 
 How does Python manage memory?
 ------------------------------
 
 The details of Python memory management depend on the implementation.  The
-standard C implementation of Python uses reference counting to detect
-inaccessible objects, and another mechanism to collect reference cycles,
+standard implementation of Python, :term:`CPython`, uses reference counting to
+detect inaccessible objects, and another mechanism to collect reference cycles,
 periodically executing a cycle detection algorithm which looks for inaccessible
 cycles and deletes the objects involved. The :mod:`gc` module provides functions
 to perform a garbage collection, obtain debugging statistics, and tune the
 collector's parameters.
 
-Jython relies on the Java runtime so the JVM's garbage collector is used.  This
-difference can cause some subtle porting problems if your Python code depends on
-the behavior of the reference counting implementation.
+Other implementations (such as `Jython <http://www.jython.org>`_ or
+`PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism
+such as a full-blown garbage collector.  This difference can cause some
+subtle porting problems if your Python code depends on the behavior of the
+reference counting implementation.
 
-.. XXX relevant for Python 3?
+In some Python implementations, the following code (which is fine in CPython)
+will probably run out of file descriptors::
 
-   Sometimes objects get stuck in traceback temporarily and hence are not
-   deallocated when you might expect.  Clear the traceback with::
+   for file in very_long_list_of_files:
+       f = open(file)
+       c = f.read(1)
 
-     import sys
-     sys.last_traceback = None
+Indeed, using CPython's reference counting and destructor scheme, each new
+assignment to *f* closes the previous file.  With a traditional GC, however,
+those file objects will only get collected (and closed) at varying and possibly
+long intervals.
 
-   Tracebacks are used for reporting errors, implementing debuggers and related
-   things.  They contain a portion of the program state extracted during the
-   handling of an exception (usually the most recent exception).
+If you want to write code that will work with any Python implementation,
+you should explicitly close the file or use the :keyword:`with` statement;
+this will work regardless of memory management scheme::
 
-In the absence of circularities, Python programs do not need to manage memory
-explicitly.
+   for file in very_long_list_of_files:
+       with open(file) as f:
+           c = f.read(1)
 
-Why doesn't Python use a more traditional garbage collection scheme?  For one
-thing, this is not a C standard feature and hence it's not portable.  (Yes, we
-know about the Boehm GC library.  It has bits of assembler code for *most*
-common platforms, not for all of them, and although it is mostly transparent, it
-isn't completely transparent; patches are required to get Python to work with
-it.)
+
+Why doesn't CPython use a more traditional garbage collection scheme?
+---------------------------------------------------------------------
+
+For one thing, this is not a C standard feature and hence it's not portable.
+(Yes, we know about the Boehm GC library.  It has bits of assembler code for
+*most* common platforms, not for all of them, and although it is mostly
+transparent, it isn't completely transparent; patches are required to get
+Python to work with it.)
 
 Traditional GC also becomes a problem when Python is embedded into other
 applications.  While in a standalone Python it's fine to replace the standard
 malloc() and free() with versions provided by the GC library, an application
 embedding Python may want to have its *own* substitute for malloc() and free(),
-and may not want Python's.  Right now, Python works with anything that
+and may not want Python's.  Right now, CPython works with anything that
 implements malloc() and free() properly.
 
-In Jython, the following code (which is fine in CPython) will probably run out
-of file descriptors long before it runs out of memory::
 
-   for file in very_long_list_of_files:
-       f = open(file)
-       c = f.read(1)
-
-Using the current reference counting and destructor scheme, each new assignment
-to f closes the previous file.  Using GC, this is not guaranteed.  If you want
-to write code that will work with any Python implementation, you should
-explicitly close the file or use the :keyword:`with` statement; this will work
-regardless of GC::
-
-   for file in very_long_list_of_files:
-       with open(file) as f:
-           c = f.read(1)
-
-
-Why isn't all memory freed when Python exits?
----------------------------------------------
+Why isn't all memory freed when CPython exits?
+----------------------------------------------
 
 Objects referenced from the global namespaces of Python modules are not always
 deallocated when Python exits.  This may happen if there are circular
@@ -647,10 +625,10 @@
 you won't be fooled into accidentally overwriting a list when you need a sorted
 copy but also need to keep the unsorted version around.
 
-In Python 2.4 a new built-in function -- :func:`sorted` -- has been added.
-This function creates a new list from a provided iterable, sorts it and returns
-it.  For example, here's how to iterate over the keys of a dictionary in sorted
-order::
+If you want to return a new list, use the built-in :func:`sorted` function
+instead.  This function creates a new list from a provided iterable, sorts
+it and returns it.  For example, here's how to iterate over the keys of a
+dictionary in sorted order::
 
    for key in sorted(mydict):
        ... # do whatever with mydict[key]...

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec  4 03:05:11 2011
From: python-checkins at python.org (nick.coghlan)
Date: Sun, 04 Dec 2011 03:05:11 +0100
Subject: [Python-checkins] =?utf8?q?peps=3A_PEP_406_=28ImportEngine_API=29?=
 =?utf8?q?=3A_defer_this_until_3=2E4_at_the_earliest=2C_don=27t_propose?=
Message-ID: <E1RX1S3-0006M2-Gy@dinsdale.python.org>

http://hg.python.org/peps/rev/e2ccffa4d3b5
changeset:   4009:e2ccffa4d3b5
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Sun Dec 04 12:05:01 2011 +1000
summary:
  PEP 406 (ImportEngine API): defer this until 3.4 at the earliest, don't propose altering the PEP 302 APIs

files:
  pep-0406.txt |  127 +++++++++++++++++++++++---------------
  1 files changed, 77 insertions(+), 50 deletions(-)


diff --git a/pep-0406.txt b/pep-0406.txt
--- a/pep-0406.txt
+++ b/pep-0406.txt
@@ -3,11 +3,12 @@
 Version: $Revision$
 Last-Modified: $Date$
 Author: Nick Coghlan <ncoghlan at gmail.com>, Greg Slodkowicz <jergosh at gmail.com>
-Status: Draft
+Status: Deferred
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 4-Jul-2011
-Post-History: 31-Jul-2011, 13-Nov-2011
+Python-Version: 3.4
+Post-History: 31-Jul-2011, 13-Nov-2011, 4-Dec-2011
 
 Abstract
 ========
@@ -18,17 +19,27 @@
 an alternative to completely replacing the built-in implementation of the
 import statement, by overriding the ``__import__()`` function. To work with
 the builtin import functionality and importing via import engine objects,
-module importers and loaders must accept an optional ``engine`` parameter. In
-that sense, this PEP constitutes a revision of finder and loader interfaces
-described in PEP 302 [1]_. However, the standard import process will not
-supply the additional argument, so this proposal remains fully backwards
-compatible.
+this PEP proposes a context management based approach to temporarily replacing
+the global import state.
 
 The PEP also proposes inclusion of a ``GlobalImportEngine`` subclass and a
 globally accessible instance of that class, which "writes through" to the
-process global state and invokes importers and loaders without the additional
-``engine`` argument. This provides a backwards compatible bridge between the
-proposed encapsulated API and the legacy process global state.
+process global state. This provides a backwards compatible bridge between the
+proposed encapsulated API and the legacy process global state, and allows
+straightforward support for related state updates (e.g. selectively
+invalidating path cache entries when ``sys.path`` is modified).
+
+
+PEP Deferral
+============
+
+The import system is already seeing substantial changes in Python 3.3, to
+natively handle packages split across multiple directories (PEP 382) and
+(potentially) to make the import semantics in the main module better match
+those in other modules (PEP 395).
+
+Accordingly, the proposal in this PEP will not be seriously considered until
+Python 3.4 at the earliest.
 
 
 Rationale
@@ -58,6 +69,10 @@
 *additional* process global state, in order to correctly update package paths
 as ``sys.path`` is modified.
 
+Finally, providing a coherent object for all this state makes it feasible to
+also provide context management features that allow the import state to be
+temporarily substituted.
+
 
 Proposal
 ========
@@ -68,11 +83,10 @@
 desired and also an ``import_module()`` method, equivalent to
 ``importlib.import_module()`` [3]_.
 
-Since the new style finders and loaders should also have the option to
-modify the global import state, we introduce a ``GlobalImportState``
-class with an interface identical to ``ImportEngine`` but taking
-advantage of the global state. This can be easily implemented using
-class properties.
+Since there are global import state invariants that are assumed and should be
+maintained, we introduce a ``GlobalImportState`` class with an interface
+identical to ``ImportEngine`` but directly accessing the current global import
+state. This can be easily implemented using class properties.
 
 
 Specification
@@ -121,6 +135,14 @@
     methods like ``ImportEngine`` but writes through to the global state
     in ``sys``.
 
+To support various namespace package mechanisms, when ``sys.path`` is altered,
+tools like ``pkgutil.extend_path`` should be used to also modify other parts
+of the import state (in this case, package ``__path__`` attributes). The path
+importer cache should also be invalidated when a variety of changes are made.
+
+The ``ImportEngine`` API will provide convenience methods that automatically
+make related import state updates as part of a single operation.
+
 
 Global variables
 ~~~~~~~~~~~~~~~~
@@ -133,24 +155,26 @@
     a copy of the process global import state.
 
 
-Necessary changes to finder/loader interfaces:
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+No changes to finder/loader interfaces
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-``find_module (cls, fullname, path=None, engine=None)``
+Rather than attempting to update the PEP 302 APIs to accept additional state,
+this PEP proposes that ``ImportEngine`` support the content management
+protocol (similar to the context substitution mechanisms in the ``decimal``
+module).
 
-``load_module (cls, fullname, path=None, engine=None)``
+The context management mechanism for ``ImportEngine`` would:
 
-The only difference between engine compatible and PEP 302 compatible
-finders/loaders is the presence of an additional ``engine`` parameter.
-This is intended to specify an ImportEngine instance or subclass thereof. 
-This parameter is optional so that engine compatible finders and
-loaders can be made backwards compatible with PEP 302 calling conventions by
-falling back on ``engine.sysengine`` with the following simple pattern::
+* On entry:
+  * Acquire the import lock
+  * Substitute the global import state with the import engine's own state
+* On exit:
+  * Restore the previous global import state
+  * Release the import lock
 
-    def find_module(cls, fullname, path=None, engine=None):
-        if not engine:
-            engine = importlib.engine.sysengine
-        ...
+The precise API for this is TBD (but will probably use a distinct context
+management object, along the lines of that created by
+``decimal.localcontext``).
 
 
 Open Issues
@@ -185,35 +209,38 @@
 cache - it's only loading them directly which causes problems.
 
 
-Nested imports
-~~~~~~~~~~~~~~
+Scope of substitution
+~~~~~~~~~~~~~~~~~~~~~
 
-The reference implementation currently applies only to the outermost import.
-Any imports by the module being imported will be handled using the standard
-import machinery.
+Related to the previous open issue is the question of what state to substitute
+when using the context management API. It is currently the case that replacing
+``sys.modules`` can be unreliable due to cached references and there's the
+underlying fact that having independent copies of some modules is simply
+impossible due to platform limitations.
 
-One way to handle this is to place the burden on the implementation of module
-loaders to set ``module.__dict__["__import__"] = engine.__import__`` before
-running the module's code. The ``importlib`` design facilities this by
-allowing the change to be made in one place (``_LoaderBasics._load_module``).
+As part of this PEP, it will be necessary to document explicitly:
 
-
-Scope of API updates
-~~~~~~~~~~~~~~~~~~~~
-
-The reference implementation focuses on finding and loading modules. There
-may be other PEP 302 APIs that should also be updated to accept an optional
-``engine`` parameter.
+* Which parts of the global import state can be substituted (and declare code
+  which caches references to that state without dealing with the substitution
+  case buggy)
+* Which parts must be modified in-place (and hence are not substituted by the
+  ``ImportEngine`` context management API, or otherwise scoped to
+  ``ImportEngine`` instances)
 
 
 Reference Implementation
 ========================
 
-A reference implementation [4]_ based on Brett Cannon's importlib has been
-developed by Greg Slodkowicz as part of the 2011 Google Summer of Code. Note
-that the current implementation avoids modifying existing code, and hence
-duplicates a lot of things unnecessarily. An actual implementation would just
-modify any such affected code in place.
+A reference implementation [4]_ for an earlier draft of this PEP, based on
+Brett Cannon's importlib has been developed by Greg Slodkowicz as part of the
+2011 Google Summer of Code. Note that the current implementation avoids
+modifying existing code, and hence duplicates a lot of things unnecessarily.
+An actual implementation would just modify any such affected code in place.
+
+That earlier draft of the PEP proposed change the PEP 302 APIs to support passing
+in an optional engine instance. This had the (serious) downside of not correctly
+affecting further imports from the imported module, hence the change to the
+context management based proposal for substituting the global state.
 
 
 References

-- 
Repository URL: http://hg.python.org/peps

From ncoghlan at gmail.com  Sun Dec  4 05:11:58 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Dec 2011 14:11:58 +1000
Subject: [Python-checkins] cpython (3.2): Issue #13211: Add .reason
 attribute to HTTPError to implement parent class
In-Reply-To: <E1RWqr1-0000qP-2f@dinsdale.python.org>
References: <E1RWqr1-0000qP-2f@dinsdale.python.org>
Message-ID: <CADiSq7ckdv32_JEdLxLzq6fRhx3cFs2eTpR3yAtYYR=fPz9-yA@mail.gmail.com>

On Sun, Dec 4, 2011 at 12:46 AM, jason.coombs
<python-checkins at python.org> wrote:
> +def test_HTTPError_interface():
> + ? ?"""
> + ? ?Issue 13211 reveals that HTTPError didn't implement the URLError
> + ? ?interface even though HTTPError is a subclass of URLError.
> +
> + ? ?>>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None)
> + ? ?>>> assert hasattr(err, 'reason')
> + ? ?>>> err.reason
> + ? ?'something bad happened'
> + ? ?"""
> +

Did you re-run the test suite after forward-porting to 3.3? I'm
consistently getting failures:

$ ./python -m test test_urllib2
[1/1] test_urllib2
**********************************************************************
File "/home/ncoghlan/devel/py3k/Lib/test/test_urllib2.py", line 1457,
in test.test_urllib2.test_HTTPError_interface
Failed example:
    err = urllib.error.HTTPError(msg='something bad happened',
url=None, code=None, hdrs=None, fp=None)
Exception raised:
    Traceback (most recent call last):
      File "/home/ncoghlan/devel/py3k/Lib/doctest.py", line 1253, in __run
        compileflags, 1), test.globs)
      File "<doctest test.test_urllib2.test_HTTPError_interface[0]>",
line 1, in <module>
        err = urllib.error.HTTPError(msg='something bad happened',
url=None, code=None, hdrs=None, fp=None)
    TypeError: HTTPError does not take keyword arguments
**********************************************************************
File "/home/ncoghlan/devel/py3k/Lib/test/test_urllib2.py", line 1458,
in test.test_urllib2.test_HTTPError_interface
Failed example:
    assert hasattr(err, 'reason')
Exception raised:
    Traceback (most recent call last):
      File "/home/ncoghlan/devel/py3k/Lib/doctest.py", line 1253, in __run
        compileflags, 1), test.globs)
      File "<doctest test.test_urllib2.test_HTTPError_interface[1]>",
line 1, in <module>
        assert hasattr(err, 'reason')
    NameError: name 'err' is not defined
**********************************************************************
File "/home/ncoghlan/devel/py3k/Lib/test/test_urllib2.py", line 1459,
in test.test_urllib2.test_HTTPError_interface
Failed example:
    err.reason
Exception raised:
    Traceback (most recent call last):
      File "/home/ncoghlan/devel/py3k/Lib/doctest.py", line 1253, in __run
        compileflags, 1), test.globs)
      File "<doctest test.test_urllib2.test_HTTPError_interface[2]>",
line 1, in <module>
        err.reason
    NameError: name 'err' is not defined
**********************************************************************
1 items had failures:
   3 of   3 in test.test_urllib2.test_HTTPError_interface
***Test Failed*** 3 failures.
test test_urllib2 failed -- 3 of 65 doctests failed
1 test failed:
    test_urllib2
[142313 refs]

Now, this failure is quite possibly due to a flaw in the PEP 3151
implementation (see http://bugs.python.org/issue12555), but picking up
this kind of thing is the reason we say to always run the tests before
committing, even for a simple merge.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From python-checkins at python.org  Sun Dec  4 05:20:40 2011
From: python-checkins at python.org (jason.coombs)
Date: Sun, 04 Dec 2011 05:20:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Pass_positional_arguments_-?=
 =?utf8?q?_HTTPError_is_not_accepting_keyword_arguments=2E?=
Message-ID: <E1RX3ZA-00069U-KK@dinsdale.python.org>

http://hg.python.org/cpython/rev/a3ddee916808
changeset:   73847:a3ddee916808
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sat Dec 03 23:18:11 2011 -0500
summary:
  Pass positional arguments - HTTPError is not accepting keyword arguments. Reference #13211 and #12555.

files:
  Lib/test/test_urllib2.py |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -1454,7 +1454,9 @@
     Issue 13211 reveals that HTTPError didn't implement the URLError
     interface even though HTTPError is a subclass of URLError.
 
-    >>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None)
+    >>> msg = 'something bad happened'
+    >>> url = code = hdrs = fp = None
+    >>> err = urllib.error.HTTPError(msg, url, code, hdrs, fp)
     >>> assert hasattr(err, 'reason')
     >>> err.reason
     'something bad happened'

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Dec  4 05:38:45 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 04 Dec 2011 05:38:45 +0100
Subject: [Python-checkins] Daily reference leaks (f09477693434): sum=-417
Message-ID: <E1RX3qf-0000Ey-Ag@ap.vmr.nerim.net>

results for f09477693434 on branch "default"
--------------------------------------------

test_urllib2net leaked [1405, -1822, 0] references, sum=-417


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogF7_xgV', '-x']

From python-checkins at python.org  Sun Dec  4 11:51:38 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 04 Dec 2011 11:51:38 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogRml4IHR5cG8u?=
Message-ID: <E1RX9fW-0003L0-46@dinsdale.python.org>

http://hg.python.org/cpython/rev/0436ef8be253
changeset:   73848:0436ef8be253
branch:      3.2
parent:      73845:bf0c5cd303e5
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 04 11:51:21 2011 +0100
summary:
  Fix typo.

files:
  Doc/tools/sphinxext/static/copybutton.js |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/tools/sphinxext/static/copybutton.js b/Doc/tools/sphinxext/static/copybutton.js
--- a/Doc/tools/sphinxext/static/copybutton.js
+++ b/Doc/tools/sphinxext/static/copybutton.js
@@ -8,8 +8,8 @@
 
     // get the styles from the current theme
     pre.parent().parent().css('position', 'relative');
-    var hide_text = 'Hide the prompts and ouput';
-    var show_text = 'Show the prompts and ouput';
+    var hide_text = 'Hide the prompts and output';
+    var show_text = 'Show the prompts and output';
     var border_width = pre.css('border-top-width');
     var border_style = pre.css('border-top-style');
     var border_color = pre.css('border-top-color');

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec  4 11:51:38 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 04 Dec 2011 11:51:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_with_3=2E2=2E?=
Message-ID: <E1RX9fW-0003LA-S4@dinsdale.python.org>

http://hg.python.org/cpython/rev/2e4fd707201c
changeset:   73849:2e4fd707201c
parent:      73847:a3ddee916808
parent:      73848:0436ef8be253
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 04 11:51:33 2011 +0100
summary:
  Merge with 3.2.

files:
  Doc/tools/sphinxext/static/copybutton.js |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/tools/sphinxext/static/copybutton.js b/Doc/tools/sphinxext/static/copybutton.js
--- a/Doc/tools/sphinxext/static/copybutton.js
+++ b/Doc/tools/sphinxext/static/copybutton.js
@@ -8,8 +8,8 @@
 
     // get the styles from the current theme
     pre.parent().parent().css('position', 'relative');
-    var hide_text = 'Hide the prompts and ouput';
-    var show_text = 'Show the prompts and ouput';
+    var hide_text = 'Hide the prompts and output';
+    var show_text = 'Show the prompts and output';
     var border_width = pre.css('border-top-width');
     var border_style = pre.css('border-top-style');
     var border_color = pre.css('border-top-color');

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec  4 14:26:02 2011
From: python-checkins at python.org (jason.coombs)
Date: Sun, 04 Dec 2011 14:26:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Corrected_order_of_paramete?=
 =?utf8?q?rs_to_HTTPError_in_test=5Furllib2=2Epy=2E?=
Message-ID: <E1RXC4w-0004Jc-8n@dinsdale.python.org>

http://hg.python.org/cpython/rev/8fa1dc66de5d
changeset:   73850:8fa1dc66de5d
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Dec 04 08:14:18 2011 -0500
summary:
  Corrected order of parameters to HTTPError in test_urllib2.py.

files:
  Lib/test/test_urllib2.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -1456,7 +1456,7 @@
 
     >>> msg = 'something bad happened'
     >>> url = code = hdrs = fp = None
-    >>> err = urllib.error.HTTPError(msg, url, code, hdrs, fp)
+    >>> err = urllib.error.HTTPError(url, code, msg, hdrs, fp)
     >>> assert hasattr(err, 'reason')
     >>> err.reason
     'something bad happened'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 00:05:07 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 00:05:07 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTI3?=
 =?utf8?q?=3A_remove_mention_of_Python_megawidgets_and_Tkinter3000_WCK?=
Message-ID: <E1RXL7L-0004Nh-5p@dinsdale.python.org>

http://hg.python.org/cpython/rev/2111bf7e5bca
changeset:   73851:2111bf7e5bca
branch:      3.2
parent:      73848:0436ef8be253
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 04 23:56:30 2011 +0100
summary:
  Issue #13527: remove mention of Python megawidgets and Tkinter3000 WCK
from the doc. These two projects appear dead.

files:
  Doc/library/othergui.rst |  30 +--------------------------
  1 files changed, 2 insertions(+), 28 deletions(-)


diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -3,34 +3,8 @@
 Other Graphical User Interface Packages
 =======================================
 
-There are an number of extension widget sets to :mod:`tkinter`.
-
-.. seealso::
-
-   `Python megawidgets <http://pmw.sourceforge.net/>`_
-      is a toolkit for building high-level compound widgets in Python using the
-      :mod:`tkinter` package.  It consists of a set of base classes and a library of
-      flexible and extensible megawidgets built on this foundation. These megawidgets
-      include notebooks, comboboxes, selection widgets, paned widgets, scrolled
-      widgets, dialog windows, etc.  Also, with the Pmw.Blt interface to BLT, the
-      busy, graph, stripchart, tabset and vector commands are be available.
-
-      The initial ideas for Pmw were taken from the Tk ``itcl`` extensions ``[incr
-      Tk]`` by Michael McLennan and ``[incr Widgets]`` by Mark Ulferts. Several of the
-      megawidgets are direct translations from the itcl to Python. It offers most of
-      the range of widgets that ``[incr Widgets]`` does, and is almost as complete as
-      Tix, lacking however Tix's fast :class:`HList` widget for drawing trees.
-
-   `Tkinter3000 Widget Construction Kit (WCK) <http://tkinter.effbot.org/>`_
-      is a library that allows you to write new Tkinter widgets in pure Python.  The
-      WCK framework gives you full control over widget creation, configuration, screen
-      appearance, and event handling.  WCK widgets can be very fast and light-weight,
-      since they can operate directly on Python data structures, without having to
-      transfer data through the Tk/Tcl layer.
-
-
-The major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits that are
-also available for Python:
+Major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits are
+available for Python:
 
 .. seealso::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 00:05:08 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 00:05:08 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313527=3A_remove_mention_of_Python_megawidgets_and_T?=
 =?utf8?q?kinter3000_WCK?=
Message-ID: <E1RXL7M-0004O8-1H@dinsdale.python.org>

http://hg.python.org/cpython/rev/f0008683585c
changeset:   73852:f0008683585c
parent:      73850:8fa1dc66de5d
parent:      73851:2111bf7e5bca
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 04 23:57:55 2011 +0100
summary:
  Issue #13527: remove mention of Python megawidgets and Tkinter3000 WCK
from the doc. These two projects appear dead.

files:
  Doc/library/othergui.rst |  30 +--------------------------
  1 files changed, 2 insertions(+), 28 deletions(-)


diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -3,34 +3,8 @@
 Other Graphical User Interface Packages
 =======================================
 
-There are an number of extension widget sets to :mod:`tkinter`.
-
-.. seealso::
-
-   `Python megawidgets <http://pmw.sourceforge.net/>`_
-      is a toolkit for building high-level compound widgets in Python using the
-      :mod:`tkinter` package.  It consists of a set of base classes and a library of
-      flexible and extensible megawidgets built on this foundation. These megawidgets
-      include notebooks, comboboxes, selection widgets, paned widgets, scrolled
-      widgets, dialog windows, etc.  Also, with the Pmw.Blt interface to BLT, the
-      busy, graph, stripchart, tabset and vector commands are be available.
-
-      The initial ideas for Pmw were taken from the Tk ``itcl`` extensions ``[incr
-      Tk]`` by Michael McLennan and ``[incr Widgets]`` by Mark Ulferts. Several of the
-      megawidgets are direct translations from the itcl to Python. It offers most of
-      the range of widgets that ``[incr Widgets]`` does, and is almost as complete as
-      Tix, lacking however Tix's fast :class:`HList` widget for drawing trees.
-
-   `Tkinter3000 Widget Construction Kit (WCK) <http://tkinter.effbot.org/>`_
-      is a library that allows you to write new Tkinter widgets in pure Python.  The
-      WCK framework gives you full control over widget creation, configuration, screen
-      appearance, and event handling.  WCK widgets can be very fast and light-weight,
-      since they can operate directly on Python data structures, without having to
-      transfer data through the Tk/Tcl layer.
-
-
-The major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits that are
-also available for Python:
+Major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits are
+available for Python:
 
 .. seealso::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 00:05:08 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 00:05:08 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTI3?=
 =?utf8?q?=3A_remove_mention_of_Python_megawidgets_and_Tkinter3000_WCK?=
Message-ID: <E1RXL7M-0004OC-P2@dinsdale.python.org>

http://hg.python.org/cpython/rev/478b4e9551fa
changeset:   73853:478b4e9551fa
branch:      2.7
parent:      73826:fb8b6d310fb8
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 04 23:56:30 2011 +0100
summary:
  Issue #13527: remove mention of Python megawidgets and Tkinter3000 WCK
from the doc. These two projects appear dead.

files:
  Doc/library/othergui.rst |  30 +--------------------------
  1 files changed, 2 insertions(+), 28 deletions(-)


diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
--- a/Doc/library/othergui.rst
+++ b/Doc/library/othergui.rst
@@ -3,34 +3,8 @@
 Other Graphical User Interface Packages
 =======================================
 
-There are an number of extension widget sets to :mod:`Tkinter`.
-
-.. seealso::
-
-   `Python megawidgets <http://pmw.sourceforge.net/>`_
-      is a toolkit for building high-level compound widgets in Python using the
-      :mod:`Tkinter` module.  It consists of a set of base classes and a library of
-      flexible and extensible megawidgets built on this foundation. These megawidgets
-      include notebooks, comboboxes, selection widgets, paned widgets, scrolled
-      widgets, dialog windows, etc.  Also, with the Pmw.Blt interface to BLT, the
-      busy, graph, stripchart, tabset and vector commands are be available.
-
-      The initial ideas for Pmw were taken from the Tk ``itcl`` extensions ``[incr
-      Tk]`` by Michael McLennan and ``[incr Widgets]`` by Mark Ulferts. Several of the
-      megawidgets are direct translations from the itcl to Python. It offers most of
-      the range of widgets that ``[incr Widgets]`` does, and is almost as complete as
-      Tix, lacking however Tix's fast :class:`HList` widget for drawing trees.
-
-   `Tkinter3000 Widget Construction Kit (WCK) <http://tkinter.effbot.org/>`_
-      is a library that allows you to write new Tkinter widgets in pure Python.  The
-      WCK framework gives you full control over widget creation, configuration, screen
-      appearance, and event handling.  WCK widgets can be very fast and light-weight,
-      since they can operate directly on Python data structures, without having to
-      transfer data through the Tk/Tcl layer.
-
-
-The major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits that are
-also available for Python:
+Major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits are
+available for Python:
 
 .. seealso::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 00:47:08 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 00:47:08 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_obsolete?=
	=?utf8?q?_references_to_bsddb?=
Message-ID: <E1RXLm0-00072T-Gl@dinsdale.python.org>

http://hg.python.org/cpython/rev/1494e021cb9f
changeset:   73854:1494e021cb9f
branch:      3.2
parent:      73851:2111bf7e5bca
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 00:41:19 2011 +0100
summary:
  Remove obsolete references to bsddb

files:
  Doc/faq/library.rst |  44 ---------------------------------
  1 files changed, 0 insertions(+), 44 deletions(-)


diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst
--- a/Doc/faq/library.rst
+++ b/Doc/faq/library.rst
@@ -814,52 +814,6 @@
 general such as using gdbm with pickle/shelve.
 
 
-If my program crashes with a bsddb (or anydbm) database open, it gets corrupted. How come?
-------------------------------------------------------------------------------------------
-
-.. XXX move this FAQ entry elsewhere?
-
-.. note::
-
-   The bsddb module is now available as a standalone package `pybsddb
-   <http://www.jcea.es/programacion/pybsddb.htm>`_.
-
-Databases opened for write access with the bsddb module (and often by the anydbm
-module, since it will preferentially use bsddb) must explicitly be closed using
-the ``.close()`` method of the database.  The underlying library caches database
-contents which need to be converted to on-disk form and written.
-
-If you have initialized a new bsddb database but not written anything to it
-before the program crashes, you will often wind up with a zero-length file and
-encounter an exception the next time the file is opened.
-
-
-I tried to open Berkeley DB file, but bsddb produces bsddb.error: (22, 'Invalid argument'). Help! How can I restore my data?
-----------------------------------------------------------------------------------------------------------------------------
-
-.. XXX move this FAQ entry elsewhere?
-
-.. note::
-
-   The bsddb module is now available as a standalone package `pybsddb
-   <http://www.jcea.es/programacion/pybsddb.htm>`_.
-
-Don't panic! Your data is probably intact. The most frequent cause for the error
-is that you tried to open an earlier Berkeley DB file with a later version of
-the Berkeley DB library.
-
-Many Linux systems now have all three versions of Berkeley DB available.  If you
-are migrating from version 1 to a newer version use db_dump185 to dump a plain
-text version of the database.  If you are migrating from version 2 to version 3
-use db2_dump to create a plain text version of the database.  In either case,
-use db_load to create a new native database for the latest version installed on
-your computer.  If you have version 3 of Berkeley DB installed, you should be
-able to use db2_load to create a native version 2 database.
-
-You should move away from Berkeley DB version 1 files because the hash file code
-contains known bugs that can corrupt your data.
-
-
 Mathematics and Numerics
 ========================
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 00:47:09 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 00:47:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Remove_obsolete_references_to_bsddb?=
Message-ID: <E1RXLm1-00072d-8S@dinsdale.python.org>

http://hg.python.org/cpython/rev/e80a4e3b799b
changeset:   73855:e80a4e3b799b
parent:      73852:f0008683585c
parent:      73854:1494e021cb9f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 00:41:51 2011 +0100
summary:
  Remove obsolete references to bsddb

files:
  Doc/faq/library.rst |  44 ---------------------------------
  1 files changed, 0 insertions(+), 44 deletions(-)


diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst
--- a/Doc/faq/library.rst
+++ b/Doc/faq/library.rst
@@ -814,52 +814,6 @@
 general such as using gdbm with pickle/shelve.
 
 
-If my program crashes with a bsddb (or anydbm) database open, it gets corrupted. How come?
-------------------------------------------------------------------------------------------
-
-.. XXX move this FAQ entry elsewhere?
-
-.. note::
-
-   The bsddb module is now available as a standalone package `pybsddb
-   <http://www.jcea.es/programacion/pybsddb.htm>`_.
-
-Databases opened for write access with the bsddb module (and often by the anydbm
-module, since it will preferentially use bsddb) must explicitly be closed using
-the ``.close()`` method of the database.  The underlying library caches database
-contents which need to be converted to on-disk form and written.
-
-If you have initialized a new bsddb database but not written anything to it
-before the program crashes, you will often wind up with a zero-length file and
-encounter an exception the next time the file is opened.
-
-
-I tried to open Berkeley DB file, but bsddb produces bsddb.error: (22, 'Invalid argument'). Help! How can I restore my data?
-----------------------------------------------------------------------------------------------------------------------------
-
-.. XXX move this FAQ entry elsewhere?
-
-.. note::
-
-   The bsddb module is now available as a standalone package `pybsddb
-   <http://www.jcea.es/programacion/pybsddb.htm>`_.
-
-Don't panic! Your data is probably intact. The most frequent cause for the error
-is that you tried to open an earlier Berkeley DB file with a later version of
-the Berkeley DB library.
-
-Many Linux systems now have all three versions of Berkeley DB available.  If you
-are migrating from version 1 to a newer version use db_dump185 to dump a plain
-text version of the database.  If you are migrating from version 2 to version 3
-use db2_dump to create a plain text version of the database.  In either case,
-use db_load to create a new native database for the latest version installed on
-your computer.  If you have version 3 of Berkeley DB installed, you should be
-able to use db2_load to create a native version 2 database.
-
-You should move away from Berkeley DB version 1 files because the hash file code
-contains known bugs that can corrupt your data.
-
-
 Mathematics and Numerics
 ========================
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:11:11 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:11:11 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_The_functional_?=
 =?utf8?q?module_hasn=27t_been_maintained_since_2006_and_doesn=27t_work_wi?=
 =?utf8?q?th?=
Message-ID: <E1RXM9H-0008UA-Vf@dinsdale.python.org>

http://hg.python.org/cpython/rev/2aeef275bec8
changeset:   73856:2aeef275bec8
branch:      3.2
parent:      73854:1494e021cb9f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:05:32 2011 +0100
summary:
  The functional module hasn't been maintained since 2006 and doesn't work with Python 3.
Remove section about it from the functional programming FAQ.

files:
  Doc/howto/functional.rst |  128 ---------------------------
  1 files changed, 0 insertions(+), 128 deletions(-)


diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -1010,135 +1010,6 @@
 Consult the operator module's documentation for a complete list.
 
 
-
-The functional module
----------------------
-
-Collin Winter's `functional module <http://oakwinter.com/code/functional/>`__
-provides a number of more advanced tools for functional programming. It also
-reimplements several Python built-ins, trying to make them more intuitive to
-those used to functional programming in other languages.
-
-This section contains an introduction to some of the most important functions in
-``functional``; full documentation can be found at `the project's website
-<http://oakwinter.com/code/functional/documentation/>`__.
-
-``compose(outer, inner, unpack=False)``
-
-The ``compose()`` function implements function composition.  In other words, it
-returns a wrapper around the ``outer`` and ``inner`` callables, such that the
-return value from ``inner`` is fed directly to ``outer``.  That is, ::
-
-    >>> def add(a, b):
-    ...     return a + b
-    ...
-    >>> def double(a):
-    ...     return 2 * a
-    ...
-    >>> compose(double, add)(5, 6)
-    22
-
-is equivalent to ::
-
-    >>> double(add(5, 6))
-    22
-
-The ``unpack`` keyword is provided to work around the fact that Python functions
-are not always `fully curried <http://en.wikipedia.org/wiki/Currying>`__.  By
-default, it is expected that the ``inner`` function will return a single object
-and that the ``outer`` function will take a single argument. Setting the
-``unpack`` argument causes ``compose`` to expect a tuple from ``inner`` which
-will be expanded before being passed to ``outer``. Put simply, ::
-
-    compose(f, g)(5, 6)
-
-is equivalent to::
-
-    f(g(5, 6))
-
-while ::
-
-    compose(f, g, unpack=True)(5, 6)
-
-is equivalent to::
-
-    f(*g(5, 6))
-
-Even though ``compose()`` only accepts two functions, it's trivial to build up a
-version that will compose any number of functions. We'll use
-:func:`functools.reduce`, ``compose()`` and ``partial()`` (the last of which is
-provided by both ``functional`` and ``functools``). ::
-
-    from functional import compose, partial
-    import functools
-
-
-    multi_compose = partial(functools.reduce, compose)
-
-
-We can also use ``map()``, ``compose()`` and ``partial()`` to craft a version of
-``"".join(...)`` that converts its arguments to string::
-
-    from functional import compose, partial
-
-    join = compose("".join, partial(map, str))
-
-
-``flip(func)``
-
-``flip()`` wraps the callable in ``func`` and causes it to receive its
-non-keyword arguments in reverse order. ::
-
-    >>> def triple(a, b, c):
-    ...     return (a, b, c)
-    ...
-    >>> triple(5, 6, 7)
-    (5, 6, 7)
-    >>>
-    >>> flipped_triple = flip(triple)
-    >>> flipped_triple(5, 6, 7)
-    (7, 6, 5)
-
-``foldl(func, start, iterable)``
-
-``foldl()`` takes a binary function, a starting value (usually some kind of
-'zero'), and an iterable.  The function is applied to the starting value and the
-first element of the list, then the result of that and the second element of the
-list, then the result of that and the third element of the list, and so on.
-
-This means that a call such as::
-
-    foldl(f, 0, [1, 2, 3])
-
-is equivalent to::
-
-    f(f(f(0, 1), 2), 3)
-
-
-``foldl()`` is roughly equivalent to the following recursive function::
-
-    def foldl(func, start, seq):
-        if len(seq) == 0:
-            return start
-
-        return foldl(func, func(start, seq[0]), seq[1:])
-
-Speaking of equivalence, the above ``foldl`` call can be expressed in terms of
-the built-in :func:`functools.reduce` like so::
-
-    import functools
-    functools.reduce(f, [1, 2, 3], 0)
-
-
-We can use ``foldl()``, ``operator.concat()`` and ``partial()`` to write a
-cleaner, more aesthetically-pleasing version of Python's ``"".join(...)``
-idiom::
-
-    from functional import foldl, partial from operator import concat
-
-    join = partial(foldl, concat, "")
-
-
 Small functions and the lambda expression
 =========================================
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:11:12 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:11:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_The_functional_module_hasn=27t_been_maintained_since_2006_an?=
 =?utf8?q?d_doesn=27t_work_with?=
Message-ID: <E1RXM9I-0008UL-Mr@dinsdale.python.org>

http://hg.python.org/cpython/rev/3828f81a64e7
changeset:   73857:3828f81a64e7
parent:      73855:e80a4e3b799b
parent:      73856:2aeef275bec8
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:05:55 2011 +0100
summary:
  The functional module hasn't been maintained since 2006 and doesn't work with Python 3.
Remove section about it from the functional programming FAQ.

files:
  Doc/howto/functional.rst |  128 ---------------------------
  1 files changed, 0 insertions(+), 128 deletions(-)


diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -1010,135 +1010,6 @@
 Consult the operator module's documentation for a complete list.
 
 
-
-The functional module
----------------------
-
-Collin Winter's `functional module <http://oakwinter.com/code/functional/>`__
-provides a number of more advanced tools for functional programming. It also
-reimplements several Python built-ins, trying to make them more intuitive to
-those used to functional programming in other languages.
-
-This section contains an introduction to some of the most important functions in
-``functional``; full documentation can be found at `the project's website
-<http://oakwinter.com/code/functional/documentation/>`__.
-
-``compose(outer, inner, unpack=False)``
-
-The ``compose()`` function implements function composition.  In other words, it
-returns a wrapper around the ``outer`` and ``inner`` callables, such that the
-return value from ``inner`` is fed directly to ``outer``.  That is, ::
-
-    >>> def add(a, b):
-    ...     return a + b
-    ...
-    >>> def double(a):
-    ...     return 2 * a
-    ...
-    >>> compose(double, add)(5, 6)
-    22
-
-is equivalent to ::
-
-    >>> double(add(5, 6))
-    22
-
-The ``unpack`` keyword is provided to work around the fact that Python functions
-are not always `fully curried <http://en.wikipedia.org/wiki/Currying>`__.  By
-default, it is expected that the ``inner`` function will return a single object
-and that the ``outer`` function will take a single argument. Setting the
-``unpack`` argument causes ``compose`` to expect a tuple from ``inner`` which
-will be expanded before being passed to ``outer``. Put simply, ::
-
-    compose(f, g)(5, 6)
-
-is equivalent to::
-
-    f(g(5, 6))
-
-while ::
-
-    compose(f, g, unpack=True)(5, 6)
-
-is equivalent to::
-
-    f(*g(5, 6))
-
-Even though ``compose()`` only accepts two functions, it's trivial to build up a
-version that will compose any number of functions. We'll use
-:func:`functools.reduce`, ``compose()`` and ``partial()`` (the last of which is
-provided by both ``functional`` and ``functools``). ::
-
-    from functional import compose, partial
-    import functools
-
-
-    multi_compose = partial(functools.reduce, compose)
-
-
-We can also use ``map()``, ``compose()`` and ``partial()`` to craft a version of
-``"".join(...)`` that converts its arguments to string::
-
-    from functional import compose, partial
-
-    join = compose("".join, partial(map, str))
-
-
-``flip(func)``
-
-``flip()`` wraps the callable in ``func`` and causes it to receive its
-non-keyword arguments in reverse order. ::
-
-    >>> def triple(a, b, c):
-    ...     return (a, b, c)
-    ...
-    >>> triple(5, 6, 7)
-    (5, 6, 7)
-    >>>
-    >>> flipped_triple = flip(triple)
-    >>> flipped_triple(5, 6, 7)
-    (7, 6, 5)
-
-``foldl(func, start, iterable)``
-
-``foldl()`` takes a binary function, a starting value (usually some kind of
-'zero'), and an iterable.  The function is applied to the starting value and the
-first element of the list, then the result of that and the second element of the
-list, then the result of that and the third element of the list, and so on.
-
-This means that a call such as::
-
-    foldl(f, 0, [1, 2, 3])
-
-is equivalent to::
-
-    f(f(f(0, 1), 2), 3)
-
-
-``foldl()`` is roughly equivalent to the following recursive function::
-
-    def foldl(func, start, seq):
-        if len(seq) == 0:
-            return start
-
-        return foldl(func, func(start, seq[0]), seq[1:])
-
-Speaking of equivalence, the above ``foldl`` call can be expressed in terms of
-the built-in :func:`functools.reduce` like so::
-
-    import functools
-    functools.reduce(f, [1, 2, 3], 0)
-
-
-We can use ``foldl()``, ``operator.concat()`` and ``partial()`` to write a
-cleaner, more aesthetically-pleasing version of Python's ``"".join(...)``
-idiom::
-
-    from functional import foldl, partial from operator import concat
-
-    join = partial(foldl, concat, "")
-
-
 Small functions and the lambda expression
 =========================================
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:27:26 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:27:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_referenc?=
 =?utf8?q?e_to_the_base64_encoding=2E?=
Message-ID: <E1RXMP0-00011i-3L@dinsdale.python.org>

http://hg.python.org/cpython/rev/427b9dae1ae3
changeset:   73858:427b9dae1ae3
branch:      3.2
parent:      73856:2aeef275bec8
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:21:46 2011 +0100
summary:
  Remove reference to the base64 encoding.

files:
  Doc/howto/unicode.rst |  27 +++++----------------------
  1 files changed, 5 insertions(+), 22 deletions(-)


diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
--- a/Doc/howto/unicode.rst
+++ b/Doc/howto/unicode.rst
@@ -552,7 +552,6 @@
 i.e. Unix systems.
 
 
-
 Tips for Writing Unicode-aware Programs
 ---------------------------------------
 
@@ -572,28 +571,12 @@
 When using data coming from a web browser or some other untrusted source, a
 common technique is to check for illegal characters in a string before using the
 string in a generated command line or storing it in a database.  If you're doing
-this, be careful to check the string once it's in the form that will be used or
-stored; it's possible for encodings to be used to disguise characters.  This is
-especially true if the input data also specifies the encoding; many encodings
-leave the commonly checked-for characters alone, but Python includes some
-encodings such as ``'base64'`` that modify every single character.
+this, be careful to check the decoded string, not the encoded bytes data;
+some encodings may have interesting properties, such as not being bijective
+or not being fully ASCII-compatible.  This is especially true if the input
+data also specifies the encoding, since the attacker can then choose a
+clever way to hide malicious text in the encoded bytestream.
 
-For example, let's say you have a content management system that takes a Unicode
-filename, and you want to disallow paths with a '/' character.  You might write
-this code::
-
-    def read_file(filename, encoding):
-        if '/' in filename:
-            raise ValueError("'/' not allowed in filenames")
-        unicode_name = filename.decode(encoding)
-        with open(unicode_name, 'r') as f:
-            # ... return contents of file ...
-
-However, if an attacker could specify the ``'base64'`` encoding, they could pass
-``'L2V0Yy9wYXNzd2Q='``, which is the base-64 encoded form of the string
-``'/etc/passwd'``, to read a system file.  The above code looks for ``'/'``
-characters in the encoded form and misses the dangerous character in the
-resulting decoded form.
 
 References
 ----------

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:27:26 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:27:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Remove_reference_to_the_base64_encoding=2E?=
Message-ID: <E1RXMP0-00011s-Qq@dinsdale.python.org>

http://hg.python.org/cpython/rev/8701f6373d0b
changeset:   73859:8701f6373d0b
parent:      73857:3828f81a64e7
parent:      73858:427b9dae1ae3
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:22:03 2011 +0100
summary:
  Remove reference to the base64 encoding.

files:
  Doc/howto/unicode.rst |  27 +++++----------------------
  1 files changed, 5 insertions(+), 22 deletions(-)


diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
--- a/Doc/howto/unicode.rst
+++ b/Doc/howto/unicode.rst
@@ -552,7 +552,6 @@
 i.e. Unix systems.
 
 
-
 Tips for Writing Unicode-aware Programs
 ---------------------------------------
 
@@ -572,28 +571,12 @@
 When using data coming from a web browser or some other untrusted source, a
 common technique is to check for illegal characters in a string before using the
 string in a generated command line or storing it in a database.  If you're doing
-this, be careful to check the string once it's in the form that will be used or
-stored; it's possible for encodings to be used to disguise characters.  This is
-especially true if the input data also specifies the encoding; many encodings
-leave the commonly checked-for characters alone, but Python includes some
-encodings such as ``'base64'`` that modify every single character.
+this, be careful to check the decoded string, not the encoded bytes data;
+some encodings may have interesting properties, such as not being bijective
+or not being fully ASCII-compatible.  This is especially true if the input
+data also specifies the encoding, since the attacker can then choose a
+clever way to hide malicious text in the encoded bytestream.
 
-For example, let's say you have a content management system that takes a Unicode
-filename, and you want to disallow paths with a '/' character.  You might write
-this code::
-
-    def read_file(filename, encoding):
-        if '/' in filename:
-            raise ValueError("'/' not allowed in filenames")
-        unicode_name = filename.decode(encoding)
-        with open(unicode_name, 'r') as f:
-            # ... return contents of file ...
-
-However, if an attacker could specify the ``'base64'`` encoding, they could pass
-``'L2V0Yy9wYXNzd2Q='``, which is the base-64 encoded form of the string
-``'/etc/passwd'``, to read a system file.  The above code looks for ``'/'``
-characters in the encoded form and misses the dangerous character in the
-resulting decoded form.
 
 References
 ----------

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:53:05 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:53:05 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_the_outd?=
 =?utf8?q?ated_notion_that_multithreading_doesn=27t_work_well_on_Unices=2E?=
Message-ID: <E1RXMnp-0002in-Le@dinsdale.python.org>

http://hg.python.org/cpython/rev/64d980770571
changeset:   73860:64d980770571
branch:      3.2
parent:      73858:427b9dae1ae3
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:32:29 2011 +0100
summary:
  Remove the outdated notion that multithreading doesn't work well on Unices.

files:
  Doc/howto/sockets.rst |  18 ++++++------------
  1 files changed, 6 insertions(+), 12 deletions(-)


diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst
--- a/Doc/howto/sockets.rst
+++ b/Doc/howto/sockets.rst
@@ -395,19 +395,13 @@
 
 There's no question that the fastest sockets code uses non-blocking sockets and
 select to multiplex them. You can put together something that will saturate a
-LAN connection without putting any strain on the CPU. The trouble is that an app
-written this way can't do much of anything else - it needs to be ready to
-shuffle bytes around at all times.
+LAN connection without putting any strain on the CPU.
 
-Assuming that your app is actually supposed to do something more than that,
-threading is the optimal solution, (and using non-blocking sockets will be
-faster than using blocking sockets). Unfortunately, threading support in Unixes
-varies both in API and quality. So the normal Unix solution is to fork a
-subprocess to deal with each connection. The overhead for this is significant
-(and don't do this on Windows - the overhead of process creation is enormous
-there). It also means that unless each subprocess is completely independent,
-you'll need to use another form of IPC, say a pipe, or shared memory and
-semaphores, to communicate between the parent and child processes.
+The trouble is that an app written this way can't do much of anything else -
+it needs to be ready to shuffle bytes around at all times. Assuming that your
+app is actually supposed to do something more than that, threading is the
+optimal solution, (and using non-blocking sockets will be faster than using
+blocking sockets).
 
 Finally, remember that even though blocking sockets are somewhat slower than
 non-blocking, in many cases they are the "right" solution. After all, if your

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:53:06 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:53:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Use_www=2Epytho?=
 =?utf8?q?n=2Eorg_instead_of_a_hostname_pointing_to_a_parked_=28or_squatte?=
 =?utf8?q?d=29?=
Message-ID: <E1RXMnq-0002ix-Bw@dinsdale.python.org>

http://hg.python.org/cpython/rev/7e310c9cf46e
changeset:   73861:7e310c9cf46e
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:37:34 2011 +0100
summary:
  Use www.python.org instead of a hostname pointing to a parked (or squatted) domain.
Also, reformat a bit.

files:
  Doc/howto/sockets.rst |  23 ++++++++++-------------
  1 files changed, 10 insertions(+), 13 deletions(-)


diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst
--- a/Doc/howto/sockets.rst
+++ b/Doc/howto/sockets.rst
@@ -60,11 +60,10 @@
 Roughly speaking, when you clicked on the link that brought you to this page,
 your browser did something like the following::
 
-   #create an INET, STREAMing socket
+   # create an INET, STREAMing socket
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-   #now connect to the web server on port 80
-   # - the normal http port
-   s.connect(("www.mcmillan-inc.com", 80))
+   # now connect to the web server on port 80 - the normal http port
+   s.connect(("www.python.org", 80))
 
 When the ``connect`` completes, the socket ``s`` can be used to send
 in a request for the text of the page. The same socket will read the
@@ -75,13 +74,11 @@
 What happens in the web server is a bit more complex. First, the web server
 creates a "server socket"::
 
-   #create an INET, STREAMing socket
-   serversocket = socket.socket(
-       socket.AF_INET, socket.SOCK_STREAM)
-   #bind the socket to a public host,
-   # and a well-known port
+   # create an INET, STREAMing socket
+   serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+   # bind the socket to a public host, and a well-known port
    serversocket.bind((socket.gethostname(), 80))
-   #become a server socket
+   # become a server socket
    serversocket.listen(5)
 
 A couple things to notice: we used ``socket.gethostname()`` so that the socket
@@ -101,10 +98,10 @@
 mainloop of the web server::
 
    while True:
-       #accept connections from outside
+       # accept connections from outside
        (clientsocket, address) = serversocket.accept()
-       #now do something with the clientsocket
-       #in this case, we'll pretend this is a threaded server
+       # now do something with the clientsocket
+       # in this case, we'll pretend this is a threaded server
        ct = client_thread(clientsocket)
        ct.run()
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:53:07 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:53:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Reword_IPC_sect?=
	=?utf8?q?ion?=
Message-ID: <E1RXMnr-0002j1-1h@dinsdale.python.org>

http://hg.python.org/cpython/rev/9d8a0cfcd6d9
changeset:   73862:9d8a0cfcd6d9
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:43:32 2011 +0100
summary:
  Reword IPC section

files:
  Doc/howto/sockets.rst |  11 ++++++-----
  1 files changed, 6 insertions(+), 5 deletions(-)


diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst
--- a/Doc/howto/sockets.rst
+++ b/Doc/howto/sockets.rst
@@ -123,12 +123,13 @@
 ---
 
 If you need fast IPC between two processes on one machine, you should look into
-whatever form of shared memory the platform offers. A simple protocol based
-around shared memory and locks or semaphores is by far the fastest technique.
+pipes or shared memory.  If you do decide to use AF_INET sockets, bind the
+"server" socket to ``'localhost'``. On most platforms, this will take a
+shortcut around a couple of layers of network code and be quite a bit faster.
 
-If you do decide to use sockets, bind the "server" socket to ``'localhost'``. On
-most platforms, this will take a shortcut around a couple of layers of network
-code and be quite a bit faster.
+.. seealso::
+   The :mod:`multiprocessing` integrates cross-platform IPC into a higher-level
+   API.
 
 
 Using a Socket

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:53:07 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:53:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_s/SOCKSTREAM/TC?=
	=?utf8?q?P/?=
Message-ID: <E1RXMnr-0002jX-Nm@dinsdale.python.org>

http://hg.python.org/cpython/rev/c34188efb965
changeset:   73863:c34188efb965
branch:      3.2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:46:35 2011 +0100
summary:
  s/SOCKSTREAM/TCP/

files:
  Doc/howto/sockets.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst
--- a/Doc/howto/sockets.rst
+++ b/Doc/howto/sockets.rst
@@ -298,7 +298,7 @@
 
 Probably the worst thing about using blocking sockets is what happens when the
 other side comes down hard (without doing a ``close``). Your socket is likely to
-hang. SOCKSTREAM is a reliable protocol, and it will wait a long, long time
+hang. TCP is a reliable protocol, and it will wait a long, long time
 before giving up on a connection. If you're using threads, the entire thread is
 essentially dead. There's not much you can do about it. As long as you aren't
 doing something dumb, like holding a lock while doing a blocking read, the

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 01:53:08 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 01:53:08 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_assorted_fixes_from_3=2E2?=
Message-ID: <E1RXMns-0002jb-Dx@dinsdale.python.org>

http://hg.python.org/cpython/rev/f2be1e010cc8
changeset:   73864:f2be1e010cc8
parent:      73859:8701f6373d0b
parent:      73863:c34188efb965
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 01:47:40 2011 +0100
summary:
  Merge assorted fixes from 3.2

files:
  Doc/howto/sockets.rst |  36 ++++++++++++++----------------
  1 files changed, 17 insertions(+), 19 deletions(-)


diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst
--- a/Doc/howto/sockets.rst
+++ b/Doc/howto/sockets.rst
@@ -60,11 +60,10 @@
 Roughly speaking, when you clicked on the link that brought you to this page,
 your browser did something like the following::
 
-   #create an INET, STREAMing socket
+   # create an INET, STREAMing socket
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-   #now connect to the web server on port 80
-   # - the normal http port
-   s.connect(("www.mcmillan-inc.com", 80))
+   # now connect to the web server on port 80 - the normal http port
+   s.connect(("www.python.org", 80))
 
 When the ``connect`` completes, the socket ``s`` can be used to send
 in a request for the text of the page. The same socket will read the
@@ -75,13 +74,11 @@
 What happens in the web server is a bit more complex. First, the web server
 creates a "server socket"::
 
-   #create an INET, STREAMing socket
-   serversocket = socket.socket(
-       socket.AF_INET, socket.SOCK_STREAM)
-   #bind the socket to a public host,
-   # and a well-known port
+   # create an INET, STREAMing socket
+   serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+   # bind the socket to a public host, and a well-known port
    serversocket.bind((socket.gethostname(), 80))
-   #become a server socket
+   # become a server socket
    serversocket.listen(5)
 
 A couple things to notice: we used ``socket.gethostname()`` so that the socket
@@ -101,10 +98,10 @@
 mainloop of the web server::
 
    while True:
-       #accept connections from outside
+       # accept connections from outside
        (clientsocket, address) = serversocket.accept()
-       #now do something with the clientsocket
-       #in this case, we'll pretend this is a threaded server
+       # now do something with the clientsocket
+       # in this case, we'll pretend this is a threaded server
        ct = client_thread(clientsocket)
        ct.run()
 
@@ -126,12 +123,13 @@
 ---
 
 If you need fast IPC between two processes on one machine, you should look into
-whatever form of shared memory the platform offers. A simple protocol based
-around shared memory and locks or semaphores is by far the fastest technique.
+pipes or shared memory.  If you do decide to use AF_INET sockets, bind the
+"server" socket to ``'localhost'``. On most platforms, this will take a
+shortcut around a couple of layers of network code and be quite a bit faster.
 
-If you do decide to use sockets, bind the "server" socket to ``'localhost'``. On
-most platforms, this will take a shortcut around a couple of layers of network
-code and be quite a bit faster.
+.. seealso::
+   The :mod:`multiprocessing` integrates cross-platform IPC into a higher-level
+   API.
 
 
 Using a Socket
@@ -300,7 +298,7 @@
 
 Probably the worst thing about using blocking sockets is what happens when the
 other side comes down hard (without doing a ``close``). Your socket is likely to
-hang. SOCKSTREAM is a reliable protocol, and it will wait a long, long time
+hang. TCP is a reliable protocol, and it will wait a long, long time
 before giving up on a connection. If you're using threads, the entire thread is
 essentially dead. There's not much you can do about it. As long as you aren't
 doing something dumb, like holding a lock while doing a blocking read, the

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Dec  5 05:32:51 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 05 Dec 2011 05:32:51 +0100
Subject: [Python-checkins] Daily reference leaks (f2be1e010cc8): sum=0
Message-ID: <E1RXQEV-0007lL-4b@ap.vmr.nerim.net>

results for f2be1e010cc8 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloglEA4bV', '-x']

From python-checkins at python.org  Mon Dec  5 20:45:48 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 05 Dec 2011 20:45:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313503=3A_Use_a_mor?=
 =?utf8?q?e_efficient_reduction_format_for_bytearrays_with?=
Message-ID: <E1RXeU0-0001bl-CF@dinsdale.python.org>

http://hg.python.org/cpython/rev/e2959a6a1440
changeset:   73865:e2959a6a1440
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 05 20:40:08 2011 +0100
summary:
  Issue #13503: Use a more efficient reduction format for bytearrays with
pickle protocol >= 3.  The old reduction format is kept with older
protocols in order to allow unpickling under Python 2.

Patch by Irmen de Jong.

files:
  Misc/NEWS                 |   4 ++
  Objects/bytearrayobject.c |  52 +++++++++++++++++++++-----
  2 files changed, 46 insertions(+), 10 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #13503: Use a more efficient reduction format for bytearrays with
+  pickle protocol >= 3.  The old reduction format is kept with older protocols
+  in order to allow unpickling under Python 2.  Patch by Irmen de Jong.
+
 - Issue #7111: Python can now be run without a stdin, stdout or stderr
   stream.  It was already the case with Python 2.  However, the corresponding
   sys module entries are now set to None (instead of an unusable file object).
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -2725,20 +2725,13 @@
     return NULL;
 }
 
-PyDoc_STRVAR(reduce_doc, "Return state information for pickling.");
 
 static PyObject *
-bytearray_reduce(PyByteArrayObject *self)
+_common_reduce(PyByteArrayObject *self, int proto)
 {
-    PyObject *latin1, *dict;
+    PyObject *dict;
     _Py_IDENTIFIER(__dict__);
 
-    if (self->ob_bytes)
-        latin1 = PyUnicode_DecodeLatin1(self->ob_bytes,
-                                        Py_SIZE(self), NULL);
-    else
-        latin1 = PyUnicode_FromString("");
-
     dict = _PyObject_GetAttrId((PyObject *)self, &PyId___dict__);
     if (dict == NULL) {
         PyErr_Clear();
@@ -2746,7 +2739,45 @@
         Py_INCREF(dict);
     }
 
-    return Py_BuildValue("(O(Ns)N)", Py_TYPE(self), latin1, "latin-1", dict);
+    if (proto < 3) {
+        /* use str based reduction for backwards compatibility with Python 2.x */
+        PyObject *latin1;
+        if (self->ob_bytes)
+            latin1 = PyUnicode_DecodeLatin1(self->ob_bytes, Py_SIZE(self), NULL);
+        else
+            latin1 = PyUnicode_FromString("");
+        return Py_BuildValue("(O(Ns)N)", Py_TYPE(self), latin1, "latin-1", dict);
+    }
+    else {
+        /* use more efficient byte based reduction */
+        if (self->ob_bytes) {
+            return Py_BuildValue("(O(y#)N)", Py_TYPE(self), self->ob_bytes, Py_SIZE(self), dict);
+        }
+        else {
+            return Py_BuildValue("(O()N)", Py_TYPE(self), dict);
+        }
+    }
+}
+
+PyDoc_STRVAR(reduce_doc, "Return state information for pickling.");
+
+static PyObject *
+bytearray_reduce(PyByteArrayObject *self)
+{
+    return _common_reduce(self, 2);
+}
+
+PyDoc_STRVAR(reduce_ex_doc, "Return state information for pickling.");
+
+static PyObject *
+bytearray_reduce_ex(PyByteArrayObject *self, PyObject *args)
+{
+    int proto = 0;
+
+    if (!PyArg_ParseTuple(args, "|i:__reduce_ex__", &proto))
+        return NULL;
+
+    return _common_reduce(self, proto);
 }
 
 PyDoc_STRVAR(sizeof_doc,
@@ -2790,6 +2821,7 @@
 bytearray_methods[] = {
     {"__alloc__", (PyCFunction)bytearray_alloc, METH_NOARGS, alloc_doc},
     {"__reduce__", (PyCFunction)bytearray_reduce, METH_NOARGS, reduce_doc},
+    {"__reduce_ex__", (PyCFunction)bytearray_reduce_ex, METH_VARARGS, reduce_ex_doc},
     {"__sizeof__", (PyCFunction)bytearray_sizeof, METH_NOARGS, sizeof_doc},
     {"append", (PyCFunction)bytearray_append, METH_O, append__doc__},
     {"capitalize", (PyCFunction)stringlib_capitalize, METH_NOARGS,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 22:50:49 2011
From: python-checkins at python.org (barry.warsaw)
Date: Mon, 05 Dec 2011 22:50:49 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_-_Issue_=231114?=
 =?utf8?q?7=3A_Fix_an_unused_argument_in_=5FPy=5FANNOTATE=5FMEMORY=5FORDER?=
 =?utf8?q?=2E__=28Fix?=
Message-ID: <E1RXgQz-0004yQ-LJ@dinsdale.python.org>

http://hg.python.org/cpython/rev/4579cd952156
changeset:   73866:4579cd952156
branch:      3.2
parent:      73863:c34188efb965
user:        Barry Warsaw <barry at python.org>
date:        Mon Dec 05 16:45:02 2011 -0500
summary:
  - Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
  given by Campbell Barton).

files:
  Include/pyatomic.h |  1 +
  Misc/NEWS          |  3 +++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Include/pyatomic.h b/Include/pyatomic.h
--- a/Include/pyatomic.h
+++ b/Include/pyatomic.h
@@ -58,6 +58,7 @@
 static __inline__ void
 _Py_ANNOTATE_MEMORY_ORDER(const volatile void *address, _Py_memory_order order)
 {
+    (void)address;		/* shut up -Wunused-parameter */
     switch(order) {
     case _Py_memory_order_release:
     case _Py_memory_order_acq_rel:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
+  given by Campbell Barton).
+
 - Issue #7111: Python can now be run without a stdin, stdout or stderr
   stream.  It was already the case with Python 2.  However, the corresponding
   sys module entries are now set to None (instead of an unusable file object).

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec  5 22:50:50 2011
From: python-checkins at python.org (barry.warsaw)
Date: Mon, 05 Dec 2011 22:50:50 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_-_Issue_=2311147=3A_Fix_an_unused_argument_in_=5FPy=5FANNOTA?=
 =?utf8?q?TE=5FMEMORY=5FORDER=2E__=28Fix?=
Message-ID: <E1RXgR0-0004yU-DC@dinsdale.python.org>

http://hg.python.org/cpython/rev/6b6c79eba944
changeset:   73867:6b6c79eba944
parent:      73865:e2959a6a1440
parent:      73866:4579cd952156
user:        Barry Warsaw <barry at python.org>
date:        Mon Dec 05 16:50:41 2011 -0500
summary:
  - Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
  given by Campbell Barton).

files:
  Include/pyatomic.h |  1 +
  Misc/NEWS          |  3 +++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Include/pyatomic.h b/Include/pyatomic.h
--- a/Include/pyatomic.h
+++ b/Include/pyatomic.h
@@ -58,6 +58,7 @@
 static __inline__ void
 _Py_ANNOTATE_MEMORY_ORDER(const volatile void *address, _Py_memory_order order)
 {
+    (void)address;		/* shut up -Wunused-parameter */
     switch(order) {
     case _Py_memory_order_release:
     case _Py_memory_order_acq_rel:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
+  given by Campbell Barton).
+
 - Issue #13503: Use a more efficient reduction format for bytearrays with
   pickle protocol >= 3.  The old reduction format is kept with older protocols
   in order to allow unpickling under Python 2.  Patch by Irmen de Jong.

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Dec  6 05:33:11 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 06 Dec 2011 05:33:11 +0100
Subject: [Python-checkins] Daily reference leaks (6b6c79eba944): sum=0
Message-ID: <E1RXmiN-0006xq-SQ@ap.vmr.nerim.net>

results for 6b6c79eba944 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflognNz12r', '-x']

From python-checkins at python.org  Tue Dec  6 13:10:22 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Tue, 06 Dec 2011 13:10:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Correctly_detec?=
 =?utf8?q?t_bzip2_compressed_streams_with_blocksizes_other_than_900k=2E?=
Message-ID: <E1RXtqo-0007K4-52@dinsdale.python.org>

http://hg.python.org/cpython/rev/80876df8adce
changeset:   73868:80876df8adce
branch:      3.2
parent:      73866:4579cd952156
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Tue Dec 06 12:56:38 2011 +0100
summary:
  Correctly detect bzip2 compressed streams with blocksizes other than 900k.

files:
  Lib/tarfile.py           |   2 +-
  Lib/test/test_tarfile.py |  22 ++++++++++++++++++----
  Misc/NEWS                |   3 +++
  3 files changed, 22 insertions(+), 5 deletions(-)


diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -627,7 +627,7 @@
     def getcomptype(self):
         if self.buf.startswith(b"\037\213\010"):
             return "gz"
-        if self.buf.startswith(b"BZh91"):
+        if self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
             return "bz2"
         return "tar"
 
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -529,6 +529,23 @@
     def test_detect_fileobj(self):
         self._test_modes(self._testfunc_fileobj)
 
+    def test_detect_stream_bz2(self):
+        # Originally, tarfile's stream detection looked for the string
+        # "BZh91" at the start of the file. This is incorrect because
+        # the '9' represents the blocksize (900kB). If the file was
+        # compressed using another blocksize autodetection fails.
+        if not bz2:
+            return
+
+        with open(tarname, "rb") as fobj:
+            data = fobj.read()
+
+        # Compress with blocksize 100kB, the file starts with "BZh11".
+        with bz2.BZ2File(tmpname, "wb", compresslevel=1) as fobj:
+            fobj.write(data)
+
+        self._testfunc_file(tmpname, "r|*")
+
 
 class MemberReadTest(ReadTest):
 
@@ -1818,11 +1835,8 @@
     if bz2:
         # Create testtar.tar.bz2 and add bz2-specific tests.
         support.unlink(bz2name)
-        tar = bz2.BZ2File(bz2name, "wb")
-        try:
+        with bz2.BZ2File(bz2name, "wb") as tar:
             tar.write(data)
-        finally:
-            tar.close()
 
         tests += [
             Bz2MiscReadTest,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -90,6 +90,9 @@
 Library
 -------
 
+- tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
+  other than 900k.
+
 - Issue #13439: Fix many errors in turtle docstrings.
 
 - Issue #13487: Make inspect.getmodule robust against changes done to

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 13:10:30 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Tue, 06 Dec 2011 13:10:30 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_with_3=2E2=3A_Correctly_detect_bzip2_compressed_stream?=
 =?utf8?q?s_with_blocksizes_other?=
Message-ID: <E1RXtqw-0007KG-6Q@dinsdale.python.org>

http://hg.python.org/cpython/rev/9149aefff883
changeset:   73869:9149aefff883
parent:      73867:6b6c79eba944
parent:      73868:80876df8adce
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Tue Dec 06 13:00:58 2011 +0100
summary:
  Merge with 3.2: Correctly detect bzip2 compressed streams with blocksizes other than 900k.

files:
  Lib/tarfile.py           |   2 +-
  Lib/test/test_tarfile.py |  22 ++++++++++++++++++----
  Misc/NEWS                |   3 +++
  3 files changed, 22 insertions(+), 5 deletions(-)


diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -624,7 +624,7 @@
     def getcomptype(self):
         if self.buf.startswith(b"\037\213\010"):
             return "gz"
-        if self.buf.startswith(b"BZh91"):
+        if self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
             return "bz2"
         return "tar"
 
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -529,6 +529,23 @@
     def test_detect_fileobj(self):
         self._test_modes(self._testfunc_fileobj)
 
+    def test_detect_stream_bz2(self):
+        # Originally, tarfile's stream detection looked for the string
+        # "BZh91" at the start of the file. This is incorrect because
+        # the '9' represents the blocksize (900kB). If the file was
+        # compressed using another blocksize autodetection fails.
+        if not bz2:
+            return
+
+        with open(tarname, "rb") as fobj:
+            data = fobj.read()
+
+        # Compress with blocksize 100kB, the file starts with "BZh11".
+        with bz2.BZ2File(tmpname, "wb", compresslevel=1) as fobj:
+            fobj.write(data)
+
+        self._testfunc_file(tmpname, "r|*")
+
 
 class MemberReadTest(ReadTest):
 
@@ -1818,11 +1835,8 @@
     if bz2:
         # Create testtar.tar.bz2 and add bz2-specific tests.
         support.unlink(bz2name)
-        tar = bz2.BZ2File(bz2name, "wb")
-        try:
+        with bz2.BZ2File(bz2name, "wb") as tar:
             tar.write(data)
-        finally:
-            tar.close()
 
         tests += [
             Bz2MiscReadTest,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,9 @@
 Library
 -------
 
+- tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
+  other than 900k.
+
 - Issue #13439: Fix many errors in turtle docstrings.
 
 - Issue #6715: Add a module 'lzma' for compression using the LZMA algorithm.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 13:10:37 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Tue, 06 Dec 2011 13:10:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Correctly_detec?=
 =?utf8?q?t_bzip2_compressed_streams_with_blocksizes_other_than_900k=2E?=
Message-ID: <E1RXtr3-0007LG-LJ@dinsdale.python.org>

http://hg.python.org/cpython/rev/6d1a91e9f506
changeset:   73870:6d1a91e9f506
branch:      2.7
parent:      73853:478b4e9551fa
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Tue Dec 06 13:07:09 2011 +0100
summary:
  Correctly detect bzip2 compressed streams with blocksizes other than 900k.

files:
  Lib/tarfile.py           |   2 +-
  Lib/test/test_tarfile.py |  17 +++++++++++++++++
  Misc/NEWS                |   3 +++
  3 files changed, 21 insertions(+), 1 deletions(-)


diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -627,7 +627,7 @@
     def getcomptype(self):
         if self.buf.startswith("\037\213\010"):
             return "gz"
-        if self.buf.startswith("BZh91"):
+        if self.buf[0:3] == "BZh" and self.buf[4:10] == "1AY&SY":
             return "bz2"
         return "tar"
 
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -440,6 +440,23 @@
     def test_detect_fileobj(self):
         self._test_modes(self._testfunc_fileobj)
 
+    def test_detect_stream_bz2(self):
+        # Originally, tarfile's stream detection looked for the string
+        # "BZh91" at the start of the file. This is incorrect because
+        # the '9' represents the blocksize (900kB). If the file was
+        # compressed using another blocksize autodetection fails.
+        if not bz2:
+            return
+
+        with open(tarname, "rb") as fobj:
+            data = fobj.read()
+
+        # Compress with blocksize 100kB, the file starts with "BZh11".
+        with bz2.BZ2File(tmpname, "wb", compresslevel=1) as fobj:
+            fobj.write(data)
+
+        self._testfunc_file(tmpname, "r|*")
+
 
 class MemberReadTest(ReadTest):
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -79,6 +79,9 @@
 Library
 -------
 
+- tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
+  other than 900k.
+
 - Issue #13439: Fix many errors in turtle docstrings.
 
 - Issue #12856: Ensure child processes do not inherit the parent's random

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 13:49:05 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Tue, 06 Dec 2011 13:49:05 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Remove_no_longer_needed_wor?=
 =?utf8?q?k-around_for_bz2_file_object_support=2E?=
Message-ID: <E1RXuSH-0002H3-LP@dinsdale.python.org>

http://hg.python.org/cpython/rev/56b7b1ecc240
changeset:   73871:56b7b1ecc240
parent:      73869:9149aefff883
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Tue Dec 06 13:44:10 2011 +0100
summary:
  Remove no longer needed work-around for bz2 file object support.

files:
  Lib/tarfile.py           |  66 +---------------------------
  Lib/test/test_tarfile.py |   3 +
  2 files changed, 5 insertions(+), 64 deletions(-)


diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -632,66 +632,6 @@
         self.fileobj.close()
 # class StreamProxy
 
-class _BZ2Proxy(object):
-    """Small proxy class that enables external file object
-       support for "r:bz2" and "w:bz2" modes. This is actually
-       a workaround for a limitation in bz2 module's BZ2File
-       class which (unlike gzip.GzipFile) has no support for
-       a file object argument.
-    """
-
-    blocksize = 16 * 1024
-
-    def __init__(self, fileobj, mode):
-        self.fileobj = fileobj
-        self.mode = mode
-        self.name = getattr(self.fileobj, "name", None)
-        self.init()
-
-    def init(self):
-        import bz2
-        self.pos = 0
-        if self.mode == "r":
-            self.bz2obj = bz2.BZ2Decompressor()
-            self.fileobj.seek(0)
-            self.buf = b""
-        else:
-            self.bz2obj = bz2.BZ2Compressor()
-
-    def read(self, size):
-        x = len(self.buf)
-        while x < size:
-            raw = self.fileobj.read(self.blocksize)
-            if not raw:
-                break
-            data = self.bz2obj.decompress(raw)
-            self.buf += data
-            x += len(data)
-
-        buf = self.buf[:size]
-        self.buf = self.buf[size:]
-        self.pos += len(buf)
-        return buf
-
-    def seek(self, pos):
-        if pos < self.pos:
-            self.init()
-        self.read(pos - self.pos)
-
-    def tell(self):
-        return self.pos
-
-    def write(self, data):
-        self.pos += len(data)
-        raw = self.bz2obj.compress(data)
-        self.fileobj.write(raw)
-
-    def close(self):
-        if self.mode == "w":
-            raw = self.bz2obj.flush()
-            self.fileobj.write(raw)
-# class _BZ2Proxy
-
 #------------------------
 # Extraction file object
 #------------------------
@@ -1829,10 +1769,8 @@
         except ImportError:
             raise CompressionError("bz2 module is not available")
 
-        if fileobj is not None:
-            fileobj = _BZ2Proxy(fileobj, mode)
-        else:
-            fileobj = bz2.BZ2File(name, mode, compresslevel=compresslevel)
+        fileobj = bz2.BZ2File(filename=name if fileobj is None else None,
+                mode=mode, fileobj=fileobj, compresslevel=compresslevel)
 
         try:
             t = cls.taropen(name, mode, fileobj, **kwargs)
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -222,6 +222,9 @@
 class MiscReadTest(CommonReadTest):
 
     def test_no_name_argument(self):
+        if self.mode.endswith("bz2"):
+            # BZ2File has no name attribute.
+            return
         with open(self.tarname, "rb") as fobj:
             tar = tarfile.open(fileobj=fobj, mode=self.mode)
             self.assertEqual(tar.name, os.path.abspath(fobj.name))

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 20:47:51 2011
From: python-checkins at python.org (jesus.cea)
Date: Tue, 06 Dec 2011 20:47:51 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2UgIzEzNTAw?=
 =?utf8?q?=3A_Hitting_EOF_gets_cmd=2Epy_into_a_infinite_EOF_on_return_loop?=
Message-ID: <E1RY0zX-0001Wm-Kw@dinsdale.python.org>

http://hg.python.org/cpython/rev/5910c385fab6
changeset:   73872:5910c385fab6
branch:      2.7
parent:      73870:6d1a91e9f506
user:        Jesus Cea <jcea at jcea.es>
date:        Tue Dec 06 20:46:04 2011 +0100
summary:
  Close #13500: Hitting EOF gets cmd.py into a infinite EOF on return loop

files:
  Lib/cmd.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/cmd.py b/Lib/cmd.py
--- a/Lib/cmd.py
+++ b/Lib/cmd.py
@@ -209,6 +209,8 @@
         if cmd is None:
             return self.default(line)
         self.lastcmd = line
+        if line == 'EOF' :
+            self.lastcmd = ''
         if cmd == '':
             return self.default(line)
         else:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 20:47:53 2011
From: python-checkins at python.org (jesus.cea)
Date: Tue, 06 Dec 2011 20:47:53 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogQ2xvc2UgIzEzNTAw?=
 =?utf8?q?=3A_Hitting_EOF_gets_cmd=2Epy_into_a_infinite_EOF_on_return_loop?=
Message-ID: <E1RY0zZ-0001XJ-2R@dinsdale.python.org>

http://hg.python.org/cpython/rev/b6b4d74b8d42
changeset:   73873:b6b4d74b8d42
branch:      3.2
parent:      73868:80876df8adce
user:        Jesus Cea <jcea at jcea.es>
date:        Tue Dec 06 20:46:57 2011 +0100
summary:
  Close #13500: Hitting EOF gets cmd.py into a infinite EOF on return loop

files:
  Lib/cmd.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/cmd.py b/Lib/cmd.py
--- a/Lib/cmd.py
+++ b/Lib/cmd.py
@@ -205,6 +205,8 @@
         if cmd is None:
             return self.default(line)
         self.lastcmd = line
+        if line == 'EOF' :
+            self.lastcmd = ''
         if cmd == '':
             return self.default(line)
         else:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 20:47:53 2011
From: python-checkins at python.org (jesus.cea)
Date: Tue, 06 Dec 2011 20:47:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_MERGE=3A_Close_=2313500=3A_Hitting_EOF_gets_cmd=2Epy_into_a_?=
 =?utf8?q?infinite_EOF_on_return_loop?=
Message-ID: <E1RY0zZ-0001XN-Qz@dinsdale.python.org>

http://hg.python.org/cpython/rev/70ba352f9586
changeset:   73874:70ba352f9586
parent:      73871:56b7b1ecc240
parent:      73873:b6b4d74b8d42
user:        Jesus Cea <jcea at jcea.es>
date:        Tue Dec 06 20:47:38 2011 +0100
summary:
  MERGE: Close #13500: Hitting EOF gets cmd.py into a infinite EOF on return loop

files:
  Lib/cmd.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/cmd.py b/Lib/cmd.py
--- a/Lib/cmd.py
+++ b/Lib/cmd.py
@@ -205,6 +205,8 @@
         if cmd is None:
             return self.default(line)
         self.lastcmd = line
+        if line == 'EOF' :
+            self.lastcmd = ''
         if cmd == '':
             return self.default(line)
         else:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 22:39:53 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 06 Dec 2011 22:39:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313464=3A_Add_a_rea?=
 =?utf8?q?dinto=28=29_method_to_http=2Eclient=2EHTTPResponse=2E?=
Message-ID: <E1RY2jx-00035C-HW@dinsdale.python.org>

http://hg.python.org/cpython/rev/806cfe39f729
changeset:   73875:806cfe39f729
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 06 22:33:57 2011 +0100
summary:
  Issue #13464: Add a readinto() method to http.client.HTTPResponse.
Patch by Jon Kuhn.

files:
  Doc/library/http.client.rst |    6 +
  Lib/http/client.py          |  167 +++++++++++++++++------
  Lib/test/test_httplib.py    |   86 ++++++++++++
  Misc/ACKS                   |    1 +
  Misc/NEWS                   |    3 +
  5 files changed, 215 insertions(+), 48 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -502,6 +502,12 @@
 
    Reads and returns the response body, or up to the next *amt* bytes.
 
+.. method:: HTTPResponse.readinto(b)
+
+   Reads up to the next len(b) bytes of the response body into the buffer *b*.
+   Returns the number of bytes read.
+
+   .. versionadded:: 3.3
 
 .. method:: HTTPResponse.getheader(name, default=None)
 
diff --git a/Lib/http/client.py b/Lib/http/client.py
--- a/Lib/http/client.py
+++ b/Lib/http/client.py
@@ -485,11 +485,17 @@
             self.close()
             return b""
 
-        if self.chunked:
-            return self._read_chunked(amt)
+        if amt is not None:
+            # Amount is given, so call base class version
+            # (which is implemented in terms of self.readinto)
+            return super(HTTPResponse, self).read(amt)
+        else:
+            # Amount is not given (unbounded read) so we must check self.length
+            # and self.chunked
 
-        if amt is None:
-            # unbounded read
+            if self.chunked:
+                return self._readall_chunked()
+
             if self.length is None:
                 s = self.fp.read()
             else:
@@ -498,61 +504,49 @@
             self.close()        # we read everything
             return s
 
+    def readinto(self, b):
+        if self.fp is None:
+            return 0
+
+        if self._method == "HEAD":
+            self.close()
+            return 0
+
+        if self.chunked:
+            return self._readinto_chunked(b)
+
         if self.length is not None:
-            if amt > self.length:
+            if len(b) > self.length:
                 # clip the read to the "end of response"
-                amt = self.length
+                b = memoryview(b)[0:self.length]
 
         # we do not use _safe_read() here because this may be a .will_close
         # connection, and the user is reading more bytes than will be provided
         # (for example, reading in 1k chunks)
-        s = self.fp.read(amt)
+        n = self.fp.readinto(b)
         if self.length is not None:
-            self.length -= len(s)
+            self.length -= n
             if not self.length:
                 self.close()
-        return s
+        return n
 
-    def _read_chunked(self, amt):
-        assert self.chunked != _UNKNOWN
-        chunk_left = self.chunk_left
-        value = []
-        while True:
-            if chunk_left is None:
-                line = self.fp.readline(_MAXLINE + 1)
-                if len(line) > _MAXLINE:
-                    raise LineTooLong("chunk size")
-                i = line.find(b";")
-                if i >= 0:
-                    line = line[:i] # strip chunk-extensions
-                try:
-                    chunk_left = int(line, 16)
-                except ValueError:
-                    # close the connection as protocol synchronisation is
-                    # probably lost
-                    self.close()
-                    raise IncompleteRead(b''.join(value))
-                if chunk_left == 0:
-                    break
-            if amt is None:
-                value.append(self._safe_read(chunk_left))
-            elif amt < chunk_left:
-                value.append(self._safe_read(amt))
-                self.chunk_left = chunk_left - amt
-                return b''.join(value)
-            elif amt == chunk_left:
-                value.append(self._safe_read(amt))
-                self._safe_read(2)  # toss the CRLF at the end of the chunk
-                self.chunk_left = None
-                return b''.join(value)
-            else:
-                value.append(self._safe_read(chunk_left))
-                amt -= chunk_left
+    def _read_next_chunk_size(self):
+        # Read the next chunk size from the file
+        line = self.fp.readline(_MAXLINE + 1)
+        if len(line) > _MAXLINE:
+            raise LineTooLong("chunk size")
+        i = line.find(b";")
+        if i >= 0:
+            line = line[:i] # strip chunk-extensions
+        try:
+            return int(line, 16)
+        except ValueError:
+            # close the connection as protocol synchronisation is
+            # probably lost
+            self.close()
+            raise
 
-            # we read the whole chunk, get another
-            self._safe_read(2)      # toss the CRLF at the end of the chunk
-            chunk_left = None
-
+    def _read_and_discard_trailer(self):
         # read and discard trailer up to the CRLF terminator
         ### note: we shouldn't have any trailers!
         while True:
@@ -566,11 +560,72 @@
             if line == b"\r\n":
                 break
 
+    def _readall_chunked(self):
+        assert self.chunked != _UNKNOWN
+        chunk_left = self.chunk_left
+        value = []
+        while True:
+            if chunk_left is None:
+                try:
+                    chunk_left = self._read_next_chunk_size()
+                    if chunk_left == 0:
+                        break
+                except ValueError:
+                    raise IncompleteRead(b''.join(value))
+            value.append(self._safe_read(chunk_left))
+
+            # we read the whole chunk, get another
+            self._safe_read(2)      # toss the CRLF at the end of the chunk
+            chunk_left = None
+
+        self._read_and_discard_trailer()
+
         # we read everything; close the "file"
         self.close()
 
         return b''.join(value)
 
+    def _readinto_chunked(self, b):
+        assert self.chunked != _UNKNOWN
+        chunk_left = self.chunk_left
+
+        total_bytes = 0
+        mvb = memoryview(b)
+        while True:
+            if chunk_left is None:
+                try:
+                    chunk_left = self._read_next_chunk_size()
+                    if chunk_left == 0:
+                        break
+                except ValueError:
+                    raise IncompleteRead(bytes(b[0:total_bytes]))
+                    
+            if len(mvb) < chunk_left:
+                n = self._safe_readinto(mvb)
+                self.chunk_left = chunk_left - n
+                return n
+            elif len(mvb) == chunk_left:
+                n = self._safe_readinto(mvb)
+                self._safe_read(2)  # toss the CRLF at the end of the chunk
+                self.chunk_left = None
+                return n
+            else:
+                temp_mvb = mvb[0:chunk_left]
+                n = self._safe_readinto(temp_mvb)
+                mvb = mvb[n:]
+                total_bytes += n
+
+            # we read the whole chunk, get another
+            self._safe_read(2)      # toss the CRLF at the end of the chunk
+            chunk_left = None
+
+        self._read_and_discard_trailer()
+
+        # we read everything; close the "file"
+        self.close()
+
+        return total_bytes
+
     def _safe_read(self, amt):
         """Read the number of bytes requested, compensating for partial reads.
 
@@ -594,6 +649,22 @@
             amt -= len(chunk)
         return b"".join(s)
 
+    def _safe_readinto(self, b):
+        """Same as _safe_read, but for reading into a buffer."""
+        total_bytes = 0
+        mvb = memoryview(b)
+        while total_bytes < len(b):
+            if MAXAMOUNT < len(mvb):
+                temp_mvb = mvb[0:MAXAMOUNT]
+                n = self.fp.readinto(temp_mvb)
+            else:
+                n = self.fp.readinto(mvb)
+            if not n:
+                raise IncompleteRead(bytes(mvb[0:total_bytes]), len(b))
+            mvb = mvb[n:]
+            total_bytes += n
+        return total_bytes
+
     def fileno(self):
         return self.fp.fileno()
 
diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py
--- a/Lib/test/test_httplib.py
+++ b/Lib/test/test_httplib.py
@@ -158,6 +158,23 @@
         self.assertEqual(resp.read(2), b'xt')
         self.assertTrue(resp.isclosed())
 
+    def test_partial_readintos(self):
+        # if we have a lenght, the system knows when to close itself
+        # same behaviour than when we read the whole thing with read()
+        body = "HTTP/1.1 200 Ok\r\nContent-Length: 4\r\n\r\nText"
+        sock = FakeSocket(body)
+        resp = client.HTTPResponse(sock)
+        resp.begin()
+        b = bytearray(2)
+        n = resp.readinto(b)
+        self.assertEqual(n, 2)
+        self.assertEqual(bytes(b), b'Te')
+        self.assertFalse(resp.isclosed())
+        n = resp.readinto(b)
+        self.assertEqual(n, 2)
+        self.assertEqual(bytes(b), b'xt')
+        self.assertTrue(resp.isclosed())
+
     def test_host_port(self):
         # Check invalid host_port
 
@@ -206,6 +223,21 @@
         if resp.read():
             self.fail("Did not expect response from HEAD request")
 
+    def test_readinto_head(self):
+        # Test that the library doesn't attempt to read any data
+        # from a HEAD request.  (Tickles SF bug #622042.)
+        sock = FakeSocket(
+            'HTTP/1.1 200 OK\r\n'
+            'Content-Length: 14432\r\n'
+            '\r\n',
+            NoEOFStringIO)
+        resp = client.HTTPResponse(sock, method="HEAD")
+        resp.begin()
+        b = bytearray(5)
+        if resp.readinto(b) != 0:
+            self.fail("Did not expect response from HEAD request")
+        self.assertEqual(bytes(b), b'\x00'*5)
+
     def test_send_file(self):
         expected = (b'GET /foo HTTP/1.1\r\nHost: example.com\r\n'
                     b'Accept-Encoding: identity\r\nContent-Length:')
@@ -285,6 +317,40 @@
             finally:
                 resp.close()
 
+    def test_readinto_chunked(self):
+        chunked_start = (
+            'HTTP/1.1 200 OK\r\n'
+            'Transfer-Encoding: chunked\r\n\r\n'
+            'a\r\n'
+            'hello worl\r\n'
+            '1\r\n'
+            'd\r\n'
+        )
+        sock = FakeSocket(chunked_start + '0\r\n')
+        resp = client.HTTPResponse(sock, method="GET")
+        resp.begin()
+        b = bytearray(16)
+        n = resp.readinto(b)
+        self.assertEqual(b[:11], b'hello world')
+        self.assertEqual(n, 11)
+        resp.close()
+
+        for x in ('', 'foo\r\n'):
+            sock = FakeSocket(chunked_start + x)
+            resp = client.HTTPResponse(sock, method="GET")
+            resp.begin()
+            try:
+                b = bytearray(16)
+                n = resp.readinto(b)
+            except client.IncompleteRead as i:
+                self.assertEqual(i.partial, b'hello world')
+                self.assertEqual(repr(i),'IncompleteRead(11 bytes read)')
+                self.assertEqual(str(i),'IncompleteRead(11 bytes read)')
+            else:
+                self.fail('IncompleteRead expected')
+            finally:
+                resp.close()
+
     def test_chunked_head(self):
         chunked_start = (
             'HTTP/1.1 200 OK\r\n'
@@ -302,6 +368,26 @@
         self.assertEqual(resp.reason, 'OK')
         self.assertTrue(resp.isclosed())
 
+    def test_readinto_chunked_head(self):
+        chunked_start = (
+            'HTTP/1.1 200 OK\r\n'
+            'Transfer-Encoding: chunked\r\n\r\n'
+            'a\r\n'
+            'hello world\r\n'
+            '1\r\n'
+            'd\r\n'
+        )
+        sock = FakeSocket(chunked_start + '0\r\n')
+        resp = client.HTTPResponse(sock, method="HEAD")
+        resp.begin()
+        b = bytearray(5)
+        n = resp.readinto(b)
+        self.assertEqual(n, 0)
+        self.assertEqual(bytes(b), b'\x00'*5)
+        self.assertEqual(resp.status, 200)
+        self.assertEqual(resp.reason, 'OK')
+        self.assertTrue(resp.isclosed())
+
     def test_negative_content_length(self):
         sock = FakeSocket(
             'HTTP/1.1 200 OK\r\nContent-Length: -1\r\n\r\nHello\r\n')
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -547,6 +547,7 @@
 Andrej Krpic
 Ivan Krsti?
 Andrew Kuchling
+Jon Kuhn
 Vladimir Kushnir
 Ross Lagerwall
 Cameron Laird
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,9 @@
 Library
 -------
 
+- Issue #13464: Add a readinto() method to http.client.HTTPResponse.  Patch
+  by Jon Kuhn.
+
 - tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
   other than 900k.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec  6 22:39:54 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 06 Dec 2011 22:39:54 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_dangling_whitespace?=
Message-ID: <E1RY2jy-00035G-A3@dinsdale.python.org>

http://hg.python.org/cpython/rev/daaacc0ec584
changeset:   73876:daaacc0ec584
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 06 22:34:36 2011 +0100
summary:
  Fix dangling whitespace

files:
  Lib/http/client.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/http/client.py b/Lib/http/client.py
--- a/Lib/http/client.py
+++ b/Lib/http/client.py
@@ -599,7 +599,7 @@
                         break
                 except ValueError:
                     raise IncompleteRead(bytes(b[0:total_bytes]))
-                    
+
             if len(mvb) < chunk_left:
                 n = self._safe_readinto(mvb)
                 self.chunk_left = chunk_left - n

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Wed Dec  7 05:34:00 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 07 Dec 2011 05:34:00 +0100
Subject: [Python-checkins] Daily reference leaks (daaacc0ec584): sum=0
Message-ID: <E1RY9Ci-0006DR-5f@ap.vmr.nerim.net>

results for daaacc0ec584 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloglUfbkU', '-x']

From python-checkins at python.org  Wed Dec  7 10:14:52 2011
From: python-checkins at python.org (ned.deily)
Date: Wed, 07 Dec 2011 10:14:52 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzg2NDE6?=
 =?utf8?q?_Update_IDLE_3_syntax_coloring_to_recognize_b=22=2E=2E=22_and_no?=
 =?utf8?b?dCB1Ii4uIi4=?=
Message-ID: <E1RYDaW-00072f-6J@dinsdale.python.org>

http://hg.python.org/cpython/rev/3822c8087d70
changeset:   73877:3822c8087d70
branch:      3.2
parent:      73873:b6b4d74b8d42
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 07 01:08:35 2011 -0800
summary:
  Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
(Patch by Tal Einat)

files:
  Lib/idlelib/ColorDelegator.py |  8 ++++----
  Misc/ACKS                     |  1 +
  Misc/NEWS                     |  3 +++
  3 files changed, 8 insertions(+), 4 deletions(-)


diff --git a/Lib/idlelib/ColorDelegator.py b/Lib/idlelib/ColorDelegator.py
--- a/Lib/idlelib/ColorDelegator.py
+++ b/Lib/idlelib/ColorDelegator.py
@@ -20,10 +20,10 @@
     # 1st 'file' colorized normal, 2nd as builtin, 3rd as string
     builtin = r"([^.'\"\\#]\b|^)" + any("BUILTIN", builtinlist) + r"\b"
     comment = any("COMMENT", [r"#[^\n]*"])
-    sqstring = r"(\b[rRuU])?'[^'\\\n]*(\\.[^'\\\n]*)*'?"
-    dqstring = r'(\b[rRuU])?"[^"\\\n]*(\\.[^"\\\n]*)*"?'
-    sq3string = r"(\b[rRuU])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?"
-    dq3string = r'(\b[rRuU])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?'
+    sqstring = r"(\b[rRbB])?'[^'\\\n]*(\\.[^'\\\n]*)*'?"
+    dqstring = r'(\b[rRbB])?"[^"\\\n]*(\\.[^"\\\n]*)*"?'
+    sq3string = r"(\b[rRbB])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?"
+    dq3string = r'(\b[rRbB])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?'
     string = any("STRING", [sq3string, dq3string, sqstring, dqstring])
     return kw + "|" + builtin + "|" + comment + "|" + string +\
            "|" + any("SYNC", [r"\n"])
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -256,6 +256,7 @@
 Rodolpho Eckhardt
 Grant Edwards
 John Ehresman
+Tal Einat
 Eric Eisner
 Andrew Eland
 Julien ?lie
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -90,6 +90,9 @@
 Library
 -------
 
+- Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
+  Patch by Tal Einat.
+
 - tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
   other than 900k.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 10:14:55 2011
From: python-checkins at python.org (ned.deily)
Date: Wed, 07 Dec 2011 10:14:55 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=238641=3A_Update_IDLE_3_syntax_coloring_to_recognize_?=
 =?utf8?b?YiIuLiIgYW5kIG5vdCB1Ii4uIi4=?=
Message-ID: <E1RYDaZ-00072k-7z@dinsdale.python.org>

http://hg.python.org/cpython/rev/e49220f4c31f
changeset:   73878:e49220f4c31f
parent:      73876:daaacc0ec584
parent:      73877:3822c8087d70
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 07 01:12:50 2011 -0800
summary:
  Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
(Patch by Tal Einat)

files:
  Lib/idlelib/ColorDelegator.py |  8 ++++----
  Misc/ACKS                     |  1 +
  Misc/NEWS                     |  3 +++
  3 files changed, 8 insertions(+), 4 deletions(-)


diff --git a/Lib/idlelib/ColorDelegator.py b/Lib/idlelib/ColorDelegator.py
--- a/Lib/idlelib/ColorDelegator.py
+++ b/Lib/idlelib/ColorDelegator.py
@@ -20,10 +20,10 @@
     # 1st 'file' colorized normal, 2nd as builtin, 3rd as string
     builtin = r"([^.'\"\\#]\b|^)" + any("BUILTIN", builtinlist) + r"\b"
     comment = any("COMMENT", [r"#[^\n]*"])
-    sqstring = r"(\b[rRuU])?'[^'\\\n]*(\\.[^'\\\n]*)*'?"
-    dqstring = r'(\b[rRuU])?"[^"\\\n]*(\\.[^"\\\n]*)*"?'
-    sq3string = r"(\b[rRuU])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?"
-    dq3string = r'(\b[rRuU])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?'
+    sqstring = r"(\b[rRbB])?'[^'\\\n]*(\\.[^'\\\n]*)*'?"
+    dqstring = r'(\b[rRbB])?"[^"\\\n]*(\\.[^"\\\n]*)*"?'
+    sq3string = r"(\b[rRbB])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?"
+    dq3string = r'(\b[rRbB])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?'
     string = any("STRING", [sq3string, dq3string, sqstring, dqstring])
     return kw + "|" + builtin + "|" + comment + "|" + string +\
            "|" + any("SYNC", [r"\n"])
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -275,6 +275,7 @@
 John Edmonds
 Grant Edwards
 John Ehresman
+Tal Einat
 Eric Eisner
 Andrew Eland
 Julien ?lie
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,9 @@
 Library
 -------
 
+- Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
+  Patch by Tal Einat.
+
 - Issue #13464: Add a readinto() method to http.client.HTTPResponse.  Patch
   by Jon Kuhn.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 19:16:31 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 07 Dec 2011 19:16:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2311051=3A_Reduce_th?=
 =?utf8?q?e_number_of_syscalls_per_import=2E?=
Message-ID: <E1RYM2h-0007rl-Fd@dinsdale.python.org>

http://hg.python.org/cpython/rev/a541bda2f5e2
changeset:   73879:a541bda2f5e2
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 07 19:16:01 2011 +0100
summary:
  Issue #11051: Reduce the number of syscalls per import.

files:
  Python/import.c |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1944,8 +1944,7 @@
             if (Py_VerboseFlag > 1)
                 PySys_FormatStderr("# trying %R\n", filename);
 
-            if (_Py_stat(filename, &statbuf) == 0 &&         /* it exists */
-                S_ISDIR(statbuf.st_mode))           /* it's a directory */
+            if (_Py_stat(filename, &statbuf) != 0 || S_ISDIR(statbuf.st_mode))
             {
                 Py_DECREF(filename);
                 continue;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 21:51:15 2011
From: python-checkins at python.org (amaury.forgeotdarc)
Date: Wed, 07 Dec 2011 21:51:15 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTQ2?=
 =?utf8?q?=3A_Fixed_an_overflow_issue_that_could_crash_the_intepreter_when?=
Message-ID: <E1RYOSR-0005kh-1g@dinsdale.python.org>

http://hg.python.org/cpython/rev/57de1ad15c54
changeset:   73880:57de1ad15c54
branch:      2.7
parent:      73872:5910c385fab6
user:        Amaury Forgeot d'Arc <amauryfa at gmail.com>
date:        Wed Dec 07 21:46:48 2011 +0100
summary:
  Issue #13546: Fixed an overflow issue that could crash the intepreter when
calling sys.setrecursionlimit((1<<31)-1).

2.7 only.

files:
  Lib/test/test_sys.py |  12 ++++++++++++
  Misc/NEWS            |   3 +++
  Python/errors.c      |   6 ++++--
  3 files changed, 19 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py
--- a/Lib/test/test_sys.py
+++ b/Lib/test/test_sys.py
@@ -224,6 +224,18 @@
         self.assertEqual(sys.getrecursionlimit(), 10000)
         sys.setrecursionlimit(oldlimit)
 
+        self.assertRaises(OverflowError, sys.setrecursionlimit, 1 << 31)
+        try:
+            sys.setrecursionlimit((1 << 31) - 5)
+            try:
+                # issue13546: isinstance(e, ValueError) used to fail
+                # when the recursion limit is close to 1<<31
+                raise ValueError()
+            except ValueError, e:
+                pass
+        finally:
+            sys.setrecursionlimit(oldlimit)
+
     def test_getwindowsversion(self):
         # Raise SkipTest if sys doesn't have getwindowsversion attribute
         test.test_support.get_attribute(sys, "getwindowsversion")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,9 @@
 Core and Builtins
 -----------------
 
+- Issue #13546: Fixed an overflow issue that could crash the intepreter when
+  calling sys.setrecursionlimit((1<<31)-1).
+
 - Issue #13333: The UTF-7 decoder now accepts lone surrogates (the encoder
   already accepts them).
 
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -111,9 +111,11 @@
         PyErr_Fetch(&exception, &value, &tb);
         /* Temporarily bump the recursion limit, so that in the most
            common case PyObject_IsSubclass will not raise a recursion
-           error we have to ignore anyway. */
+           error we have to ignore anyway.  Don't do it when the limit
+           is already insanely high, to avoid overflow */
         reclimit = Py_GetRecursionLimit();
-        Py_SetRecursionLimit(reclimit + 5);
+        if (reclimit < (1 << 30))
+            Py_SetRecursionLimit(reclimit + 5);
         res = PyObject_IsSubclass(err, exc);
         Py_SetRecursionLimit(reclimit);
         /* This function must not fail, so print the error here */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 23:04:09 2011
From: python-checkins at python.org (ezio.melotti)
Date: Wed, 07 Dec 2011 23:04:09 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNTMxOiBhZGQg?=
 =?utf8?q?a_test_for_defaultdict_with_a_non-callable_arg=2E__Patch_by_Mike?=
Message-ID: <E1RYPaz-0004IQ-FZ@dinsdale.python.org>

http://hg.python.org/cpython/rev/a8deeb549e1a
changeset:   73881:a8deeb549e1a
branch:      2.7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Thu Dec 08 00:00:49 2011 +0200
summary:
  #13531: add a test for defaultdict with a non-callable arg.  Patch by Mike Cheng.

files:
  Lib/test/test_defaultdict.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py
--- a/Lib/test/test_defaultdict.py
+++ b/Lib/test/test_defaultdict.py
@@ -171,6 +171,8 @@
         finally:
             os.remove(tfn)
 
+    def test_callable_arg(self):
+        self.assertRaises(TypeError, defaultdict, {})
 
 def test_main():
     test_support.run_unittest(TestDefaultDict)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 23:04:10 2011
From: python-checkins at python.org (ezio.melotti)
Date: Wed, 07 Dec 2011 23:04:10 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNTMxOiBhZGQg?=
 =?utf8?q?a_test_for_defaultdict_with_a_non-callable_arg=2E__Patch_by_Mike?=
Message-ID: <E1RYPb0-0004IU-7E@dinsdale.python.org>

http://hg.python.org/cpython/rev/17ceebc61b65
changeset:   73882:17ceebc61b65
branch:      3.2
parent:      73877:3822c8087d70
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Thu Dec 08 00:02:00 2011 +0200
summary:
  #13531: add a test for defaultdict with a non-callable arg.  Patch by Mike Cheng.

files:
  Lib/test/test_defaultdict.py |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py
--- a/Lib/test/test_defaultdict.py
+++ b/Lib/test/test_defaultdict.py
@@ -172,6 +172,9 @@
         finally:
             os.remove(tfn)
 
+    def test_callable_arg(self):
+        self.assertRaises(TypeError, defaultdict, {})
+
     def test_pickleing(self):
         d = defaultdict(int)
         d[1]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 23:04:10 2011
From: python-checkins at python.org (ezio.melotti)
Date: Wed, 07 Dec 2011 23:04:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313531=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RYPb0-0004IY-Ur@dinsdale.python.org>

http://hg.python.org/cpython/rev/4180308547d9
changeset:   73883:4180308547d9
parent:      73879:a541bda2f5e2
parent:      73882:17ceebc61b65
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Thu Dec 08 00:03:59 2011 +0200
summary:
  #13531: merge with 3.2.

files:
  Lib/test/test_defaultdict.py |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py
--- a/Lib/test/test_defaultdict.py
+++ b/Lib/test/test_defaultdict.py
@@ -172,6 +172,9 @@
         finally:
             os.remove(tfn)
 
+    def test_callable_arg(self):
+        self.assertRaises(TypeError, defaultdict, {})
+
     def test_pickleing(self):
         d = defaultdict(int)
         d[1]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec  7 23:20:03 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 07 Dec 2011 23:20:03 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Followup_to_a541bda2f5e2=3A?=
	=?utf8?q?_Add_a_short_comment=2E?=
Message-ID: <E1RYPqN-0005li-7U@dinsdale.python.org>

http://hg.python.org/cpython/rev/174fbbed8747
changeset:   73884:174fbbed8747
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 07 23:17:58 2011 +0100
summary:
  Followup to a541bda2f5e2: Add a short comment.

files:
  Python/import.c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1946,6 +1946,7 @@
 
             if (_Py_stat(filename, &statbuf) != 0 || S_ISDIR(statbuf.st_mode))
             {
+                /* it doesn't exist, or it's a directory */
                 Py_DECREF(filename);
                 continue;
             }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 00:06:28 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 00:06:28 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_libpython=2Epy=3A_defer_cal?=
 =?utf8?q?l_to_gdb=2Elookup=5Ftype=28=27PyUnicodeObject=27=29?=
Message-ID: <E1RYQZI-0001DS-Bh@dinsdale.python.org>

http://hg.python.org/cpython/rev/50c7ac2fe13c
changeset:   73885:50c7ac2fe13c
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 00:08:22 2011 +0100
summary:
  libpython.py: defer call to gdb.lookup_type('PyUnicodeObject')

The lookup fails at startup if Python is linked to a shared library.

files:
  Tools/gdb/libpython.py |  7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -53,7 +53,8 @@
 _type_unsigned_short_ptr = gdb.lookup_type('unsigned short').pointer()
 _type_unsigned_int_ptr = gdb.lookup_type('unsigned int').pointer()
 
-_is_pep393 = 'data' in [f.name for f in gdb.lookup_type('PyUnicodeObject').target().fields()]
+# value computed later, see PyUnicodeObjectPtr.proxy()
+_is_pep393 = None
 
 SIZEOF_VOID_P = _type_void_ptr.sizeof
 
@@ -1123,6 +1124,10 @@
         return _type_Py_UNICODE.sizeof
 
     def proxyval(self, visited):
+        global _is_pep393
+        if _is_pep393 is None:
+            fields = gdb.lookup_type('PyUnicodeObject').target().fields()
+            _is_pep393 = 'data' in [f.name for f in fields]
         if _is_pep393:
             # Python 3.3 and newer
             may_have_surrogates = False

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 00:32:58 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 00:32:58 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExODg2?=
 =?utf8?q?=3A_workaround_an_OS_bug_=28time_zone_data=29_in_test=5Ftime?=
Message-ID: <E1RYQyw-0003B5-Q1@dinsdale.python.org>

http://hg.python.org/cpython/rev/c143e66e5efe
changeset:   73886:c143e66e5efe
branch:      3.2
parent:      73882:17ceebc61b65
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 00:32:51 2011 +0100
summary:
  Issue #11886: workaround an OS bug (time zone data) in test_time

Australian Eastern Standard Time (UTC+10) is called "EST" (as Eastern Standard
Time, UTC-5) instead of "AEST" on some operating systems (e.g. FreeBSD), which
is wrong. See for example this bug:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810

files:
  Lib/test/test_time.py |  7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
--- a/Lib/test/test_time.py
+++ b/Lib/test/test_time.py
@@ -206,7 +206,12 @@
             environ['TZ'] = victoria
             time.tzset()
             self.assertNotEqual(time.gmtime(xmas2002), time.localtime(xmas2002))
-            self.assertTrue(time.tzname[0] == 'AEST', str(time.tzname[0]))
+
+            # Issue #11886: Australian Eastern Standard Time (UTC+10) is called
+            # "EST" (as Eastern Standard Time, UTC-5) instead of "AEST" on some
+            # operating systems (e.g. FreeBSD), which is wrong. See for example
+            # this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810
+            self.assertIn(time.tzname[0], ('AEST' 'EST'), time.tzname[0])
             self.assertTrue(time.tzname[1] == 'AEDT', str(time.tzname[1]))
             self.assertEqual(len(time.tzname), 2)
             self.assertEqual(time.daylight, 1)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 00:32:59 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 00:32:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=28Merge_3=2E2=29_Issue_=2311886=3A_workaround_an_OS_bug_=28?=
 =?utf8?q?time_zone_data=29_in_test=5Ftime?=
Message-ID: <E1RYQyx-0003B9-LF@dinsdale.python.org>

http://hg.python.org/cpython/rev/c7638be1e430
changeset:   73887:c7638be1e430
parent:      73885:50c7ac2fe13c
parent:      73886:c143e66e5efe
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 00:33:14 2011 +0100
summary:
  (Merge 3.2) Issue #11886: workaround an OS bug (time zone data) in test_time

Australian Eastern Standard Time (UTC+10) is called "EST" (as Eastern Standard
Time, UTC-5) instead of "AEST" on some operating systems (e.g. FreeBSD), which
is wrong. See for example this bug:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810

files:
  Lib/test/test_time.py |  7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
--- a/Lib/test/test_time.py
+++ b/Lib/test/test_time.py
@@ -250,7 +250,12 @@
             environ['TZ'] = victoria
             time.tzset()
             self.assertNotEqual(time.gmtime(xmas2002), time.localtime(xmas2002))
-            self.assertTrue(time.tzname[0] == 'AEST', str(time.tzname[0]))
+
+            # Issue #11886: Australian Eastern Standard Time (UTC+10) is called
+            # "EST" (as Eastern Standard Time, UTC-5) instead of "AEST" on some
+            # operating systems (e.g. FreeBSD), which is wrong. See for example
+            # this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810
+            self.assertIn(time.tzname[0], ('AEST' 'EST'), time.tzname[0])
             self.assertTrue(time.tzname[1] == 'AEDT', str(time.tzname[1]))
             self.assertEqual(len(time.tzname), 2)
             self.assertEqual(time.daylight, 1)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 00:33:03 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 00:33:03 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzExODg2?=
 =?utf8?q?=3A_workaround_an_OS_bug_=28time_zone_data=29_in_test=5Ftime?=
Message-ID: <E1RYQz1-0003BH-21@dinsdale.python.org>

http://hg.python.org/cpython/rev/2bca2cee79a1
changeset:   73888:2bca2cee79a1
branch:      2.7
parent:      73881:a8deeb549e1a
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 00:32:51 2011 +0100
summary:
  Issue #11886: workaround an OS bug (time zone data) in test_time

Australian Eastern Standard Time (UTC+10) is called "EST" (as Eastern Standard
Time, UTC-5) instead of "AEST" on some operating systems (e.g. FreeBSD), which
is wrong. See for example this bug:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810

files:
  Lib/test/test_time.py |  7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
--- a/Lib/test/test_time.py
+++ b/Lib/test/test_time.py
@@ -184,7 +184,12 @@
             environ['TZ'] = victoria
             time.tzset()
             self.assertNotEqual(time.gmtime(xmas2002), time.localtime(xmas2002))
-            self.assertTrue(time.tzname[0] == 'AEST', str(time.tzname[0]))
+
+            # Issue #11886: Australian Eastern Standard Time (UTC+10) is called
+            # "EST" (as Eastern Standard Time, UTC-5) instead of "AEST" on some
+            # operating systems (e.g. FreeBSD), which is wrong. See for example
+            # this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810
+            self.assertIn(time.tzname[0], ('AEST' 'EST'), time.tzname[0])
             self.assertTrue(time.tzname[1] == 'AEDT', str(time.tzname[1]))
             self.assertEqual(len(time.tzname), 2)
             self.assertEqual(time.daylight, 1)

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Dec  8 05:36:56 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 08 Dec 2011 05:36:56 +0100
Subject: [Python-checkins] Daily reference leaks (c7638be1e430): sum=0
Message-ID: <E1RYVj6-0005Xx-3s@ap.vmr.nerim.net>

results for c7638be1e430 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog2HGBjZ', '-x']

From python-checkins at python.org  Thu Dec  8 22:12:09 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 22:12:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_PyUnicode=5FFromWideChar=28?=
 =?utf8?q?=29_and_PyUnicode=5FFromUnicode=28=29_raise_a_ValueError_if_a?=
Message-ID: <E1RYlGD-0003Pf-L3@dinsdale.python.org>

http://hg.python.org/cpython/rev/489ea02ed351
changeset:   73889:489ea02ed351
parent:      73887:c7638be1e430
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 22:14:11 2011 +0100
summary:
  PyUnicode_FromWideChar() and PyUnicode_FromUnicode() raise a ValueError if a
character in not in range [U+0000; U+10ffff].

files:
  Objects/unicodeobject.c |  67 ++++++++++++++--------------
  1 files changed, 34 insertions(+), 33 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -66,6 +66,9 @@
 extern "C" {
 #endif
 
+/* Maximum code point of Unicode 6.0: 0x10ffff (1,114,111) */
+#define MAX_UNICODE 0x10ffff
+
 #ifdef Py_DEBUG
 #  define _PyUnicode_CHECK(op) _PyUnicode_CheckConsistency(op, 0)
 #else
@@ -393,9 +396,7 @@
         }
         else {
             assert(maxchar >= 0x10000);
-            /* FIXME: Issue #13441: on Solaris, localeconv() and strxfrm()
-               return characters outside the range U+0000-U+10FFFF. */
-            /* assert(maxchar <= 0x10FFFF); */
+            assert(maxchar <= MAX_UNICODE);
         }
     }
     return 1;
@@ -1295,36 +1296,37 @@
                         Py_UCS4 *maxchar, Py_ssize_t *num_surrogates)
 {
     const wchar_t *iter;
+    Py_UCS4 ch;
 
     assert(num_surrogates != NULL && maxchar != NULL);
     *num_surrogates = 0;
     *maxchar = 0;
 
     for (iter = begin; iter < end; ) {
-        if (*iter > *maxchar) {
-            *maxchar = *iter;
-#if SIZEOF_WCHAR_T != 2
-            if (*maxchar >= 0x10000)
-                return 0;
-#endif
-        }
 #if SIZEOF_WCHAR_T == 2
         if (Py_UNICODE_IS_HIGH_SURROGATE(iter[0])
             && (iter+1) < end
             && Py_UNICODE_IS_LOW_SURROGATE(iter[1]))
         {
-            Py_UCS4 surrogate_val;
-            surrogate_val = Py_UNICODE_JOIN_SURROGATES(iter[0], iter[1]);
+            ch = Py_UNICODE_JOIN_SURROGATES(iter[0], iter[1]);
             ++(*num_surrogates);
-            if (surrogate_val > *maxchar)
-                *maxchar = surrogate_val;
             iter += 2;
         }
         else
+#endif
+        {
+            ch = *iter;
             iter++;
-#else
-        iter++;
-#endif
+        }
+        if (ch > *maxchar) {
+            *maxchar = ch;
+            if (*maxchar > MAX_UNICODE) {
+                PyErr_Format(PyExc_ValueError,
+                             "character U+%x is not in range [U+0000; U+10ffff]",
+                             ch);
+                return -1;
+            }
+        }
     }
     return 0;
 }
@@ -1669,8 +1671,7 @@
                                 &maxchar, &num_surrogates) == -1)
         return NULL;
 
-    unicode = PyUnicode_New(size - num_surrogates,
-                                                maxchar);
+    unicode = PyUnicode_New(size - num_surrogates, maxchar);
     if (!unicode)
         return NULL;
 
@@ -1808,7 +1809,7 @@
         return 0x10000;
     default:
         assert(0 && "invalid kind");
-        return 0x10ffff;
+        return MAX_UNICODE;
     }
 }
 
@@ -2796,7 +2797,7 @@
 PyUnicode_FromOrdinal(int ordinal)
 {
     PyObject *v;
-    if (ordinal < 0 || ordinal > 0x10ffff) {
+    if (ordinal < 0 || ordinal > MAX_UNICODE) {
         PyErr_SetString(PyExc_ValueError,
                         "chr() arg not in range(0x110000)");
         return NULL;
@@ -3472,7 +3473,7 @@
             four_bytes = PyUnicode_4BYTE_DATA(unicode);
             for (; four_bytes < ucs4_end; ++four_bytes, ++w) {
                 if (*four_bytes > 0xFFFF) {
-                    assert(*four_bytes <= 0x10FFFF);
+                    assert(*four_bytes <= MAX_UNICODE);
                     /* encode surrogate pair in this case */
                     *w++ = Py_UNICODE_HIGH_SURROGATE(*four_bytes);
                     *w   = Py_UNICODE_LOW_SURROGATE(*four_bytes);
@@ -4118,7 +4119,7 @@
         continue;
 encode_char:
         if (ch >= 0x10000) {
-            assert(ch <= 0x10FFFF);
+            assert(ch <= MAX_UNICODE);
 
             /* code first surrogate */
             base64bits += 16;
@@ -4577,7 +4578,7 @@
             }
             ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
                  ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
-            assert ((ch > 0xFFFF) && (ch <= 0x10ffff));
+            assert ((ch > 0xFFFF) && (ch <= MAX_UNICODE));
 
             WRITE_MAYBE_FAIL(i++, ch);
             break;
@@ -4714,7 +4715,7 @@
             }
             ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
                  ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
-            assert ((ch > 0xFFFF) && (ch <= 0x10ffff));
+            assert ((ch > 0xFFFF) && (ch <= MAX_UNICODE));
 
 #if SIZEOF_WCHAR_T == 4
             *p++ = (wchar_t)ch;
@@ -4884,7 +4885,7 @@
             *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
             *p++ = (char)(0x80 | (ch & 0x3f));
         } else /* ch >= 0x10000 */ {
-            assert(ch <= 0x10FFFF);
+            assert(ch <= MAX_UNICODE);
             /* Encode UCS4 Unicode ordinals */
             *p++ = (char)(0xf0 | (ch >> 18));
             *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
@@ -5792,7 +5793,7 @@
                 break;
         store:
             /* when we get here, chr is a 32-bit unicode character */
-            if (chr <= 0x10ffff) {
+            if (chr <= MAX_UNICODE) {
                 WRITECHAR(chr);
             } else {
                 endinpos = s-starts;
@@ -5957,7 +5958,7 @@
 
         /* Map 21-bit characters to '\U00xxxxxx' */
         else if (ch >= 0x10000) {
-            assert(ch <= 0x10FFFF);
+            assert(ch <= MAX_UNICODE);
             *p++ = '\\';
             *p++ = 'U';
             *p++ = Py_hexdigits[(ch >> 28) & 0x0000000F];
@@ -6108,7 +6109,7 @@
             else
                 x += 10 + c - 'A';
         }
-        if (x <= 0x10ffff) {
+        if (x <= MAX_UNICODE) {
             if (unicode_putchar(&v, &outpos, x) < 0)
                 goto onError;
         } else {
@@ -6175,7 +6176,7 @@
         Py_UCS4 ch = PyUnicode_READ(kind, data, pos);
         /* Map 32-bit characters to '\Uxxxxxxxx' */
         if (ch >= 0x10000) {
-            assert(ch <= 0x10FFFF);
+            assert(ch <= MAX_UNICODE);
             *p++ = '\\';
             *p++ = 'U';
             *p++ = Py_hexdigits[(ch >> 28) & 0xf];
@@ -6536,7 +6537,7 @@
                     else if (ch < 1000000)
                         repsize += 2+6+1;
                     else {
-                        assert(ch <= 0x10FFFF);
+                        assert(ch <= MAX_UNICODE);
                         repsize += 2+7+1;
                     }
                 }
@@ -9275,7 +9276,7 @@
     else if (maxchar_new <= 65535)
         maxchar_new = 65535;
     else
-        maxchar_new = 1114111; /* 0x10ffff */
+        maxchar_new = MAX_UNICODE;
 
     if (!maxchar_new && PyUnicode_CheckExact(self)) {
         /* fixfct should return TRUE if it modified the buffer. If
@@ -13059,7 +13060,7 @@
         if (x == -1 && PyErr_Occurred())
             goto onError;
 
-        if (x < 0 || x > 0x10ffff) {
+        if (x < 0 || x > MAX_UNICODE) {
             PyErr_SetString(PyExc_OverflowError,
                             "%c arg not in range(0x110000)");
             return (Py_UCS4) -1;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 22:31:15 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 22:31:15 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExMTQ5?=
 =?utf8?q?=3A_Also_enable_-fwrapv_if_=24CC_is_a_full_path?=
Message-ID: <E1RYlYh-00052d-8X@dinsdale.python.org>

http://hg.python.org/cpython/rev/7efad6256e58
changeset:   73890:7efad6256e58
branch:      3.2
parent:      73886:c143e66e5efe
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 22:20:31 2011 +0100
summary:
  Issue #11149: Also enable -fwrapv if $CC is a full path
or has a trailing version number.

files:
  configure    |  7 ++++---
  configure.in |  7 ++++---
  2 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5498,9 +5498,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -928,9 +928,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 22:31:16 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 22:31:16 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_second_fix_for_issue_=2311149=2E?=
Message-ID: <E1RYlYi-00052i-A9@dinsdale.python.org>

http://hg.python.org/cpython/rev/e48df59af394
changeset:   73891:e48df59af394
parent:      73887:c7638be1e430
parent:      73890:7efad6256e58
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 22:22:58 2011 +0100
summary:
  Merge second fix for issue #11149.

files:
  configure    |  7 ++++---
  configure.in |  7 ++++---
  2 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5450,9 +5450,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -907,9 +907,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 22:31:17 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 22:31:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Backport_second?=
	=?utf8?q?_fix_for_issue_=2311149=2E?=
Message-ID: <E1RYlYj-00052m-3W@dinsdale.python.org>

http://hg.python.org/cpython/rev/9d329adbbb01
changeset:   73892:9d329adbbb01
branch:      2.7
parent:      73888:2bca2cee79a1
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 22:26:06 2011 +0100
summary:
  Backport second fix for issue #11149.

files:
  configure    |  7 ++++---
  configure.in |  7 ++++---
  2 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5413,9 +5413,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -934,9 +934,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 22:31:27 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 22:31:27 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?b?KTogTWVyZ2Uu?=
Message-ID: <E1RYlYt-00052q-7B@dinsdale.python.org>

http://hg.python.org/cpython/rev/090574ed8db1
changeset:   73893:090574ed8db1
parent:      73889:489ea02ed351
parent:      73891:e48df59af394
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 22:30:18 2011 +0100
summary:
  Merge.

files:
  configure    |  7 ++++---
  configure.in |  7 ++++---
  2 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5450,9 +5450,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -907,9 +907,10 @@
         fi
 
         # Clang also needs -fwrapv
-        if test "$CC" = "clang" ; then
-            WRAP="-fwrapv"
-        fi
+        case $CC in
+            *clang*) WRAP="-fwrapv"
+            ;;
+        esac
 
 	case $ac_cv_prog_cc_g in
 	yes)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 23:34:27 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 23:34:27 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTQ3?=
 =?utf8?q?=3A_clean_Lib/=5Fsysconfigdata=2Epy_and_Modules/=5Ftestembed?=
Message-ID: <E1RYmXr-0001tJ-78@dinsdale.python.org>

http://hg.python.org/cpython/rev/8ed2c4d4df62
changeset:   73894:8ed2c4d4df62
branch:      3.2
parent:      73890:7efad6256e58
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 23:25:15 2011 +0100
summary:
  Issue #13547: clean Lib/_sysconfigdata.py and Modules/_testembed

files:
  Makefile.pre.in |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1252,6 +1252,7 @@
 	find build -name 'fficonfig.h' -exec rm -f {} ';' || true
 	find build -name 'fficonfig.py' -exec rm -f {} ';' || true
 	-rm -f Lib/lib2to3/*Grammar*.pickle
+	-rm -f Modules/_testembed
 
 profile-removal:
 	find . -name '*.gc??' -exec rm -f {} ';'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 23:34:28 2011
From: python-checkins at python.org (stefan.krah)
Date: Thu, 08 Dec 2011 23:34:28 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_fix_for_issue_=2313547=2E?=
Message-ID: <E1RYmXs-0001tN-1K@dinsdale.python.org>

http://hg.python.org/cpython/rev/053c95ad09cf
changeset:   73895:053c95ad09cf
parent:      73893:090574ed8db1
parent:      73894:8ed2c4d4df62
user:        Stefan Krah <skrah at bytereef.org>
date:        Thu Dec 08 23:31:40 2011 +0100
summary:
  Merge fix for issue #13547.

files:
  Makefile.pre.in |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1315,6 +1315,8 @@
 	find build -name 'fficonfig.h' -exec rm -f {} ';' || true
 	find build -name 'fficonfig.py' -exec rm -f {} ';' || true
 	-rm -f Lib/lib2to3/*Grammar*.pickle
+	-rm -f Lib/_sysconfigdata.py
+	-rm -f Modules/_testembed
 
 profile-removal:
 	find . -name '*.gc??' -exec rm -f {} ';'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec  8 23:41:00 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 08 Dec 2011 23:41:00 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313441=3A_Log_the_l?=
 =?utf8?q?ocale_when_localeconv=28=29_fails?=
Message-ID: <E1RYmeC-0002XD-5I@dinsdale.python.org>

http://hg.python.org/cpython/rev/93bab8400ca5
changeset:   73896:93bab8400ca5
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 08 23:42:52 2011 +0100
summary:
  Issue #13441: Log the locale when localeconv() fails

files:
  Lib/test/test__locale.py |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py
--- a/Lib/test/test__locale.py
+++ b/Lib/test/test__locale.py
@@ -86,9 +86,13 @@
                 setlocale(LC_CTYPE, loc)
             except Error:
                 continue
+            try:
+                formatting = localeconv()
+            except Exception as err:
+                self.fail("localeconv() failed with %s locale: %s" % (loc, err))
             for lc in ("decimal_point",
                         "thousands_sep"):
-                self.numeric_tester('localeconv', localeconv()[lc], lc, loc)
+                self.numeric_tester('localeconv', formatting[lc], lc, loc)
 
     @unittest.skipUnless(nl_langinfo, "nl_langinfo is not available")
     def test_lc_numeric_basic(self):

-- 
Repository URL: http://hg.python.org/cpython

From tjreedy at udel.edu  Thu Dec  8 22:50:35 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 08 Dec 2011 16:50:35 -0500
Subject: [Python-checkins] cpython: PyUnicode_FromWideChar() and
 PyUnicode_FromUnicode() raise a ValueError if a
In-Reply-To: <E1RYlGD-0003Pf-L3@dinsdale.python.org>
References: <E1RYlGD-0003Pf-L3@dinsdale.python.org>
Message-ID: <4EE1312B.4030904@udel.edu>



On 12/8/2011 4:12 PM, victor.stinner wrote:
> http://hg.python.org/cpython/rev/489ea02ed351
> changeset:   73889:489ea02ed351
> parent:      73887:c7638be1e430
> user:        Victor Stinner<victor.stinner at haypocalc.com>
> date:        Thu Dec 08 22:14:11 2011 +0100
> summary:
>    PyUnicode_FromWideChar() and PyUnicode_FromUnicode() raise a ValueError if a
> character in not in range [U+0000; U+10ffff].
>
> files:
>    Objects/unicodeobject.c |  67 ++++++++++++++--------------
>    1 files changed, 34 insertions(+), 33 deletions(-)
>
>
> diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c

> +/* Maximum code point of Unicode 6.0: 0x10ffff (1,114,111) */
> +#define MAX_UNICODE 0x10ffff

Isn't this the value assigned, on all systems, to sys.maxunicode, in 
3.3? If so, it must already be defined somewhere else.

From python-checkins at python.org  Fri Dec  9 00:08:38 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 00:08:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_What=27s_New_in_Python_3=2E?=
 =?utf8?q?3=3A_Add_a_Deprecated_section?=
Message-ID: <E1RYn4w-0004W0-Kx@dinsdale.python.org>

http://hg.python.org/cpython/rev/0846fc6bf6a8
changeset:   73897:0846fc6bf6a8
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 00:10:41 2011 +0100
summary:
  What's New in Python 3.3: Add a Deprecated section

files:
  Doc/whatsnew/3.3.rst |  9 ++++++---
  1 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -711,8 +711,11 @@
 
 
 
+Deprecated
+==========
+
 Unsupported Operating Systems
-=============================
+-----------------------------
 
 OS/2 and VMS are no longer supported due to the lack of a maintainer.
 
@@ -721,7 +724,7 @@
 
 
 Deprecated Python modules, functions and methods
-================================================
+------------------------------------------------
 
 * The :mod:`packaging` module replaces the :mod:`distutils` module
 * The ``unicode_internal`` codec has been deprecated because of the
@@ -737,7 +740,7 @@
 
 
 Deprecated functions and types of the C API
-===========================================
+-------------------------------------------
 
 The :c:type:`Py_UNICODE` has been deprecated by the :pep:`393` and will be
 removed in Python 4. All functions using this type are deprecated:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 00:16:02 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 00:16:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Document_PyUnicode=5FCopy?=
 =?utf8?q?=28=29_and_PyUnicode=5FEncodeCodePage=28=29?=
Message-ID: <E1RYnC6-0005HV-Ep@dinsdale.python.org>

http://hg.python.org/cpython/rev/e7d94d35506b
changeset:   73898:e7d94d35506b
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 00:18:11 2011 +0100
summary:
  Document PyUnicode_Copy() and PyUnicode_EncodeCodePage()

files:
  Doc/c-api/unicode.rst |  18 +++++++++++++++++-
  1 files changed, 17 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -386,6 +386,13 @@
    .. versionadded:: 3.3
 
 
+.. c:function:: PyObject* PyUnicode_Copy(PyObject *unicode)
+
+   Get a new copy of a Unicode object.
+
+   .. versionadded:: 3.3
+
+
 .. c:function:: PyObject* PyUnicode_FromKindAndData(int kind, const void *buffer, \
                                                     Py_ssize_t size)
 
@@ -1379,6 +1386,15 @@
    raised by the codec.
 
 
+.. c:function:: PyObject* PyUnicode_EncodeCodePage(int code_page, PyObject *unicode, const char *errors)
+
+   Encode the Unicode object using the specified code page and return a Python
+   bytes object.  Return *NULL* if an exception was raised by the codec. Use
+   :c:data:`CP_ACP` code page to get the MBCS encoder.
+
+   .. versionadded:: 3.3
+
+
 .. c:function:: PyObject* PyUnicode_EncodeMBCS(const Py_UNICODE *s, Py_ssize_t size, const char *errors)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using MBCS and return
@@ -1387,7 +1403,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsMBCSString`.
+      :c:func:`PyUnicode_AsMBCSString` or :c:func:`PyUnicode_EncodeCodePage`.
 
 
 Methods & Slots

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 00:34:38 2011
From: python-checkins at python.org (nadeem.vawda)
Date: Fri, 09 Dec 2011 00:34:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_What=27s_New_in_Python_3=2E?=
 =?utf8?q?3=3A_Add_entry_for_lzma_module_=28issue_=236715=29=2E?=
Message-ID: <E1RYnU6-0006fz-H5@dinsdale.python.org>

http://hg.python.org/cpython/rev/66df5ace0eee
changeset:   73899:66df5ace0eee
user:        Nadeem Vawda <nadeem.vawda at gmail.com>
date:        Fri Dec 09 01:32:46 2011 +0200
summary:
  What's New in Python 3.3: Add entry for lzma module (issue #6715).

files:
  Doc/whatsnew/3.3.rst |  10 ++++++++++
  1 files changed, 10 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -391,6 +391,16 @@
 (Contributed by Sijin Joseph in :issue:`8808`)
 
 
+lzma
+----
+
+The newly-added :mod:`lzma` module provides data compression and decompression
+using the LZMA algorithm, including support for the ``.xz`` and ``.lzma``
+file formats.
+
+(Contributed by Nadeem Vawda and Per ?yvind Karlsen in :issue:`6715`)
+
+
 math
 ----
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 01:18:19 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 01:18:19 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313441=3A_Don=27t_t?=
 =?utf8?q?est_the_hu=5FHU_locale_on_Solaris_to_workaround_a_mbstowcs=28=29?=
Message-ID: <E1RYoAN-0001NJ-Pe@dinsdale.python.org>

http://hg.python.org/cpython/rev/87c6be1e393a
changeset:   73900:87c6be1e393a
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 01:20:03 2011 +0100
summary:
  Issue #13441: Don't test the hu_HU locale on Solaris to workaround a mbstowcs()
bug. On Solaris, if the locale is hu_HU (and if the locale encoding is not
UTF-8), the thousauds separator is b'\xA0' which is decoded as U+30000020
instead of U+0020 by mbstowcs().

files:
  Lib/test/test__locale.py |  9 ++++++++-
  1 files changed, 8 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py
--- a/Lib/test/test__locale.py
+++ b/Lib/test/test__locale.py
@@ -17,7 +17,7 @@
 candidate_locales = ['es_UY', 'fr_FR', 'fi_FI', 'es_CO', 'pt_PT', 'it_IT',
     'et_EE', 'es_PY', 'no_NO', 'nl_NL', 'lv_LV', 'el_GR', 'be_BY', 'fr_BE',
     'ro_RO', 'ru_UA', 'ru_RU', 'es_VE', 'ca_ES', 'se_NO', 'es_EC', 'id_ID',
-    'ka_GE', 'es_CL', 'hu_HU', 'wa_BE', 'lt_LT', 'sl_SI', 'hr_HR', 'es_AR',
+    'ka_GE', 'es_CL', 'wa_BE', 'lt_LT', 'sl_SI', 'hr_HR', 'es_AR',
     'es_ES', 'oc_FR', 'gl_ES', 'bg_BG', 'is_IS', 'mk_MK', 'de_AT', 'pt_BR',
     'da_DK', 'nn_NO', 'cs_CZ', 'de_LU', 'es_BO', 'sq_AL', 'sk_SK', 'fr_CH',
     'de_DE', 'sr_YU', 'br_FR', 'nl_BE', 'sv_FI', 'pl_PL', 'fr_CA', 'fo_FO',
@@ -25,6 +25,13 @@
     'eu_ES', 'vi_VN', 'af_ZA', 'nb_NO', 'en_DK', 'tg_TJ', 'en_US',
     'es_ES.ISO8859-1', 'fr_FR.ISO8859-15', 'ru_RU.KOI8-R', 'ko_KR.eucKR']
 
+# Issue #13441: Don't test the hu_HU locale on Solaris to workaround a
+# mbstowcs() bug. On Solaris, if the locale is hu_HU (and if the locale
+# encoding is not UTF-8), the thousauds separator is b'\xA0' which is decoded
+# as U+30000020 instead of U+0020 by mbstowcs().
+if sys.platform != 'sunos5':
+    candidate_locales.append('hu_HU')
+
 # Workaround for MSVC6(debug) crash bug
 if "MSC v.1200" in sys.version:
     def accept(loc):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 04:25:19 2011
From: python-checkins at python.org (jason.coombs)
Date: Fri, 09 Dec 2011 04:25:19 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Implemented_sug?=
 =?utf8?q?gested_improvements_for_pdb_test_by_=C3=89ric_Araujo?=
Message-ID: <E1RYr5L-0006bd-RK@dinsdale.python.org>

http://hg.python.org/cpython/rev/70337a6d5dde
changeset:   73901:70337a6d5dde
branch:      2.7
parent:      73892:9d329adbbb01
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Thu Dec 08 22:14:56 2011 -0500
summary:
  Implemented suggested improvements for pdb test by ?ric Araujo

files:
  Lib/test/test_pdb.py |  35 ++++++++++++++++---------------
  1 files changed, 18 insertions(+), 17 deletions(-)


diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -280,35 +280,36 @@
     4
     """
 
-class Tester7750(unittest.TestCase):
-    # if the filename has something that resolves to a python
-    #  escape character (such as \t), it will fail
-    test_fn = '.\\test7750.py'
+class ModuleInitTester(unittest.TestCase):
 
-    msg = "issue7750 only applies when os.sep is a backslash"
-    @unittest.skipUnless(os.path.sep == '\\', msg)
-    def test_issue7750(self):
-        with open(self.test_fn, 'w') as f:
-            f.write('print("hello world")')
-        cmd = [sys.executable, '-m', 'pdb', self.test_fn,]
+    def test_filename_correct(self):
+        """
+        In issue 7750, it was found that if the filename has a sequence that
+        resolves to an escape character in a Python string (such as \t), it
+        will be treated as the escaped character.
+        """
+        # the test_fn must contain something like \t
+        # on Windows, this will create 'test_mod.py' in the current directory.
+        # on Unix, this will create '.\test_mod.py' in the current directory.
+        test_fn = '.\\test_mod.py'
+        code = 'print("testing pdb")'
+        with open(test_fn, 'w') as f:
+            f.write(code)
+        self.addCleanup(os.remove, test_fn)
+        cmd = [sys.executable, '-m', 'pdb', test_fn,]
         proc = subprocess.Popen(cmd,
             stdout=subprocess.PIPE,
             stdin=subprocess.PIPE,
             stderr=subprocess.STDOUT,
             )
         stdout, stderr = proc.communicate('quit\n')
-        self.assertNotIn('IOError', stdout, "pdb munged the filename")
-
-    def tearDown(self):
-        if os.path.isfile(self.test_fn):
-            os.remove(self.test_fn)
+        self.assertIn(code, stdout, "pdb munged the filename")
 
 
 def test_main():
     from test import test_pdb
     test_support.run_doctest(test_pdb, verbosity=True)
-
+    test_support.run_unittest(ModuleInitTester)
 
 if __name__ == '__main__':
     test_main()
-    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Dec  9 05:36:03 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 09 Dec 2011 05:36:03 +0100
Subject: [Python-checkins] Daily reference leaks (87c6be1e393a): sum=0
Message-ID: <E1RYsBn-0004jZ-G1@ap.vmr.nerim.net>

results for 87c6be1e393a on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogf1EUl2', '-x']

From python-checkins at python.org  Fri Dec  9 10:28:40 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 10:28:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313441=3A_Skip_some?=
 =?utf8?q?_locales_=28e=2Eg=2E_cs=5FCZ_and_hu=5FHU=29_on_Solaris_to_workar?=
 =?utf8?q?ound?=
Message-ID: <E1RYwky-0008B8-7O@dinsdale.python.org>

http://hg.python.org/cpython/rev/2a2d0872d993
changeset:   73902:2a2d0872d993
parent:      73900:87c6be1e393a
user:        Victor Stinner <vstinner at wyplay.com>
date:        Fri Dec 09 10:28:45 2011 +0100
summary:
  Issue #13441: Skip some locales (e.g. cs_CZ and hu_HU) on Solaris to workaround
a mbstowcs() bug. For example, on Solaris, the hu_HU locale uses the locale
encoding ISO-8859-2, the thousauds separator is b'\xA0' and it is decoded as
U+30000020 (an invalid character) by mbstowcs().

The workaround is not enabled yet (commented): I would like first to get
more information about the failing locales.

files:
  Lib/test/test__locale.py |  43 +++++++++++++++++++--------
  1 files changed, 30 insertions(+), 13 deletions(-)


diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py
--- a/Lib/test/test__locale.py
+++ b/Lib/test/test__locale.py
@@ -1,13 +1,15 @@
-from test.support import run_unittest
 from _locale import (setlocale, LC_ALL, LC_CTYPE, LC_NUMERIC, localeconv, Error)
 try:
     from _locale import (RADIXCHAR, THOUSEP, nl_langinfo)
 except ImportError:
     nl_langinfo = None
 
+import codecs
+import locale
+import sys
 import unittest
-import sys
 from platform import uname
+from test.support import run_unittest
 
 if uname()[0] == "Darwin":
     maj, min, mic = [int(part) for part in uname()[2].split(".")]
@@ -17,7 +19,7 @@
 candidate_locales = ['es_UY', 'fr_FR', 'fi_FI', 'es_CO', 'pt_PT', 'it_IT',
     'et_EE', 'es_PY', 'no_NO', 'nl_NL', 'lv_LV', 'el_GR', 'be_BY', 'fr_BE',
     'ro_RO', 'ru_UA', 'ru_RU', 'es_VE', 'ca_ES', 'se_NO', 'es_EC', 'id_ID',
-    'ka_GE', 'es_CL', 'wa_BE', 'lt_LT', 'sl_SI', 'hr_HR', 'es_AR',
+    'ka_GE', 'es_CL', 'wa_BE', 'hu_HU', 'lt_LT', 'sl_SI', 'hr_HR', 'es_AR',
     'es_ES', 'oc_FR', 'gl_ES', 'bg_BG', 'is_IS', 'mk_MK', 'de_AT', 'pt_BR',
     'da_DK', 'nn_NO', 'cs_CZ', 'de_LU', 'es_BO', 'sq_AL', 'sk_SK', 'fr_CH',
     'de_DE', 'sr_YU', 'br_FR', 'nl_BE', 'sv_FI', 'pl_PL', 'fr_CA', 'fo_FO',
@@ -25,12 +27,30 @@
     'eu_ES', 'vi_VN', 'af_ZA', 'nb_NO', 'en_DK', 'tg_TJ', 'en_US',
     'es_ES.ISO8859-1', 'fr_FR.ISO8859-15', 'ru_RU.KOI8-R', 'ko_KR.eucKR']
 
-# Issue #13441: Don't test the hu_HU locale on Solaris to workaround a
-# mbstowcs() bug. On Solaris, if the locale is hu_HU (and if the locale
-# encoding is not UTF-8), the thousauds separator is b'\xA0' which is decoded
-# as U+30000020 instead of U+0020 by mbstowcs().
-if sys.platform != 'sunos5':
-    candidate_locales.append('hu_HU')
+# Issue #13441: Skip some locales (e.g. cs_CZ and hu_HU) on Solaris to
+# workaround a mbstowcs() bug. For example, on Solaris, the hu_HU locale uses
+# the locale encoding ISO-8859-2, the thousauds separator is b'\xA0' and it is
+# decoded as U+30000020 (an invalid character) by mbstowcs().
+if sys.platform == 'sunos5':
+    old_locale = locale.setlocale(locale.LC_ALL)
+    try:
+        locales = []
+        for loc in candidate_locales:
+            try:
+                locale.setlocale(locale.LC_ALL, loc)
+            except Error:
+                continue
+            encoding = locale.getpreferredencoding(False)
+            try:
+                localeconv()
+            except Exception as err:
+                print("WARNING: Skip locale %s (encoding %s): [%s] %s"
+                      % (loc, encoding, type(err), err))
+            else:
+                locales.append(loc)
+        #candidate_locales = locales
+    finally:
+        locale.setlocale(locale.LC_ALL, old_locale)
 
 # Workaround for MSVC6(debug) crash bug
 if "MSC v.1200" in sys.version:
@@ -93,10 +113,7 @@
                 setlocale(LC_CTYPE, loc)
             except Error:
                 continue
-            try:
-                formatting = localeconv()
-            except Exception as err:
-                self.fail("localeconv() failed with %s locale: %s" % (loc, err))
+            formatting = localeconv()
             for lc in ("decimal_point",
                         "thousands_sep"):
                 self.numeric_tester('localeconv', formatting[lc], lc, loc)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 11:29:13 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 11:29:13 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313441=3A_Enable_th?=
 =?utf8?q?e_workaround_for_Solaris_locale_bug?=
Message-ID: <E1RYxhZ-0004dp-I4@dinsdale.python.org>

http://hg.python.org/cpython/rev/7ffe3d304487
changeset:   73903:7ffe3d304487
user:        Victor Stinner <vstinner at wyplay.com>
date:        Fri Dec 09 11:29:44 2011 +0100
summary:
  Issue #13441: Enable the workaround for Solaris locale bug

Skip locales triggering the mbstowcs() bug. I collected the locale list thanks
my previous commit:

 * hu_HU (ISO8859-2): character U+30000020
 * de_AT (ISO8859-1): character U+30000076
 * cs_CZ (ISO8859-2): character U+30000020
 * sk_SK (ISO8859-2): character U+30000020
 * pl_PL (ISO8859-2): character U+30000020
 * fr_CA (ISO8859-1): character U+30000020

files:
  Lib/test/test__locale.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py
--- a/Lib/test/test__locale.py
+++ b/Lib/test/test__locale.py
@@ -48,7 +48,7 @@
                       % (loc, encoding, type(err), err))
             else:
                 locales.append(loc)
-        #candidate_locales = locales
+        candidate_locales = locales
     finally:
         locale.setlocale(locale.LC_ALL, old_locale)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 20:19:11 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 20:19:11 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzU5MDU6?=
 =?utf8?q?_time=2Estrftime=28=29_is_now_using_the_locale_encoding=2C_inste?=
 =?utf8?q?ad_of?=
Message-ID: <E1RZ5yR-0001aw-ID@dinsdale.python.org>

http://hg.python.org/cpython/rev/8620e6901e58
changeset:   73904:8620e6901e58
branch:      3.2
parent:      73894:8ed2c4d4df62
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 20:19:24 2011 +0100
summary:
  Issue #5905: time.strftime() is now using the locale encoding, instead of
UTF-8, if the wcsftime() function is not available.

files:
  Misc/NEWS            |   3 +++
  Modules/timemodule.c |  13 ++++---------
  2 files changed, 7 insertions(+), 9 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -90,6 +90,9 @@
 Library
 -------
 
+- Issue #5905: time.strftime() is now using the locale encoding, instead of
+  UTF-8, if the wcsftime() function is not available.
+
 - Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
   Patch by Tal Einat.
 
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -3,8 +3,6 @@
 #include "Python.h"
 #include "_time.h"
 
-#define TZNAME_ENCODING "utf-8"
-
 #include <ctype.h>
 
 #ifdef HAVE_SYS_TYPES_H
@@ -48,8 +46,6 @@
 #if defined(MS_WINDOWS) && !defined(__BORLANDC__)
 /* Win32 has better clock replacement; we have our own version below. */
 #undef HAVE_CLOCK
-#undef TZNAME_ENCODING
-#define TZNAME_ENCODING "mbcs"
 #endif /* MS_WINDOWS && !defined(__BORLANDC__) */
 
 #if defined(PYOS_OS2)
@@ -502,7 +498,7 @@
     fmt = format;
 #else
     /* Convert the unicode string to an ascii one */
-    format = PyUnicode_AsEncodedString(format_arg, TZNAME_ENCODING, NULL);
+    format = PyUnicode_EncodeFSDefault(format_arg);
     if (format == NULL)
         return NULL;
     fmt = PyBytes_AS_STRING(format);
@@ -546,8 +542,7 @@
 #ifdef HAVE_WCSFTIME
             ret = PyUnicode_FromWideChar(outbuf, buflen);
 #else
-            ret = PyUnicode_Decode(outbuf, buflen,
-                                   TZNAME_ENCODING, NULL);
+            ret = PyUnicode_DecodeFSDefaultAndSize(outbuf, buflen);
 #endif
             PyMem_Free(outbuf);
             break;
@@ -789,8 +784,8 @@
 #endif /* PYOS_OS2 */
 #endif
     PyModule_AddIntConstant(m, "daylight", daylight);
-    otz0 = PyUnicode_Decode(tzname[0], strlen(tzname[0]), TZNAME_ENCODING, NULL);
-    otz1 = PyUnicode_Decode(tzname[1], strlen(tzname[1]), TZNAME_ENCODING, NULL);
+    otz0 = PyUnicode_DecodeFSDefaultAndSize(tzname[0], strlen(tzname[0]));
+    otz1 = PyUnicode_DecodeFSDefaultAndSize(tzname[1], strlen(tzname[1]));
     PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1));
 #else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/
 #ifdef HAVE_STRUCT_TM_TM_ZONE

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 20:19:12 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 20:19:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?b?OiAoTWVyZ2UgMy4yKSBJc3N1ZSAjNTkwNTogdGltZS5zdHJmdGltZSgpIGlzIG5v?=
 =?utf8?q?w_using_the_locale_encoding=2C?=
Message-ID: <E1RZ5yS-0001b0-C6@dinsdale.python.org>

http://hg.python.org/cpython/rev/bee7694988a4
changeset:   73905:bee7694988a4
parent:      73903:7ffe3d304487
parent:      73904:8620e6901e58
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 20:21:17 2011 +0100
summary:
  (Merge 3.2) Issue #5905: time.strftime() is now using the locale encoding,
instead of UTF-8, if the wcsftime() function is not available.

files:
  Misc/NEWS            |   3 +++
  Modules/timemodule.c |  15 ++++-----------
  2 files changed, 7 insertions(+), 11 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,9 @@
 Library
 -------
 
+- Issue #5905: time.strftime() is now using the locale encoding, instead of
+  UTF-8, if the wcsftime() function is not available.
+
 - Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
   Patch by Tal Einat.
 
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -30,12 +30,6 @@
 #endif /* MS_WINDOWS */
 #endif /* !__WATCOMC__ || __QNX__ */
 
-#if defined(HAVE_MBCS)
-#  define TZNAME_ENCODING "mbcs"
-#else
-#  define TZNAME_ENCODING "utf-8"
-#endif
-
 #if defined(PYOS_OS2)
 #define INCL_DOS
 #define INCL_ERRORS
@@ -492,7 +486,7 @@
     fmt = format;
 #else
     /* Convert the unicode string to an ascii one */
-    format = PyUnicode_AsEncodedString(format_arg, TZNAME_ENCODING, NULL);
+    format = PyUnicode_EncodeFSDefault(format_arg);
     if (format == NULL)
         return NULL;
     fmt = PyBytes_AS_STRING(format);
@@ -536,8 +530,7 @@
 #ifdef HAVE_WCSFTIME
             ret = PyUnicode_FromWideChar(outbuf, buflen);
 #else
-            ret = PyUnicode_Decode(outbuf, buflen,
-                                   TZNAME_ENCODING, NULL);
+            ret = PyUnicode_DecodeFSDefaultAndSize(outbuf, buflen);
 #endif
             PyMem_Free(outbuf);
             break;
@@ -769,8 +762,8 @@
 #endif /* PYOS_OS2 */
 #endif
     PyModule_AddIntConstant(m, "daylight", daylight);
-    otz0 = PyUnicode_Decode(tzname[0], strlen(tzname[0]), TZNAME_ENCODING, NULL);
-    otz1 = PyUnicode_Decode(tzname[1], strlen(tzname[1]), TZNAME_ENCODING, NULL);
+    otz0 = PyUnicode_DecodeFSDefaultAndSize(tzname[0], strlen(tzname[0]));
+    otz1 = PyUnicode_DecodeFSDefaultAndSize(tzname[1], strlen(tzname[1]));
     PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1));
 #else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/
 #ifdef HAVE_STRUCT_TM_TM_ZONE

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 20:47:39 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 09 Dec 2011 20:47:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Write_tests_for_invalid_cha?=
	=?utf8?q?racters_=28U+00110000=29?=
Message-ID: <E1RZ6Pz-00049G-Qh@dinsdale.python.org>

http://hg.python.org/cpython/rev/bfa9d1ba36ae
changeset:   73906:bfa9d1ba36ae
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 09 20:49:49 2011 +0100
summary:
  Write tests for invalid characters (U+00110000)

Test the following functions:

 * codecs.raw_unicode_escape_decode()
 * PyUnicode_FromWideChar()
 * PyUnicode_FromUnicode()
 * "unicode_internal" and "unicode_escape" decoders

files:
  Lib/test/test_codecs.py   |  16 ++++++++++++++++
  Modules/_testcapimodule.c |  18 ++++++++++++++++++
  2 files changed, 34 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py
--- a/Lib/test/test_codecs.py
+++ b/Lib/test/test_codecs.py
@@ -1034,6 +1034,16 @@
                                          'deprecated', DeprecationWarning)):
                 self.assertRaises(UnicodeDecodeError, internal.decode,
                                   "unicode_internal")
+        if sys.byteorder == "little":
+            invalid = b"\x00\x00\x11\x00"
+        else:
+            invalid = b"\x00\x11\x00\x00"
+        with support.check_warnings():
+            self.assertRaises(UnicodeDecodeError,
+                              invalid.decode, "unicode_internal")
+        with support.check_warnings():
+            self.assertEqual(invalid.decode("unicode_internal", "replace"),
+                             '\ufffd')
 
     @unittest.skipUnless(SIZEOF_WCHAR_T == 4, 'specific to 32-bit wchar_t')
     def test_decode_error_attributes(self):
@@ -1729,6 +1739,12 @@
         self.assertEqual(codecs.raw_unicode_escape_decode(r"\u1234"), ("\u1234", 6))
         self.assertEqual(codecs.raw_unicode_escape_decode(br"\u1234"), ("\u1234", 6))
 
+        self.assertRaises(UnicodeDecodeError, codecs.unicode_escape_decode, br"\U00110000")
+        self.assertEqual(codecs.unicode_escape_decode(r"\U00110000", "replace"), ("\ufffd", 10))
+
+        self.assertRaises(UnicodeDecodeError, codecs.raw_unicode_escape_decode, br"\U00110000")
+        self.assertEqual(codecs.raw_unicode_escape_decode(r"\U00110000", "replace"), ("\ufffd", 10))
+
 class SurrogateEscapeTest(unittest.TestCase):
 
     def test_utf8(self):
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -1409,6 +1409,7 @@
 #if defined(SIZEOF_WCHAR_T) && (SIZEOF_WCHAR_T == 4)
     const wchar_t wtext[2] = {(wchar_t)0x10ABCDu};
     size_t wtextlen = 1;
+    const wchar_t invalid[1] = {(wchar_t)0x110000u};
 #else
     const wchar_t wtext[3] = {(wchar_t)0xDBEAu, (wchar_t)0xDFCDu};
     size_t wtextlen = 2;
@@ -1444,6 +1445,23 @@
 
     Py_DECREF(wide);
     Py_DECREF(utf8);
+
+#if defined(SIZEOF_WCHAR_T) && (SIZEOF_WCHAR_T == 4)
+    wide = PyUnicode_FromWideChar(invalid, 1);
+    if (wide == NULL)
+        PyErr_Clear();
+    else
+        return raiseTestError("test_widechar",
+                              "PyUnicode_FromWideChar(L\"\\U00110000\", 1) didn't fail");
+
+    wide = PyUnicode_FromUnicode(invalid, 1);
+    if (wide == NULL)
+        PyErr_Clear();
+    else
+        return raiseTestError("test_widechar",
+                              "PyUnicode_FromUnicode(L\"\\U00110000\", 1) didn't fail");
+#endif
+
     Py_RETURN_NONE;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 22:37:18 2011
From: python-checkins at python.org (florent.xicluna)
Date: Fri, 09 Dec 2011 22:37:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Closes_=232979=3A_add_param?=
 =?utf8?q?eter_=27use=5Fbuiltin=5Ftypes=27_to_the_SimpleXMLRPCServer=2E?=
Message-ID: <E1RZ886-0004hN-1p@dinsdale.python.org>

http://hg.python.org/cpython/rev/b3c1a504ebc1
changeset:   73907:b3c1a504ebc1
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Fri Dec 09 22:35:06 2011 +0100
summary:
  Closes #2979: add parameter 'use_builtin_types' to the SimpleXMLRPCServer.

files:
  Doc/library/xmlrpc.server.rst |  28 ++++++++++++++++--
  Lib/test/test_xmlrpc.py       |  34 +++++++++++++++++++++++
  Lib/xmlrpc/server.py          |  25 ++++++++++------
  3 files changed, 73 insertions(+), 14 deletions(-)


diff --git a/Doc/library/xmlrpc.server.rst b/Doc/library/xmlrpc.server.rst
--- a/Doc/library/xmlrpc.server.rst
+++ b/Doc/library/xmlrpc.server.rst
@@ -16,7 +16,9 @@
 :class:`CGIXMLRPCRequestHandler`.
 
 
-.. class:: SimpleXMLRPCServer(addr, requestHandler=SimpleXMLRPCRequestHandler, logRequests=True, allow_none=False, encoding=None, bind_and_activate=True)
+.. class:: SimpleXMLRPCServer(addr, requestHandler=SimpleXMLRPCRequestHandler,\
+               logRequests=True, allow_none=False, encoding=None,\
+               bind_and_activate=True, use_builtin_types=False)
 
    Create a new server instance.  This class provides methods for registration of
    functions that can be called by the XML-RPC protocol.  The *requestHandler*
@@ -25,18 +27,31 @@
    are passed to the :class:`socketserver.TCPServer` constructor.  If *logRequests*
    is true (the default), requests will be logged; setting this parameter to false
    will turn off logging.   The *allow_none* and *encoding* parameters are passed
-   on to  :mod:`xmlrpc.client` and control the XML-RPC responses that will be returned
+   on to :mod:`xmlrpc.client` and control the XML-RPC responses that will be returned
    from the server. The *bind_and_activate* parameter controls whether
    :meth:`server_bind` and :meth:`server_activate` are called immediately by the
    constructor; it defaults to true. Setting it to false allows code to manipulate
    the *allow_reuse_address* class variable before the address is bound.
+   The *use_builtin_types* parameter is passed to the
+   :func:`~xmlrpc.client.loads` function and controls which types are processed
+   when date/times values or binary data are received; it defaults to false.
 
+   .. versionchanged:: 3.3
+      The *use_builtin_types* flag was added.
 
-.. class:: CGIXMLRPCRequestHandler(allow_none=False, encoding=None)
+
+.. class:: CGIXMLRPCRequestHandler(allow_none=False, encoding=None,\
+               use_builtin_types=False)
 
    Create a new instance to handle XML-RPC requests in a CGI environment.  The
    *allow_none* and *encoding* parameters are passed on to :mod:`xmlrpc.client`
    and control the XML-RPC responses that will be returned from the server.
+   The *use_builtin_types* parameter is passed to the
+   :func:`~xmlrpc.client.loads` function and controls which types are processed
+   when date/times values or binary data are received; it defaults to false.
+
+   .. versionchanged:: 3.3
+      The *use_builtin_types* flag was added.
 
 
 .. class:: SimpleXMLRPCRequestHandler()
@@ -233,12 +248,17 @@
 :class:`DocCGIXMLRPCRequestHandler`.
 
 
-.. class:: DocXMLRPCServer(addr, requestHandler=DocXMLRPCRequestHandler, logRequests=True, allow_none=False, encoding=None, bind_and_activate=True)
+.. class:: DocXMLRPCServer(addr, requestHandler=DocXMLRPCRequestHandler,\
+               logRequests=True, allow_none=False, encoding=None,\
+               bind_and_activate=True, use_builtin_types=True)
 
    Create a new server instance. All parameters have the same meaning as for
    :class:`SimpleXMLRPCServer`; *requestHandler* defaults to
    :class:`DocXMLRPCRequestHandler`.
 
+   .. versionchanged:: 3.3
+      The *use_builtin_types* flag was added.
+
 
 .. class:: DocCGIXMLRPCRequestHandler()
 
diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py
--- a/Lib/test/test_xmlrpc.py
+++ b/Lib/test/test_xmlrpc.py
@@ -1023,10 +1023,44 @@
             len(content))
 
 
+class UseBuiltinTypesTestCase(unittest.TestCase):
+
+    def test_use_builtin_types(self):
+        # SimpleXMLRPCDispatcher.__init__ accepts use_builtin_types, which
+        # makes all dispatch of binary data as bytes instances, and all
+        # dispatch of datetime argument as datetime.datetime instances.
+        self.log = []
+        expected_bytes = b"my dog has fleas"
+        expected_date = datetime.datetime(2008, 5, 26, 18, 25, 12)
+        marshaled = xmlrpclib.dumps((expected_bytes, expected_date), 'foobar')
+        def foobar(*args):
+            self.log.extend(args)
+        handler = xmlrpc.server.SimpleXMLRPCDispatcher(
+            allow_none=True, encoding=None, use_builtin_types=True)
+        handler.register_function(foobar)
+        handler._marshaled_dispatch(marshaled)
+        self.assertEqual(len(self.log), 2)
+        mybytes, mydate = self.log
+        self.assertEqual(self.log, [expected_bytes, expected_date])
+        self.assertIs(type(mydate), datetime.datetime)
+        self.assertIs(type(mybytes), bytes)
+
+    def test_cgihandler_has_use_builtin_types_flag(self):
+        handler = xmlrpc.server.CGIXMLRPCRequestHandler(use_builtin_types=True)
+        self.assertTrue(handler.use_builtin_types)
+
+    def test_xmlrpcserver_has_use_builtin_types_flag(self):
+        server = xmlrpc.server.SimpleXMLRPCServer(("localhost", 0),
+            use_builtin_types=True)
+        server.server_close()
+        self.assertTrue(server.use_builtin_types)
+
+
 @support.reap_threads
 def test_main():
     xmlrpc_tests = [XMLRPCTestCase, HelperTestCase, DateTimeTestCase,
          BinaryTestCase, FaultTestCase]
+    xmlrpc_tests.append(UseBuiltinTypesTestCase)
     xmlrpc_tests.append(SimpleServerTestCase)
     xmlrpc_tests.append(KeepaliveServerTestCase1)
     xmlrpc_tests.append(KeepaliveServerTestCase2)
diff --git a/Lib/xmlrpc/server.py b/Lib/xmlrpc/server.py
--- a/Lib/xmlrpc/server.py
+++ b/Lib/xmlrpc/server.py
@@ -160,11 +160,13 @@
     can be instanced when used by the MultiPathXMLRPCServer
     """
 
-    def __init__(self, allow_none=False, encoding=None):
+    def __init__(self, allow_none=False, encoding=None,
+                 use_builtin_types=False):
         self.funcs = {}
         self.instance = None
         self.allow_none = allow_none
         self.encoding = encoding or 'utf-8'
+        self.use_builtin_types = use_builtin_types
 
     def register_instance(self, instance, allow_dotted_names=False):
         """Registers an instance to respond to XML-RPC requests.
@@ -245,7 +247,7 @@
         """
 
         try:
-            params, method = loads(data)
+            params, method = loads(data, use_builtin_types=self.use_builtin_types)
 
             # generate response
             if dispatch_method is not None:
@@ -572,10 +574,11 @@
     _send_traceback_header = False
 
     def __init__(self, addr, requestHandler=SimpleXMLRPCRequestHandler,
-                 logRequests=True, allow_none=False, encoding=None, bind_and_activate=True):
+                 logRequests=True, allow_none=False, encoding=None,
+                 bind_and_activate=True, use_builtin_types=False):
         self.logRequests = logRequests
 
-        SimpleXMLRPCDispatcher.__init__(self, allow_none, encoding)
+        SimpleXMLRPCDispatcher.__init__(self, allow_none, encoding, use_builtin_types)
         socketserver.TCPServer.__init__(self, addr, requestHandler, bind_and_activate)
 
         # [Bug #1222790] If possible, set close-on-exec flag; if a
@@ -595,10 +598,11 @@
     Make sure that the requestHandler accepts the paths in question.
     """
     def __init__(self, addr, requestHandler=SimpleXMLRPCRequestHandler,
-                 logRequests=True, allow_none=False, encoding=None, bind_and_activate=True):
+                 logRequests=True, allow_none=False, encoding=None,
+                 bind_and_activate=True, use_builtin_types=False):
 
         SimpleXMLRPCServer.__init__(self, addr, requestHandler, logRequests, allow_none,
-                                    encoding, bind_and_activate)
+                                    encoding, bind_and_activate, use_builtin_types)
         self.dispatchers = {}
         self.allow_none = allow_none
         self.encoding = encoding or 'utf-8'
@@ -628,8 +632,8 @@
 class CGIXMLRPCRequestHandler(SimpleXMLRPCDispatcher):
     """Simple handler for XML-RPC data passed through CGI."""
 
-    def __init__(self, allow_none=False, encoding=None):
-        SimpleXMLRPCDispatcher.__init__(self, allow_none, encoding)
+    def __init__(self, allow_none=False, encoding=None, use_builtin_types=False):
+        SimpleXMLRPCDispatcher.__init__(self, allow_none, encoding, use_builtin_types)
 
     def handle_xmlrpc(self, request_text):
         """Handle a single XML-RPC request"""
@@ -924,9 +928,10 @@
 
     def __init__(self, addr, requestHandler=DocXMLRPCRequestHandler,
                  logRequests=True, allow_none=False, encoding=None,
-                 bind_and_activate=True):
+                 bind_and_activate=True, use_builtin_types=False):
         SimpleXMLRPCServer.__init__(self, addr, requestHandler, logRequests,
-                                    allow_none, encoding, bind_and_activate)
+                                    allow_none, encoding, bind_and_activate,
+                                    use_builtin_types)
         XMLRPCDocGenerator.__init__(self)
 
 class DocCGIXMLRPCRequestHandler(   CGIXMLRPCRequestHandler,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:16:42 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 09 Dec 2011 23:16:42 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTI4?=
 =?utf8?q?=3A_rework_the_performance_question_in_the_programming_FAQ?=
Message-ID: <E1RZ8kE-0008AR-NN@dinsdale.python.org>

http://hg.python.org/cpython/rev/eb30f2becb79
changeset:   73908:eb30f2becb79
branch:      3.2
parent:      73904:8620e6901e58
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 09 23:10:31 2011 +0100
summary:
  Issue #13528: rework the performance question in the programming FAQ

files:
  Doc/faq/programming.rst |  214 ++++++++-------------------
  1 files changed, 62 insertions(+), 152 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -115,159 +115,6 @@
 :pep:`8`.
 
 
-My program is too slow. How do I speed it up?
----------------------------------------------
-
-That's a tough one, in general.  There are many tricks to speed up Python code;
-consider rewriting parts in C as a last resort.
-
-`Cython <http://cython.org>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
-can compile a slightly modified version of Python code into a C extension, and
-can be used on many different platforms.  Depending on your code, Cython
-may be able to make it significantly faster than when run by the Python
-interpreter.
-
-The rest of this answer will discuss various tricks for squeezing a bit more
-speed out of Python code.  *Never* apply any optimization tricks unless you know
-you need them, after profiling has indicated that a particular function is the
-heavily executed hot spot in the code.  Optimizations almost always make the
-code less clear, and you shouldn't pay the costs of reduced clarity (increased
-development time, greater likelihood of bugs) unless the resulting performance
-benefit is worth it.
-
-There is a page on the wiki devoted to `performance tips
-<http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_.
-
-Guido van Rossum has written up an anecdote related to optimization at
-http://www.python.org/doc/essays/list2str.html.
-
-One thing to notice is that function and (especially) method calls are rather
-expensive; if you have designed a purely OO interface with lots of tiny
-functions that don't do much more than get or set an instance variable or call
-another method, you might consider using a more direct way such as directly
-accessing instance variables.  Also see the standard module :mod:`profile` which
-makes it possible to find out where your program is spending most of its time
-(if you have some patience -- the profiling itself can slow your program down by
-an order of magnitude).
-
-Remember that many standard optimization heuristics you may know from other
-programming experience may well apply to Python.  For example it may be faster
-to send output to output devices using larger writes rather than smaller ones in
-order to reduce the overhead of kernel system calls.  Thus CGI scripts that
-write all output in "one shot" may be faster than those that write lots of small
-pieces of output.
-
-Also, be sure to use Python's core features where appropriate.  For example,
-slicing allows programs to chop up lists and other sequence objects in a single
-tick of the interpreter's mainloop using highly optimized C implementations.
-Thus to get the same effect as::
-
-   L2 = []
-   for i in range(3):
-       L2.append(L1[i])
-
-it is much shorter and far faster to use ::
-
-   L2 = list(L1[:3])  # "list" is redundant if L1 is a list.
-
-Note that the functionally-oriented built-in functions such as :func:`map`,
-:func:`zip`, and friends can be a convenient accelerator for loops that
-perform a single task.  For example to pair the elements of two lists
-together::
-
-   >>> list(zip([1, 2, 3], [4, 5, 6]))
-   [(1, 4), (2, 5), (3, 6)]
-
-or to compute a number of sines::
-
-   >>> list(map(math.sin, (1, 2, 3, 4)))
-   [0.841470984808, 0.909297426826, 0.14112000806, -0.756802495308]
-
-The operation completes very quickly in such cases.
-
-Other examples include the ``join()`` and ``split()`` :ref:`methods
-of string objects <string-methods>`.
-
-For example if s1..s7 are large (10K+) strings then
-``"".join([s1,s2,s3,s4,s5,s6,s7])`` may be far faster than the more obvious
-``s1+s2+s3+s4+s5+s6+s7``, since the "summation" will compute many
-subexpressions, whereas ``join()`` does all the copying in one pass.  For
-manipulating strings, use the ``replace()`` and the ``format()`` :ref:`methods
-on string objects <string-methods>`.  Use regular expressions only when you're
-not dealing with constant string patterns.
-
-Be sure to use the :meth:`list.sort` built-in method to do sorting, and see the
-`sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples
-of moderately advanced usage.  :meth:`list.sort` beats other techniques for
-sorting in all but the most extreme circumstances.
-
-Another common trick is to "push loops into functions or methods."  For example
-suppose you have a program that runs slowly and you use the profiler to
-determine that a Python function ``ff()`` is being called lots of times.  If you
-notice that ``ff()``::
-
-   def ff(x):
-       ... # do something with x computing result...
-       return result
-
-tends to be called in loops like::
-
-   list = map(ff, oldlist)
-
-or::
-
-   for x in sequence:
-       value = ff(x)
-       ... # do something with value...
-
-then you can often eliminate function call overhead by rewriting ``ff()`` to::
-
-   def ffseq(seq):
-       resultseq = []
-       for x in seq:
-           ... # do something with x computing result...
-           resultseq.append(result)
-       return resultseq
-
-and rewrite the two examples to ``list = ffseq(oldlist)`` and to::
-
-   for value in ffseq(sequence):
-       ... # do something with value...
-
-Single calls to ``ff(x)`` translate to ``ffseq([x])[0]`` with little penalty.
-Of course this technique is not always appropriate and there are other variants
-which you can figure out.
-
-You can gain some performance by explicitly storing the results of a function or
-method lookup into a local variable.  A loop like::
-
-   for key in token:
-       dict[key] = dict.get(key, 0) + 1
-
-resolves ``dict.get`` every iteration.  If the method isn't going to change, a
-slightly faster implementation is::
-
-   dict_get = dict.get  # look up the method once
-   for key in token:
-       dict[key] = dict_get(key, 0) + 1
-
-Default arguments can be used to determine values once, at compile time instead
-of at run time.  This can only be done for functions or objects which will not
-be changed during program execution, such as replacing ::
-
-   def degree_sin(deg):
-       return math.sin(deg * math.pi / 180.0)
-
-with ::
-
-   def degree_sin(deg, factor=math.pi/180.0, sin=math.sin):
-       return sin(deg * factor)
-
-Because this trick uses default arguments for terms which should not be changed,
-it should only be used when you are not concerned with presenting a possibly
-confusing API to your users.
-
-
 Core Language
 =============
 
@@ -938,6 +785,68 @@
 See the :ref:`unicode-howto`.
 
 
+Performance
+===========
+
+My program is too slow. How do I speed it up?
+---------------------------------------------
+
+That's a tough one, in general.  First, here are a list of things to
+remember before diving further:
+
+* Performance characteristics vary accross Python implementations.  This FAQ
+  focusses on :term:`CPython`.
+* Behaviour can vary accross operating systems, especially when talking about
+  I/O or multi-threading.
+* You should always find the hot spots in your program *before* attempting to
+  optimize any code (see the :mod:`profile` module).
+* Writing benchmark scripts will allow you to iterate quickly when searching
+  for improvements (see the :mod:`timeit` module).
+* It is highly recommended to have good code coverage (through unit testing
+  or any other technique) before potentially introducing regressions hidden
+  in sophisticated optimizations.
+
+That being said, there are many tricks to speed up Python code.  Here are
+some general principles which go a long way towards reaching acceptable
+performance levels:
+
+* Making your algorithms faster (or changing to faster ones) can yield
+  much larger benefits than trying to sprinkle micro-optimization tricks
+  all over your code.
+
+* Use the right data structures.  Study documentation for the :ref:`bltin-types`
+  and the :mod:`collections` module.
+
+* When the standard library provides a primitive for doing something, it is
+  likely (although not guaranteed) to be faster than any alternative you
+  may come up with.  This is doubly true for primitives written in C, such
+  as builtins and some extension types.  For example, be sure to use
+  either the :meth:`list.sort` built-in method or the related :func:`sorted`
+  function to do sorting (and see the
+  `sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples
+  of moderately advanced usage).
+
+* Abstractions tend to create indirections and force the interpreter to work
+  more.  If the levels of indirection outweigh the amount of useful work
+  done, your program will be slower.  You should avoid excessive abstraction,
+  especially under the form of tiny functions or methods (which are also often
+  detrimental to readability).
+
+If you have reached the limit of what pure Python can allow, there are tools
+to take you further away.  For example, `Cython <http://cython.org>`_ can
+compile a slightly modified version of Python code into a C extension, and
+can be used on many different platforms.  Cython can take advantage of
+compilation (and optional type annotations) to make your code significantly
+faster than when interpreted.  If you are confident in your C programming
+skills, you can also :ref:`write a C extension module <extending-index>`
+yourself.
+
+.. seealso::
+   The wiki page devoted to `performance tips
+   <http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_.
+
+.. _efficient_string_concatenation:
+
 What is the most efficient way to concatenate many strings together?
 --------------------------------------------------------------------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:16:43 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 09 Dec 2011 23:16:43 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313528=3A_rework_the_performance_question_in_the_pro?=
 =?utf8?q?gramming_FAQ?=
Message-ID: <E1RZ8kF-0008AW-F4@dinsdale.python.org>

http://hg.python.org/cpython/rev/9fe28f52eaaa
changeset:   73909:9fe28f52eaaa
parent:      73907:b3c1a504ebc1
parent:      73908:eb30f2becb79
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 09 23:11:16 2011 +0100
summary:
  Issue #13528: rework the performance question in the programming FAQ

files:
  Doc/faq/programming.rst |  214 ++++++++-------------------
  1 files changed, 62 insertions(+), 152 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -115,159 +115,6 @@
 :pep:`8`.
 
 
-My program is too slow. How do I speed it up?
----------------------------------------------
-
-That's a tough one, in general.  There are many tricks to speed up Python code;
-consider rewriting parts in C as a last resort.
-
-`Cython <http://cython.org>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
-can compile a slightly modified version of Python code into a C extension, and
-can be used on many different platforms.  Depending on your code, Cython
-may be able to make it significantly faster than when run by the Python
-interpreter.
-
-The rest of this answer will discuss various tricks for squeezing a bit more
-speed out of Python code.  *Never* apply any optimization tricks unless you know
-you need them, after profiling has indicated that a particular function is the
-heavily executed hot spot in the code.  Optimizations almost always make the
-code less clear, and you shouldn't pay the costs of reduced clarity (increased
-development time, greater likelihood of bugs) unless the resulting performance
-benefit is worth it.
-
-There is a page on the wiki devoted to `performance tips
-<http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_.
-
-Guido van Rossum has written up an anecdote related to optimization at
-http://www.python.org/doc/essays/list2str.html.
-
-One thing to notice is that function and (especially) method calls are rather
-expensive; if you have designed a purely OO interface with lots of tiny
-functions that don't do much more than get or set an instance variable or call
-another method, you might consider using a more direct way such as directly
-accessing instance variables.  Also see the standard module :mod:`profile` which
-makes it possible to find out where your program is spending most of its time
-(if you have some patience -- the profiling itself can slow your program down by
-an order of magnitude).
-
-Remember that many standard optimization heuristics you may know from other
-programming experience may well apply to Python.  For example it may be faster
-to send output to output devices using larger writes rather than smaller ones in
-order to reduce the overhead of kernel system calls.  Thus CGI scripts that
-write all output in "one shot" may be faster than those that write lots of small
-pieces of output.
-
-Also, be sure to use Python's core features where appropriate.  For example,
-slicing allows programs to chop up lists and other sequence objects in a single
-tick of the interpreter's mainloop using highly optimized C implementations.
-Thus to get the same effect as::
-
-   L2 = []
-   for i in range(3):
-       L2.append(L1[i])
-
-it is much shorter and far faster to use ::
-
-   L2 = list(L1[:3])  # "list" is redundant if L1 is a list.
-
-Note that the functionally-oriented built-in functions such as :func:`map`,
-:func:`zip`, and friends can be a convenient accelerator for loops that
-perform a single task.  For example to pair the elements of two lists
-together::
-
-   >>> list(zip([1, 2, 3], [4, 5, 6]))
-   [(1, 4), (2, 5), (3, 6)]
-
-or to compute a number of sines::
-
-   >>> list(map(math.sin, (1, 2, 3, 4)))
-   [0.841470984808, 0.909297426826, 0.14112000806, -0.756802495308]
-
-The operation completes very quickly in such cases.
-
-Other examples include the ``join()`` and ``split()`` :ref:`methods
-of string objects <string-methods>`.
-
-For example if s1..s7 are large (10K+) strings then
-``"".join([s1,s2,s3,s4,s5,s6,s7])`` may be far faster than the more obvious
-``s1+s2+s3+s4+s5+s6+s7``, since the "summation" will compute many
-subexpressions, whereas ``join()`` does all the copying in one pass.  For
-manipulating strings, use the ``replace()`` and the ``format()`` :ref:`methods
-on string objects <string-methods>`.  Use regular expressions only when you're
-not dealing with constant string patterns.
-
-Be sure to use the :meth:`list.sort` built-in method to do sorting, and see the
-`sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples
-of moderately advanced usage.  :meth:`list.sort` beats other techniques for
-sorting in all but the most extreme circumstances.
-
-Another common trick is to "push loops into functions or methods."  For example
-suppose you have a program that runs slowly and you use the profiler to
-determine that a Python function ``ff()`` is being called lots of times.  If you
-notice that ``ff()``::
-
-   def ff(x):
-       ... # do something with x computing result...
-       return result
-
-tends to be called in loops like::
-
-   list = map(ff, oldlist)
-
-or::
-
-   for x in sequence:
-       value = ff(x)
-       ... # do something with value...
-
-then you can often eliminate function call overhead by rewriting ``ff()`` to::
-
-   def ffseq(seq):
-       resultseq = []
-       for x in seq:
-           ... # do something with x computing result...
-           resultseq.append(result)
-       return resultseq
-
-and rewrite the two examples to ``list = ffseq(oldlist)`` and to::
-
-   for value in ffseq(sequence):
-       ... # do something with value...
-
-Single calls to ``ff(x)`` translate to ``ffseq([x])[0]`` with little penalty.
-Of course this technique is not always appropriate and there are other variants
-which you can figure out.
-
-You can gain some performance by explicitly storing the results of a function or
-method lookup into a local variable.  A loop like::
-
-   for key in token:
-       dict[key] = dict.get(key, 0) + 1
-
-resolves ``dict.get`` every iteration.  If the method isn't going to change, a
-slightly faster implementation is::
-
-   dict_get = dict.get  # look up the method once
-   for key in token:
-       dict[key] = dict_get(key, 0) + 1
-
-Default arguments can be used to determine values once, at compile time instead
-of at run time.  This can only be done for functions or objects which will not
-be changed during program execution, such as replacing ::
-
-   def degree_sin(deg):
-       return math.sin(deg * math.pi / 180.0)
-
-with ::
-
-   def degree_sin(deg, factor=math.pi/180.0, sin=math.sin):
-       return sin(deg * factor)
-
-Because this trick uses default arguments for terms which should not be changed,
-it should only be used when you are not concerned with presenting a possibly
-confusing API to your users.
-
-
 Core Language
 =============
 
@@ -938,6 +785,68 @@
 See the :ref:`unicode-howto`.
 
 
+Performance
+===========
+
+My program is too slow. How do I speed it up?
+---------------------------------------------
+
+That's a tough one, in general.  First, here are a list of things to
+remember before diving further:
+
+* Performance characteristics vary accross Python implementations.  This FAQ
+  focusses on :term:`CPython`.
+* Behaviour can vary accross operating systems, especially when talking about
+  I/O or multi-threading.
+* You should always find the hot spots in your program *before* attempting to
+  optimize any code (see the :mod:`profile` module).
+* Writing benchmark scripts will allow you to iterate quickly when searching
+  for improvements (see the :mod:`timeit` module).
+* It is highly recommended to have good code coverage (through unit testing
+  or any other technique) before potentially introducing regressions hidden
+  in sophisticated optimizations.
+
+That being said, there are many tricks to speed up Python code.  Here are
+some general principles which go a long way towards reaching acceptable
+performance levels:
+
+* Making your algorithms faster (or changing to faster ones) can yield
+  much larger benefits than trying to sprinkle micro-optimization tricks
+  all over your code.
+
+* Use the right data structures.  Study documentation for the :ref:`bltin-types`
+  and the :mod:`collections` module.
+
+* When the standard library provides a primitive for doing something, it is
+  likely (although not guaranteed) to be faster than any alternative you
+  may come up with.  This is doubly true for primitives written in C, such
+  as builtins and some extension types.  For example, be sure to use
+  either the :meth:`list.sort` built-in method or the related :func:`sorted`
+  function to do sorting (and see the
+  `sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples
+  of moderately advanced usage).
+
+* Abstractions tend to create indirections and force the interpreter to work
+  more.  If the levels of indirection outweigh the amount of useful work
+  done, your program will be slower.  You should avoid excessive abstraction,
+  especially under the form of tiny functions or methods (which are also often
+  detrimental to readability).
+
+If you have reached the limit of what pure Python can allow, there are tools
+to take you further away.  For example, `Cython <http://cython.org>`_ can
+compile a slightly modified version of Python code into a C extension, and
+can be used on many different platforms.  Cython can take advantage of
+compilation (and optional type annotations) to make your code significantly
+faster than when interpreted.  If you are confident in your C programming
+skills, you can also :ref:`write a C extension module <extending-index>`
+yourself.
+
+.. seealso::
+   The wiki page devoted to `performance tips
+   <http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_.
+
+.. _efficient_string_concatenation:
+
 What is the most efficient way to concatenate many strings together?
 --------------------------------------------------------------------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:42:43 2011
From: python-checkins at python.org (florent.xicluna)
Date: Fri, 09 Dec 2011 23:42:43 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Fix_docstring_t?=
	=?utf8?q?ypo=2E?=
Message-ID: <E1RZ99P-0001oA-DQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/78a7d5f8f054
changeset:   73910:78a7d5f8f054
branch:      2.7
parent:      73901:70337a6d5dde
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Fri Dec 09 23:40:27 2011 +0100
summary:
  Fix docstring typo.

files:
  Modules/arraymodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -2050,7 +2050,7 @@
 \n\
 Return a new array whose items are restricted by typecode, and\n\
 initialized from the optional initializer value, which must be a list,\n\
-string. or iterable over elements of the appropriate type.\n\
+string or iterable over elements of the appropriate type.\n\
 \n\
 Arrays represent basic values and behave very much like lists, except\n\
 the type of objects stored in them is constrained.\n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:42:44 2011
From: python-checkins at python.org (florent.xicluna)
Date: Fri, 09 Dec 2011 23:42:44 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_docstring_t?=
	=?utf8?q?ypo=2E?=
Message-ID: <E1RZ99Q-0001oL-6T@dinsdale.python.org>

http://hg.python.org/cpython/rev/1e2880250610
changeset:   73911:1e2880250610
branch:      3.2
parent:      73908:eb30f2becb79
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Fri Dec 09 23:41:19 2011 +0100
summary:
  Fix docstring typo.

files:
  Modules/arraymodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -2543,7 +2543,7 @@
 \n\
 Return a new array whose items are restricted by typecode, and\n\
 initialized from the optional initializer value, which must be a list,\n\
-string. or iterable over elements of the appropriate type.\n\
+string or iterable over elements of the appropriate type.\n\
 \n\
 Arrays represent basic values and behave very much like lists, except\n\
 the type of objects stored in them is constrained.\n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:42:44 2011
From: python-checkins at python.org (florent.xicluna)
Date: Fri, 09 Dec 2011 23:42:44 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Remove_obsolete?=
	=?utf8?q?_py3k_comment=2E?=
Message-ID: <E1RZ99Q-0001oP-UN@dinsdale.python.org>

http://hg.python.org/cpython/rev/ee0c9ca09c84
changeset:   73912:ee0c9ca09c84
branch:      3.2
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Fri Dec 09 23:41:21 2011 +0100
summary:
  Remove obsolete py3k comment.

files:
  Python/_warnings.c |  1 -
  1 files changed, 0 insertions(+), 1 deletions(-)


diff --git a/Python/_warnings.c b/Python/_warnings.c
--- a/Python/_warnings.c
+++ b/Python/_warnings.c
@@ -888,7 +888,6 @@
 static PyObject *
 init_filters(void)
 {
-    /* Don't silence DeprecationWarning if -3 was used. */
     PyObject *filters = PyList_New(5);
     unsigned int pos = 0;  /* Post-incremented in each use. */
     unsigned int x;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec  9 23:42:45 2011
From: python-checkins at python.org (florent.xicluna)
Date: Fri, 09 Dec 2011 23:42:45 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_3=2E2?=
Message-ID: <E1RZ99R-0001oT-Ls@dinsdale.python.org>

http://hg.python.org/cpython/rev/b7c5e30582d1
changeset:   73913:b7c5e30582d1
parent:      73909:9fe28f52eaaa
parent:      73912:ee0c9ca09c84
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Fri Dec 09 23:42:29 2011 +0100
summary:
  Merge 3.2

files:
  Modules/arraymodule.c |  2 +-
  Python/_warnings.c    |  1 -
  2 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -2619,7 +2619,7 @@
 \n\
 Return a new array whose items are restricted by typecode, and\n\
 initialized from the optional initializer value, which must be a list,\n\
-string. or iterable over elements of the appropriate type.\n\
+string or iterable over elements of the appropriate type.\n\
 \n\
 Arrays represent basic values and behave very much like lists, except\n\
 the type of objects stored in them is constrained.\n\
diff --git a/Python/_warnings.c b/Python/_warnings.c
--- a/Python/_warnings.c
+++ b/Python/_warnings.c
@@ -895,7 +895,6 @@
 static PyObject *
 init_filters(void)
 {
-    /* Don't silence DeprecationWarning if -3 was used. */
     PyObject *filters = PyList_New(5);
     unsigned int pos = 0;  /* Post-incremented in each use. */
     unsigned int x;

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Dec 10 05:34:30 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 10 Dec 2011 05:34:30 +0100
Subject: [Python-checkins] Daily reference leaks (b7c5e30582d1): sum=0
Message-ID: <E1RZEdq-0004Su-2i@ap.vmr.nerim.net>

results for b7c5e30582d1 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogwfwWLq', '-x']

From python-checkins at python.org  Sat Dec 10 11:08:22 2011
From: python-checkins at python.org (florent.xicluna)
Date: Sat, 10 Dec 2011 11:08:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313248=3A_turn_3=2E?=
 =?utf8?q?2=27s_PendingDeprecationWarning_into_3=2E3=27s?=
Message-ID: <E1RZJqw-0000G0-Lg@dinsdale.python.org>

http://hg.python.org/cpython/rev/f82ebf9b3a52
changeset:   73914:f82ebf9b3a52
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Sat Dec 10 11:07:42 2011 +0100
summary:
  Issue #13248: turn 3.2's PendingDeprecationWarning into 3.3's DeprecationWarning (cgi, importlib, nntplib, smtpd).

files:
  Doc/library/nntplib.rst                      |   3 +
  Lib/cgi.py                                   |   2 +-
  Lib/importlib/abc.py                         |   4 +-
  Lib/importlib/test/source/test_abc_loader.py |   4 +-
  Lib/nntplib.py                               |   4 +-
  Lib/smtpd.py                                 |  44 +++++-----
  Lib/test/test_smtpd.py                       |  44 +++++-----
  Misc/NEWS                                    |   5 +
  8 files changed, 59 insertions(+), 51 deletions(-)


diff --git a/Doc/library/nntplib.rst b/Doc/library/nntplib.rst
--- a/Doc/library/nntplib.rst
+++ b/Doc/library/nntplib.rst
@@ -517,6 +517,9 @@
    article with message ID *id*.  Most of the time, this extension is not
    enabled by NNTP server administrators.
 
+   .. deprecated:: 3.3
+      The XPATH extension is not actively used.
+
 
 .. XXX deprecated:
 
diff --git a/Lib/cgi.py b/Lib/cgi.py
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -1012,7 +1012,7 @@
 def escape(s, quote=None):
     """Deprecated API."""
     warn("cgi.escape is deprecated, use html.escape instead",
-         PendingDeprecationWarning, stacklevel=2)
+         DeprecationWarning, stacklevel=2)
     s = s.replace("&", "&amp;") # Must be done first!
     s = s.replace("<", "&lt;")
     s = s.replace(">", "&gt;")
diff --git a/Lib/importlib/abc.py b/Lib/importlib/abc.py
--- a/Lib/importlib/abc.py
+++ b/Lib/importlib/abc.py
@@ -195,7 +195,7 @@
                             "use SourceLoader instead. "
                             "See the importlib documentation on how to be "
                             "compatible with Python 3.1 onwards.",
-                        PendingDeprecationWarning)
+                        DeprecationWarning)
         path = self.source_path(fullname)
         if path is None:
             raise ImportError
@@ -234,7 +234,7 @@
                             "removal in Python 3.4; use SourceLoader instead. "
                             "If Python 3.1 compatibility is required, see the "
                             "latest documentation for PyLoader.",
-                        PendingDeprecationWarning)
+                        DeprecationWarning)
         source_timestamp = self.source_mtime(fullname)
         # Try to use bytecode if it is available.
         bytecode_path = self.bytecode_path(fullname)
diff --git a/Lib/importlib/test/source/test_abc_loader.py b/Lib/importlib/test/source/test_abc_loader.py
--- a/Lib/importlib/test/source/test_abc_loader.py
+++ b/Lib/importlib/test/source/test_abc_loader.py
@@ -102,7 +102,7 @@
             warnings.simplefilter("always")
             path = super().get_filename(name)
             assert len(w) == 1
-            assert issubclass(w[0].category, PendingDeprecationWarning)
+            assert issubclass(w[0].category, DeprecationWarning)
             return path
 
 
@@ -198,7 +198,7 @@
             warnings.simplefilter("always")
             code_object = super().get_code(name)
             assert len(w) == 1
-            assert issubclass(w[0].category, PendingDeprecationWarning)
+            assert issubclass(w[0].category, DeprecationWarning)
             return code_object
 
 class PyLoaderTests(testing_abc.LoaderTests):
diff --git a/Lib/nntplib.py b/Lib/nntplib.py
--- a/Lib/nntplib.py
+++ b/Lib/nntplib.py
@@ -828,7 +828,7 @@
         - list: list of (name,title) strings"""
         warnings.warn("The XGTITLE extension is not actively used, "
                       "use descriptions() instead",
-                      PendingDeprecationWarning, 2)
+                      DeprecationWarning, 2)
         line_pat = re.compile('^([^ \t]+)[ \t]+(.*)$')
         resp, raw_lines = self._longcmdstring('XGTITLE ' + group, file)
         lines = []
@@ -846,7 +846,7 @@
         path: directory path to article
         """
         warnings.warn("The XPATH extension is not actively used",
-                      PendingDeprecationWarning, 2)
+                      DeprecationWarning, 2)
 
         resp = self._shortcmd('XPATH {0}'.format(id))
         if not resp.startswith('223'):
diff --git a/Lib/smtpd.py b/Lib/smtpd.py
--- a/Lib/smtpd.py
+++ b/Lib/smtpd.py
@@ -142,122 +142,122 @@
     @property
     def __server(self):
         warn("Access to __server attribute on SMTPChannel is deprecated, "
-            "use 'smtp_server' instead", PendingDeprecationWarning, 2)
+            "use 'smtp_server' instead", DeprecationWarning, 2)
         return self.smtp_server
     @__server.setter
     def __server(self, value):
         warn("Setting __server attribute on SMTPChannel is deprecated, "
-            "set 'smtp_server' instead", PendingDeprecationWarning, 2)
+            "set 'smtp_server' instead", DeprecationWarning, 2)
         self.smtp_server = value
 
     @property
     def __line(self):
         warn("Access to __line attribute on SMTPChannel is deprecated, "
-            "use 'received_lines' instead", PendingDeprecationWarning, 2)
+            "use 'received_lines' instead", DeprecationWarning, 2)
         return self.received_lines
     @__line.setter
     def __line(self, value):
         warn("Setting __line attribute on SMTPChannel is deprecated, "
-            "set 'received_lines' instead", PendingDeprecationWarning, 2)
+            "set 'received_lines' instead", DeprecationWarning, 2)
         self.received_lines = value
 
     @property
     def __state(self):
         warn("Access to __state attribute on SMTPChannel is deprecated, "
-            "use 'smtp_state' instead", PendingDeprecationWarning, 2)
+            "use 'smtp_state' instead", DeprecationWarning, 2)
         return self.smtp_state
     @__state.setter
     def __state(self, value):
         warn("Setting __state attribute on SMTPChannel is deprecated, "
-            "set 'smtp_state' instead", PendingDeprecationWarning, 2)
+            "set 'smtp_state' instead", DeprecationWarning, 2)
         self.smtp_state = value
 
     @property
     def __greeting(self):
         warn("Access to __greeting attribute on SMTPChannel is deprecated, "
-            "use 'seen_greeting' instead", PendingDeprecationWarning, 2)
+            "use 'seen_greeting' instead", DeprecationWarning, 2)
         return self.seen_greeting
     @__greeting.setter
     def __greeting(self, value):
         warn("Setting __greeting attribute on SMTPChannel is deprecated, "
-            "set 'seen_greeting' instead", PendingDeprecationWarning, 2)
+            "set 'seen_greeting' instead", DeprecationWarning, 2)
         self.seen_greeting = value
 
     @property
     def __mailfrom(self):
         warn("Access to __mailfrom attribute on SMTPChannel is deprecated, "
-            "use 'mailfrom' instead", PendingDeprecationWarning, 2)
+            "use 'mailfrom' instead", DeprecationWarning, 2)
         return self.mailfrom
     @__mailfrom.setter
     def __mailfrom(self, value):
         warn("Setting __mailfrom attribute on SMTPChannel is deprecated, "
-            "set 'mailfrom' instead", PendingDeprecationWarning, 2)
+            "set 'mailfrom' instead", DeprecationWarning, 2)
         self.mailfrom = value
 
     @property
     def __rcpttos(self):
         warn("Access to __rcpttos attribute on SMTPChannel is deprecated, "
-            "use 'rcpttos' instead", PendingDeprecationWarning, 2)
+            "use 'rcpttos' instead", DeprecationWarning, 2)
         return self.rcpttos
     @__rcpttos.setter
     def __rcpttos(self, value):
         warn("Setting __rcpttos attribute on SMTPChannel is deprecated, "
-            "set 'rcpttos' instead", PendingDeprecationWarning, 2)
+            "set 'rcpttos' instead", DeprecationWarning, 2)
         self.rcpttos = value
 
     @property
     def __data(self):
         warn("Access to __data attribute on SMTPChannel is deprecated, "
-            "use 'received_data' instead", PendingDeprecationWarning, 2)
+            "use 'received_data' instead", DeprecationWarning, 2)
         return self.received_data
     @__data.setter
     def __data(self, value):
         warn("Setting __data attribute on SMTPChannel is deprecated, "
-            "set 'received_data' instead", PendingDeprecationWarning, 2)
+            "set 'received_data' instead", DeprecationWarning, 2)
         self.received_data = value
 
     @property
     def __fqdn(self):
         warn("Access to __fqdn attribute on SMTPChannel is deprecated, "
-            "use 'fqdn' instead", PendingDeprecationWarning, 2)
+            "use 'fqdn' instead", DeprecationWarning, 2)
         return self.fqdn
     @__fqdn.setter
     def __fqdn(self, value):
         warn("Setting __fqdn attribute on SMTPChannel is deprecated, "
-            "set 'fqdn' instead", PendingDeprecationWarning, 2)
+            "set 'fqdn' instead", DeprecationWarning, 2)
         self.fqdn = value
 
     @property
     def __peer(self):
         warn("Access to __peer attribute on SMTPChannel is deprecated, "
-            "use 'peer' instead", PendingDeprecationWarning, 2)
+            "use 'peer' instead", DeprecationWarning, 2)
         return self.peer
     @__peer.setter
     def __peer(self, value):
         warn("Setting __peer attribute on SMTPChannel is deprecated, "
-            "set 'peer' instead", PendingDeprecationWarning, 2)
+            "set 'peer' instead", DeprecationWarning, 2)
         self.peer = value
 
     @property
     def __conn(self):
         warn("Access to __conn attribute on SMTPChannel is deprecated, "
-            "use 'conn' instead", PendingDeprecationWarning, 2)
+            "use 'conn' instead", DeprecationWarning, 2)
         return self.conn
     @__conn.setter
     def __conn(self, value):
         warn("Setting __conn attribute on SMTPChannel is deprecated, "
-            "set 'conn' instead", PendingDeprecationWarning, 2)
+            "set 'conn' instead", DeprecationWarning, 2)
         self.conn = value
 
     @property
     def __addr(self):
         warn("Access to __addr attribute on SMTPChannel is deprecated, "
-            "use 'addr' instead", PendingDeprecationWarning, 2)
+            "use 'addr' instead", DeprecationWarning, 2)
         return self.addr
     @__addr.setter
     def __addr(self, value):
         warn("Setting __addr attribute on SMTPChannel is deprecated, "
-            "set 'addr' instead", PendingDeprecationWarning, 2)
+            "set 'addr' instead", DeprecationWarning, 2)
         self.addr = value
 
     # Overrides base class for convenience
diff --git a/Lib/test/test_smtpd.py b/Lib/test/test_smtpd.py
--- a/Lib/test/test_smtpd.py
+++ b/Lib/test/test_smtpd.py
@@ -239,49 +239,49 @@
         self.assertEqual(self.channel.socket.last, b'501 Syntax: RSET\r\n')
 
     def test_attribute_deprecations(self):
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__server
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__server = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__line
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__line = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__state
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__state = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__greeting
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__greeting = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__mailfrom
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__mailfrom = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__rcpttos
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__rcpttos = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__data
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__data = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__fqdn
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__fqdn = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__peer
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__peer = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__conn
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__conn = 'spam'
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             spam = self.channel._SMTPChannel__addr
-        with support.check_warnings(('', PendingDeprecationWarning)):
+        with support.check_warnings(('', DeprecationWarning)):
             self.channel._SMTPChannel__addr = 'spam'
 
 def test_main():
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,11 @@
 Library
 -------
 
+- Issue #13248: Turn 3.2's PendingDeprecationWarning into 3.3's
+  DeprecationWarning.  It covers 'cgi.escape', 'importlib.abc.PyLoader',
+  'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath',
+  and private attributes of 'smtpd.SMTPChannel'.
+
 - Issue #5905: time.strftime() is now using the locale encoding, instead of
   UTF-8, if the wcsftime() function is not available.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 12:52:36 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Sat, 10 Dec 2011 12:52:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_doc_typo=2E?=
Message-ID: <E1RZLTo-0006Nz-VM@dinsdale.python.org>

http://hg.python.org/cpython/rev/caeec3e3606f
changeset:   73915:caeec3e3606f
branch:      3.2
parent:      73912:ee0c9ca09c84
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Sat Dec 10 12:45:45 2011 +0100
summary:
  Fix doc typo.

files:
  Doc/library/tarfile.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -101,10 +101,10 @@
    +-------------+--------------------------------------------+
    | ``'w|'``    | Open an uncompressed *stream* for writing. |
    +-------------+--------------------------------------------+
-   | ``'w|gz'``  | Open an gzip compressed *stream* for       |
+   | ``'w|gz'``  | Open a gzip compressed *stream* for        |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
-   | ``'w|bz2'`` | Open an bzip2 compressed *stream* for      |
+   | ``'w|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 12:52:37 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Sat, 10 Dec 2011 12:52:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_with_3=2E2=3A_Fix_doc_typo=2E?=
Message-ID: <E1RZLTp-0006O9-Mm@dinsdale.python.org>

http://hg.python.org/cpython/rev/5da2c242025f
changeset:   73916:5da2c242025f
parent:      73914:f82ebf9b3a52
parent:      73915:caeec3e3606f
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Sat Dec 10 12:48:03 2011 +0100
summary:
  Merge with 3.2: Fix doc typo.

files:
  Doc/library/tarfile.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -101,10 +101,10 @@
    +-------------+--------------------------------------------+
    | ``'w|'``    | Open an uncompressed *stream* for writing. |
    +-------------+--------------------------------------------+
-   | ``'w|gz'``  | Open an gzip compressed *stream* for       |
+   | ``'w|gz'``  | Open a gzip compressed *stream* for        |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
-   | ``'w|bz2'`` | Open an bzip2 compressed *stream* for      |
+   | ``'w|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 13:02:46 2011
From: python-checkins at python.org (florent.xicluna)
Date: Sat, 10 Dec 2011 13:02:46 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_comment_in_difflib=2E?=
Message-ID: <E1RZLde-00077p-Eg@dinsdale.python.org>

http://hg.python.org/cpython/rev/a3309634f38c
changeset:   73917:a3309634f38c
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Sat Dec 10 13:02:17 2011 +0100
summary:
  Fix comment in difflib.

files:
  Lib/difflib.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/difflib.py b/Lib/difflib.py
--- a/Lib/difflib.py
+++ b/Lib/difflib.py
@@ -204,7 +204,7 @@
         #      returning true iff the element is "junk" -- this has
         #      subtle but helpful effects on the algorithm, which I'll
         #      get around to writing up someday <0.9 wink>.
-        #      DON'T USE!  Only __chain_b uses this.  Use isbjunk.
+        #      DON'T USE!  Only __chain_b uses this.  Use "in self.bjunk".
         # bjunk
         #      the items in b for which isjunk is True.
         # bpopular
@@ -287,7 +287,6 @@
     # when self.isjunk is defined, junk elements don't show up in this
     # map at all, which stops the central find_longest_match method
     # from starting any matching block at a junk element ...
-    # also creates the fast isbjunk function ...
     # b2j also does not contain entries for "popular" elements, meaning
     # elements that account for more than 1 + 1% of the total elements, and
     # when the sequence is reasonably large (>= 200 elements); this can

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 13:18:23 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sat, 10 Dec 2011 13:18:23 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Catch_EAI=5FFAIL_in_support=2Etransient=5Finternet=2E?=
Message-ID: <E1RZLsl-0008Ss-FP@dinsdale.python.org>

http://hg.python.org/cpython/rev/5ba1a22c8988
changeset:   73918:5ba1a22c8988
branch:      2.7
parent:      73910:78a7d5f8f054
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sat Dec 10 13:16:02 2011 +0100
summary:
  Issue #13453: Catch EAI_FAIL in support.transient_internet.

files:
  Lib/test/test_support.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -764,6 +764,7 @@
     ]
     default_gai_errnos = [
         ('EAI_AGAIN', -3),
+        ('EAI_FAIL', -4),
         ('EAI_NONAME', -2),
         ('EAI_NODATA', -5),
     ]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 13:18:44 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sat, 10 Dec 2011 13:18:44 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Catch_EAI=5FFAIL_in_support=2Etransient=5Finternet=2E?=
Message-ID: <E1RZLt6-0008Tt-NS@dinsdale.python.org>

http://hg.python.org/cpython/rev/c998c6f5464b
changeset:   73919:c998c6f5464b
branch:      3.2
parent:      73915:caeec3e3606f
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sat Dec 10 13:16:44 2011 +0100
summary:
  Issue #13453: Catch EAI_FAIL in support.transient_internet.

files:
  Lib/test/support.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -877,6 +877,7 @@
     ]
     default_gai_errnos = [
         ('EAI_AGAIN', -3),
+        ('EAI_FAIL', -4),
         ('EAI_NONAME', -2),
         ('EAI_NODATA', -5),
         # Encountered when trying to resolve IPv6-only hostnames

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 13:19:06 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sat, 10 Dec 2011 13:19:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313453=3A_Catch_EAI=5FFAIL_in_support=2Etransient=5F?=
 =?utf8?q?internet=2E?=
Message-ID: <E1RZLtS-0008Ux-0N@dinsdale.python.org>

http://hg.python.org/cpython/rev/767badea87c3
changeset:   73920:767badea87c3
parent:      73917:a3309634f38c
parent:      73919:c998c6f5464b
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sat Dec 10 13:17:46 2011 +0100
summary:
  Issue #13453: Catch EAI_FAIL in support.transient_internet.

files:
  Lib/test/support.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -951,6 +951,7 @@
     ]
     default_gai_errnos = [
         ('EAI_AGAIN', -3),
+        ('EAI_FAIL', -4),
         ('EAI_NONAME', -2),
         ('EAI_NODATA', -5),
         # Encountered when trying to resolve IPv6-only hostnames

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 14:37:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 10 Dec 2011 14:37:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2311886=3A_Fix_also_?=
 =?utf8?q?test=5Ftime_for_the_non-DST_timezone_name_=28EST/AEST=29?=
Message-ID: <E1RZN7l-0004kR-SA@dinsdale.python.org>

http://hg.python.org/cpython/rev/e37a7dc8944e
changeset:   73921:e37a7dc8944e
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 10 14:37:53 2011 +0100
summary:
  Issue #11886: Fix also test_time for the non-DST timezone name (EST/AEST)

files:
  Lib/test/test_time.py |  10 ++++++----
  1 files changed, 6 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
--- a/Lib/test/test_time.py
+++ b/Lib/test/test_time.py
@@ -252,11 +252,13 @@
             self.assertNotEqual(time.gmtime(xmas2002), time.localtime(xmas2002))
 
             # Issue #11886: Australian Eastern Standard Time (UTC+10) is called
-            # "EST" (as Eastern Standard Time, UTC-5) instead of "AEST" on some
-            # operating systems (e.g. FreeBSD), which is wrong. See for example
-            # this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810
+            # "EST" (as Eastern Standard Time, UTC-5) instead of "AEST"
+            # (non-DST timezone), and "EDT" instead of "AEDT" (DST timezone),
+            # on some operating systems (e.g. FreeBSD), which is wrong. See for
+            # example this bug:
+            # http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=93810
             self.assertIn(time.tzname[0], ('AEST' 'EST'), time.tzname[0])
-            self.assertTrue(time.tzname[1] == 'AEDT', str(time.tzname[1]))
+            self.assertTrue(time.tzname[1] in ('AEDT', 'EDT'), str(time.tzname[1]))
             self.assertEqual(len(time.tzname), 2)
             self.assertEqual(time.daylight, 1)
             self.assertEqual(time.timezone, -36000)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 18:38:58 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 18:38:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_you_can=27t_get?=
 =?utf8?q?_resource=2Eerror_if_you_can=27t_import_resource?=
Message-ID: <E1RZQt0-0003kH-Qa@dinsdale.python.org>

http://hg.python.org/cpython/rev/fd5f43737546
changeset:   73922:fd5f43737546
branch:      2.7
parent:      73918:5ba1a22c8988
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 12:31:42 2011 -0500
summary:
  you can't get resource.error if you can't import resource

files:
  Lib/test/test_subprocess.py |  27 ++++++++++++++----------
  1 files changed, 16 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -10,6 +10,11 @@
 import re
 import sysconfig
 
+try:
+    import resource
+except ImportError:
+    resource = None
+
 mswindows = (sys.platform == "win32")
 
 #
@@ -633,12 +638,12 @@
 
     def __enter__(self):
         """Try to save previous ulimit, then set it to (0, 0)."""
-        try:
-            import resource
-            self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
-            resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
+                resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
+            except (ValueError, resource.error):
+                pass
 
         if sys.platform == 'darwin':
             # Check if the 'Crash Reporter' on OSX was configured
@@ -658,11 +663,11 @@
         """Return core file behavior to default."""
         if self.old_limit is None:
             return
-        try:
-            import resource
-            resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
+            except (ValueError, resource.error):
+                pass
 
     @unittest.skipUnless(hasattr(signal, 'SIGALRM'),
                          "Requires signal.SIGALRM")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 18:39:29 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 18:39:29 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_you_can=27t_get?=
 =?utf8?q?_resource=2Eerror_if_you_can=27t_import_resource?=
Message-ID: <E1RZQtV-0003lT-NW@dinsdale.python.org>

http://hg.python.org/cpython/rev/245f6094ccba
changeset:   73923:245f6094ccba
branch:      3.2
parent:      73919:c998c6f5464b
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 12:31:42 2011 -0500
summary:
  you can't get resource.error if you can't import resource

files:
  Lib/test/test_subprocess.py |  28 +++++++++++++++---------
  1 files changed, 17 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -18,6 +18,12 @@
 except ImportError:
     gc = None
 
+
+try:
+    import resource
+except ImportError:
+    resource = None
+
 mswindows = (sys.platform == "win32")
 
 #
@@ -732,12 +738,12 @@
 
     def __enter__(self):
         """Try to save previous ulimit, then set it to (0, 0)."""
-        try:
-            import resource
-            self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
-            resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
+                resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
+            except (ValueError, resource.error):
+                pass
 
         if sys.platform == 'darwin':
             # Check if the 'Crash Reporter' on OSX was configured
@@ -758,11 +764,11 @@
         """Return core file behavior to default."""
         if self.old_limit is None:
             return
-        try:
-            import resource
-            resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
+            except (ValueError, resource.error):
+                pass
 
 
 @unittest.skipIf(mswindows, "POSIX specific tests")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 18:40:00 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 18:40:00 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1RZQu0-0003mx-UV@dinsdale.python.org>

http://hg.python.org/cpython/rev/423f47499e4a
changeset:   73924:423f47499e4a
parent:      73921:e37a7dc8944e
parent:      73923:245f6094ccba
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 12:38:52 2011 -0500
summary:
  merge 3.2

files:
  Lib/test/test_subprocess.py |  28 +++++++++++++++---------
  1 files changed, 17 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -18,6 +18,12 @@
 except ImportError:
     gc = None
 
+
+try:
+    import resource
+except ImportError:
+    resource = None
+
 mswindows = (sys.platform == "win32")
 
 #
@@ -824,12 +830,12 @@
 
     def __enter__(self):
         """Try to save previous ulimit, then set it to (0, 0)."""
-        try:
-            import resource
-            self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
-            resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                self.old_limit = resource.getrlimit(resource.RLIMIT_CORE)
+                resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
+            except (ValueError, resource.error):
+                pass
 
         if sys.platform == 'darwin':
             # Check if the 'Crash Reporter' on OSX was configured
@@ -850,11 +856,11 @@
         """Return core file behavior to default."""
         if self.old_limit is None:
             return
-        try:
-            import resource
-            resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
-        except (ImportError, ValueError, resource.error):
-            pass
+        if resource is not None:
+            try:
+                resource.setrlimit(resource.RLIMIT_CORE, self.old_limit)
+            except (ValueError, resource.error):
+                pass
 
 
 @unittest.skipIf(mswindows, "POSIX specific tests")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 18:44:42 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 18:44:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_we=27re_always_?=
	=?utf8?q?going_to_have_gc?=
Message-ID: <E1RZQyY-00047z-Ca@dinsdale.python.org>

http://hg.python.org/cpython/rev/6040c248a199
changeset:   73925:6040c248a199
branch:      3.2
parent:      73923:245f6094ccba
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 12:44:25 2011 -0500
summary:
  we're always going to have gc

files:
  Lib/test/test_subprocess.py |  7 +------
  1 files changed, 1 insertions(+), 6 deletions(-)


diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -13,11 +13,7 @@
 import warnings
 import select
 import shutil
-try:
-    import gc
-except ImportError:
-    gc = None
-
+import gc
 
 try:
     import resource
@@ -859,7 +855,6 @@
             self.fail("Exception raised by preexec_fn did not make it "
                       "to the parent process.")
 
-    @unittest.skipUnless(gc, "Requires a gc module.")
     def test_preexec_gc_module_failure(self):
         # This tests the code that disables garbage collection if the child
         # process will execute any Python.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 18:45:03 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 18:45:03 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1RZQyt-0004AA-Kr@dinsdale.python.org>

http://hg.python.org/cpython/rev/aab45b904141
changeset:   73926:aab45b904141
parent:      73924:423f47499e4a
parent:      73925:6040c248a199
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 12:44:37 2011 -0500
summary:
  merge 3.2

files:
  Lib/test/test_subprocess.py |  7 +------
  1 files changed, 1 insertions(+), 6 deletions(-)


diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -13,11 +13,7 @@
 import warnings
 import select
 import shutil
-try:
-    import gc
-except ImportError:
-    gc = None
-
+import gc
 
 try:
     import resource
@@ -951,7 +947,6 @@
             self.fail("Exception raised by preexec_fn did not make it "
                       "to the parent process.")
 
-    @unittest.skipUnless(gc, "Requires a gc module.")
     def test_preexec_gc_module_failure(self):
         # This tests the code that disables garbage collection if the child
         # process will execute any Python.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 20:40:17 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Sat, 10 Dec 2011 20:40:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=235689=3A_Add_suppor?=
 =?utf8?q?t_for_lzma_compression_to_the_tarfile_module=2E?=
Message-ID: <E1RZSmP-0003XL-SZ@dinsdale.python.org>

http://hg.python.org/cpython/rev/899a8c7b2310
changeset:   73927:899a8c7b2310
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Sat Dec 10 20:38:14 2011 +0100
summary:
  Issue #5689: Add support for lzma compression to the tarfile module.

files:
  Doc/library/tarfile.rst  |  22 +++++--
  Lib/tarfile.py           |  66 +++++++++++++++++++++--
  Lib/test/test_tarfile.py |  78 ++++++++++++++++++++++++---
  Misc/NEWS                |   2 +
  4 files changed, 146 insertions(+), 22 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -13,12 +13,12 @@
 --------------
 
 The :mod:`tarfile` module makes it possible to read and write tar
-archives, including those using gzip or bz2 compression.
+archives, including those using gzip, bz2 and lzma compression.
 (:file:`.zip` files can be read and written using the :mod:`zipfile` module.)
 
 Some facts and figures:
 
-* reads and writes :mod:`gzip` and :mod:`bz2` compressed archives.
+* reads and writes :mod:`gzip`, :mod:`bz2` and :mod:`lzma` compressed archives.
 
 * read/write support for the POSIX.1-1988 (ustar) format.
 
@@ -55,6 +55,8 @@
    +------------------+---------------------------------------------+
    | ``'r:bz2'``      | Open for reading with bzip2 compression.    |
    +------------------+---------------------------------------------+
+   | ``'r:xz'``       | Open for reading with lzma compression.     |
+   +------------------+---------------------------------------------+
    | ``'a' or 'a:'``  | Open for appending with no compression. The |
    |                  | file is created if it does not exist.       |
    +------------------+---------------------------------------------+
@@ -64,11 +66,13 @@
    +------------------+---------------------------------------------+
    | ``'w:bz2'``      | Open for bzip2 compressed writing.          |
    +------------------+---------------------------------------------+
+   | ``'w:xz'``       | Open for lzma compressed writing.           |
+   +------------------+---------------------------------------------+
 
-   Note that ``'a:gz'`` or ``'a:bz2'`` is not possible. If *mode* is not suitable
-   to open a certain (compressed) file for reading, :exc:`ReadError` is raised. Use
-   *mode* ``'r'`` to avoid this.  If a compression method is not supported,
-   :exc:`CompressionError` is raised.
+   Note that ``'a:gz'``, ``'a:bz2'`` or ``'a:xz'`` is not possible. If *mode*
+   is not suitable to open a certain (compressed) file for reading,
+   :exc:`ReadError` is raised. Use *mode* ``'r'`` to avoid this.  If a
+   compression method is not supported, :exc:`CompressionError` is raised.
 
    If *fileobj* is specified, it is used as an alternative to a :term:`file object`
    opened in binary mode for *name*. It is supposed to be at position 0.
@@ -99,6 +103,9 @@
    | ``'r|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | reading.                                   |
    +-------------+--------------------------------------------+
+   | ``'r|xz'``  | Open a lzma compressed *stream* for        |
+   |             | reading.                                   |
+   +-------------+--------------------------------------------+
    | ``'w|'``    | Open an uncompressed *stream* for writing. |
    +-------------+--------------------------------------------+
    | ``'w|gz'``  | Open a gzip compressed *stream* for        |
@@ -107,6 +114,9 @@
    | ``'w|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
+   | ``'w|xz'``  | Open an lzma compressed *stream* for       |
+   |             | writing.                                   |
+   +-------------+--------------------------------------------+
 
 
 .. class:: TarFile
diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -420,10 +420,11 @@
                 self.crc = zlib.crc32(b"")
                 if mode == "r":
                     self._init_read_gz()
+                    self.exception = zlib.error
                 else:
                     self._init_write_gz()
 
-            if comptype == "bz2":
+            elif comptype == "bz2":
                 try:
                     import bz2
                 except ImportError:
@@ -431,8 +432,25 @@
                 if mode == "r":
                     self.dbuf = b""
                     self.cmp = bz2.BZ2Decompressor()
+                    self.exception = IOError
                 else:
                     self.cmp = bz2.BZ2Compressor()
+
+            elif comptype == "xz":
+                try:
+                    import lzma
+                except ImportError:
+                    raise CompressionError("lzma module is not available")
+                if mode == "r":
+                    self.dbuf = b""
+                    self.cmp = lzma.LZMADecompressor()
+                    self.exception = lzma.LZMAError
+                else:
+                    self.cmp = lzma.LZMACompressor()
+
+            elif comptype != "tar":
+                raise CompressionError("unknown compression type %r" % comptype)
+
         except:
             if not self._extfileobj:
                 self.fileobj.close()
@@ -584,7 +602,7 @@
                 break
             try:
                 buf = self.cmp.decompress(buf)
-            except IOError:
+            except self.exception:
                 raise ReadError("invalid compressed data")
             self.dbuf += buf
             c += len(buf)
@@ -622,11 +640,14 @@
         return self.buf
 
     def getcomptype(self):
-        if self.buf.startswith(b"\037\213\010"):
+        if self.buf.startswith(b"\x1f\x8b\x08"):
             return "gz"
-        if self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
+        elif self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
             return "bz2"
-        return "tar"
+        elif self.buf.startswith((b"\x5d\x00\x00\x80", b"\xfd7zXZ")):
+            return "xz"
+        else:
+            return "tar"
 
     def close(self):
         self.fileobj.close()
@@ -1651,18 +1672,22 @@
            'r:'         open for reading exclusively uncompressed
            'r:gz'       open for reading with gzip compression
            'r:bz2'      open for reading with bzip2 compression
+           'r:xz'       open for reading with lzma compression
            'a' or 'a:'  open for appending, creating the file if necessary
            'w' or 'w:'  open for writing without compression
            'w:gz'       open for writing with gzip compression
            'w:bz2'      open for writing with bzip2 compression
+           'w:xz'       open for writing with lzma compression
 
            'r|*'        open a stream of tar blocks with transparent compression
            'r|'         open an uncompressed stream of tar blocks for reading
            'r|gz'       open a gzip compressed stream of tar blocks
            'r|bz2'      open a bzip2 compressed stream of tar blocks
+           'r|xz'       open an lzma compressed stream of tar blocks
            'w|'         open an uncompressed stream for writing
            'w|gz'       open a gzip compressed stream for writing
            'w|bz2'      open a bzip2 compressed stream for writing
+           'w|xz'       open an lzma compressed stream for writing
         """
 
         if not name and not fileobj:
@@ -1780,11 +1805,40 @@
         t._extfileobj = False
         return t
 
+    @classmethod
+    def xzopen(cls, name, mode="r", fileobj=None, preset=9, **kwargs):
+        """Open lzma compressed tar archive name for reading or writing.
+           Appending is not allowed.
+        """
+        if mode not in ("r", "w"):
+            raise ValueError("mode must be 'r' or 'w'")
+
+        try:
+            import lzma
+        except ImportError:
+            raise CompressionError("lzma module is not available")
+
+        if mode == "r":
+            # LZMAFile complains about a preset argument in read mode.
+            preset = None
+
+        fileobj = lzma.LZMAFile(filename=name if fileobj is None else None,
+                mode=mode, fileobj=fileobj, preset=preset)
+
+        try:
+            t = cls.taropen(name, mode, fileobj, **kwargs)
+        except (lzma.LZMAError, EOFError):
+            fileobj.close()
+            raise ReadError("not an lzma file")
+        t._extfileobj = False
+        return t
+
     # All *open() methods are registered here.
     OPEN_METH = {
         "tar": "taropen",   # uncompressed tar
         "gz":  "gzopen",    # gzip compressed tar
-        "bz2": "bz2open"    # bzip2 compressed tar
+        "bz2": "bz2open",   # bzip2 compressed tar
+        "xz":  "xzopen"     # lzma compressed tar
     }
 
     #--------------------------------------------------------------------------
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -21,6 +21,10 @@
     import bz2
 except ImportError:
     bz2 = None
+try:
+    import lzma
+except ImportError:
+    lzma = None
 
 def md5sum(data):
     return md5(data).hexdigest()
@@ -29,6 +33,7 @@
 tarname = support.findfile("testtar.tar")
 gzipname = os.path.join(TEMPDIR, "testtar.tar.gz")
 bz2name = os.path.join(TEMPDIR, "testtar.tar.bz2")
+xzname = os.path.join(TEMPDIR, "testtar.tar.xz")
 tmpname = os.path.join(TEMPDIR, "tmp.tar")
 
 md5_regtype = "65f477c818ad9e15f7feab0c6d37742f"
@@ -201,13 +206,15 @@
             _open = gzip.GzipFile
         elif self.mode.endswith(":bz2"):
             _open = bz2.BZ2File
+        elif self.mode.endswith(":xz"):
+            _open = lzma.LZMAFile
         else:
-            _open = open
+            _open = io.FileIO
 
         for char in (b'\0', b'a'):
             # Test if EOFHeaderError ('\0') and InvalidHeaderError ('a')
             # are ignored correctly.
-            with _open(tmpname, "wb") as fobj:
+            with _open(tmpname, "w") as fobj:
                 fobj.write(char * 1024)
                 fobj.write(tarfile.TarInfo("foo").tobuf())
 
@@ -222,9 +229,10 @@
 class MiscReadTest(CommonReadTest):
 
     def test_no_name_argument(self):
-        if self.mode.endswith("bz2"):
-            # BZ2File has no name attribute.
-            return
+        if self.mode.endswith(("bz2", "xz")):
+            # BZ2File and LZMAFile have no name attribute.
+            self.skipTest("no name attribute")
+
         with open(self.tarname, "rb") as fobj:
             tar = tarfile.open(fileobj=fobj, mode=self.mode)
             self.assertEqual(tar.name, os.path.abspath(fobj.name))
@@ -265,10 +273,12 @@
             _open = gzip.GzipFile
         elif self.mode.endswith(":bz2"):
             _open = bz2.BZ2File
+        elif self.mode.endswith(":xz"):
+            _open = lzma.LZMAFile
         else:
-            _open = open
-        fobj = _open(self.tarname, "rb")
-        try:
+            _open = io.FileIO
+
+        with _open(self.tarname) as fobj:
             fobj.seek(offset)
 
             # Test if the tarfile starts with the second member.
@@ -281,8 +291,6 @@
             self.assertEqual(tar.extractfile(t).read(), data,
                     "seek back did not work")
             tar.close()
-        finally:
-            fobj.close()
 
     def test_fail_comp(self):
         # For Gzip and Bz2 Tests: fail with a ReadError on an uncompressed file.
@@ -526,6 +534,18 @@
             testfunc(bz2name, "r|*")
             testfunc(bz2name, "r|bz2")
 
+        if lzma:
+            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:xz")
+            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|xz")
+            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r:")
+            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r|")
+
+            testfunc(xzname, "r")
+            testfunc(xzname, "r:*")
+            testfunc(xzname, "r:xz")
+            testfunc(xzname, "r|*")
+            testfunc(xzname, "r|xz")
+
     def test_detect_file(self):
         self._test_modes(self._testfunc_file)
 
@@ -1096,6 +1116,9 @@
             data = dec.decompress(data)
             self.assertTrue(len(dec.unused_data) == 0,
                     "found trailing data")
+        elif self.mode.endswith("xz"):
+            with lzma.LZMAFile(tmpname) as fobj:
+                data = fobj.read()
         else:
             with open(tmpname, "rb") as fobj:
                 data = fobj.read()
@@ -1510,6 +1533,12 @@
         self._create_testtar("w:bz2")
         self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
 
+    def test_append_lzma(self):
+        if lzma is None:
+            self.skipTest("lzma module not available")
+        self._create_testtar("w:xz")
+        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
+
     # Append mode is supposed to fail if the tarfile to append to
     # does not end with a zero block.
     def _test_error(self, data):
@@ -1788,6 +1817,21 @@
         self._test_partial_input("r:bz2")
 
 
+class LzmaMiscReadTest(MiscReadTest):
+    tarname = xzname
+    mode = "r:xz"
+class LzmaUstarReadTest(UstarReadTest):
+    tarname = xzname
+    mode = "r:xz"
+class LzmaStreamReadTest(StreamReadTest):
+    tarname = xzname
+    mode = "r|xz"
+class LzmaWriteTest(WriteTest):
+    mode = "w:xz"
+class LzmaStreamWriteTest(StreamWriteTest):
+    mode = "w|xz"
+
+
 def test_main():
     support.unlink(TEMPDIR)
     os.makedirs(TEMPDIR)
@@ -1850,6 +1894,20 @@
             Bz2PartialReadTest,
         ]
 
+    if lzma:
+        # Create testtar.tar.xz and add lzma-specific tests.
+        support.unlink(xzname)
+        with lzma.LZMAFile(xzname, "w") as tar:
+            tar.write(data)
+
+        tests += [
+            LzmaMiscReadTest,
+            LzmaUstarReadTest,
+            LzmaStreamReadTest,
+            LzmaWriteTest,
+            LzmaStreamWriteTest,
+        ]
+
     try:
         support.run_unittest(*tests)
     finally:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,8 @@
 Library
 -------
 
+- Issue #5689: Add support for lzma compression to the tarfile module.
+
 - Issue #13248: Turn 3.2's PendingDeprecationWarning into 3.3's
   DeprecationWarning.  It covers 'cgi.escape', 'importlib.abc.PyLoader',
   'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath',

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 21:15:18 2011
From: python-checkins at python.org (florent.xicluna)
Date: Sat, 10 Dec 2011 21:15:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_imports_in_xml=2Edom=2E?=
Message-ID: <E1RZTKI-0005qn-P3@dinsdale.python.org>

http://hg.python.org/cpython/rev/ed6ddd59bf2c
changeset:   73928:ed6ddd59bf2c
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Sat Dec 10 21:14:53 2011 +0100
summary:
  Fix imports in xml.dom.

files:
  Lib/test/test_minidom.py |  2 +-
  Lib/xml/dom/pulldom.py   |  6 +-----
  2 files changed, 2 insertions(+), 6 deletions(-)


diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py
--- a/Lib/test/test_minidom.py
+++ b/Lib/test/test_minidom.py
@@ -47,7 +47,7 @@
 
     def checkWholeText(self, node, s):
         t = node.wholeText
-        self.confirm(t == s, "looking for %s, found %s" % (repr(s), repr(t)))
+        self.confirm(t == s, "looking for %r, found %r" % (s, t))
 
     def testParseFromFile(self):
         with open(tstfile) as file:
diff --git a/Lib/xml/dom/pulldom.py b/Lib/xml/dom/pulldom.py
--- a/Lib/xml/dom/pulldom.py
+++ b/Lib/xml/dom/pulldom.py
@@ -1,6 +1,5 @@
 import xml.sax
 import xml.sax.handler
-import types
 
 START_ELEMENT = "START_ELEMENT"
 END_ELEMENT = "END_ELEMENT"
@@ -334,10 +333,7 @@
     return DOMEventStream(stream, parser, bufsize)
 
 def parseString(string, parser=None):
-    try:
-        from io import StringIO
-    except ImportError:
-        from io import StringIO
+    from io import StringIO
 
     bufsize = len(string)
     buf = StringIO(string)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 21:25:09 2011
From: python-checkins at python.org (giampaolo.rodola)
Date: Sat, 10 Dec 2011 21:25:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_=2313563=3A_make_use_of?=
 =?utf8?q?_with_statement_in_ftplib=2Epy_where_needed?=
Message-ID: <E1RZTTp-0006RJ-Oo@dinsdale.python.org>

http://hg.python.org/cpython/rev/6cd736239b8a
changeset:   73929:6cd736239b8a
user:        Giampaolo Rodola' <g.rodola at gmail.com>
date:        Sat Dec 10 21:25:04 2011 +0100
summary:
  fix #13563: make use of with statement in ftplib.py where needed

files:
  Lib/ftplib.py |  25 +++++--------------------
  1 files changed, 5 insertions(+), 20 deletions(-)


diff --git a/Lib/ftplib.py b/Lib/ftplib.py
--- a/Lib/ftplib.py
+++ b/Lib/ftplib.py
@@ -359,8 +359,7 @@
                 conn.close()
                 raise
         else:
-            sock = self.makeport()
-            try:
+            with self.makeport() as sock:
                 if rest is not None:
                     self.sendcmd("REST %s" % rest)
                 resp = self.sendcmd(cmd)
@@ -372,8 +371,6 @@
                 conn, sockaddr = sock.accept()
                 if self.timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                     conn.settimeout(self.timeout)
-            finally:
-                sock.close()
         if resp[:3] == '150':
             # this is conditional in case we received a 125
             size = parse150(resp)
@@ -753,8 +750,7 @@
 
         def retrbinary(self, cmd, callback, blocksize=8192, rest=None):
             self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
+            with self.transfercmd(cmd, rest) as conn:
                 while 1:
                     data = conn.recv(blocksize)
                     if not data:
@@ -763,8 +759,6 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def retrlines(self, cmd, callback = None):
@@ -772,7 +766,7 @@
             resp = self.sendcmd('TYPE A')
             conn = self.transfercmd(cmd)
             fp = conn.makefile('r', encoding=self.encoding)
-            try:
+            with fp, conn:
                 while 1:
                     line = fp.readline()
                     if self.debugging > 2: print('*retr*', repr(line))
@@ -786,15 +780,11 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                fp.close()
-                conn.close()
             return self.voidresp()
 
         def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
             self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
+            with self.transfercmd(cmd, rest) as conn:
                 while 1:
                     buf = fp.read(blocksize)
                     if not buf: break
@@ -803,14 +793,11 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def storlines(self, cmd, fp, callback=None):
             self.voidcmd('TYPE A')
-            conn = self.transfercmd(cmd)
-            try:
+            with self.transfercmd(cmd) as conn:
                 while 1:
                     buf = fp.readline()
                     if not buf: break
@@ -822,8 +809,6 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def abort(self):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 23:55:36 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 23:55:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_alias_resource=2Eerror_to_O?=
	=?utf8?q?SError?=
Message-ID: <E1RZVpQ-00074p-Os@dinsdale.python.org>

http://hg.python.org/cpython/rev/8d670cb0d64a
changeset:   73930:8d670cb0d64a
parent:      73926:aab45b904141
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 17:50:22 2011 -0500
summary:
  alias resource.error to OSError

files:
  Doc/library/resource.rst |   8 +++++---
  Misc/NEWS                |   2 ++
  Modules/resource.c       |  16 +++++-----------
  3 files changed, 12 insertions(+), 14 deletions(-)


diff --git a/Doc/library/resource.rst b/Doc/library/resource.rst
--- a/Doc/library/resource.rst
+++ b/Doc/library/resource.rst
@@ -14,13 +14,15 @@
 Symbolic constants are used to specify particular system resources and to
 request usage information about either the current process or its children.
 
-A single exception is defined for errors:
+An :exc:`OSError` is raised on syscall failure.
 
 
 .. exception:: error
 
-   The functions described below may raise this error if the underlying system call
-   failures unexpectedly.
+   A deprecated alias of :exc:`OSError`.
+
+   .. versionchanged:: 3.3
+      Following :pep:`3151`, this class was made an alias of :exc:`OSError`.
 
 
 Resource Limits
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -406,6 +406,8 @@
 Library
 -------
 
+- Alias resource.error to OSError ala PEP 3151.
+
 - Issue #13248: Turn 3.2's PendingDeprecationWarning into 3.3's
   DeprecationWarning.  It covers 'cgi.escape', 'importlib.abc.PyLoader',
   'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath',
diff --git a/Modules/resource.c b/Modules/resource.c
--- a/Modules/resource.c
+++ b/Modules/resource.c
@@ -18,8 +18,6 @@
 
 #define doubletime(TV) ((double)(TV).tv_sec + (TV).tv_usec * 0.000001)
 
-static PyObject *ResourceError;
-
 PyDoc_STRVAR(struct_rusage__doc__,
 "struct_rusage: Result from getrusage.\n\n"
 "This object may be accessed either as a tuple of\n"
@@ -73,7 +71,7 @@
                             "invalid who parameter");
             return NULL;
         }
-        PyErr_SetFromErrno(ResourceError);
+        PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
 
@@ -125,7 +123,7 @@
     }
 
     if (getrlimit(resource, &rl) == -1) {
-        PyErr_SetFromErrno(ResourceError);
+        PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
 
@@ -183,7 +181,7 @@
             PyErr_SetString(PyExc_ValueError,
                             "not allowed to raise maximum limit");
         else
-            PyErr_SetFromErrno(ResourceError);
+            PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
     Py_INCREF(Py_None);
@@ -246,12 +244,8 @@
         return NULL;
 
     /* Add some symbolic constants to the module */
-    if (ResourceError == NULL) {
-        ResourceError = PyErr_NewException("resource.error",
-                                           NULL, NULL);
-    }
-    Py_INCREF(ResourceError);
-    PyModule_AddObject(m, "error", ResourceError);
+    Py_INCREF(PyExc_OSError);
+    PyModule_AddObject(m, "error", PyExc_OSError);
     if (!initialized)
         PyStructSequence_InitType(&StructRUsageType,
                                   &struct_rusage_desc);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 10 23:55:37 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 10 Dec 2011 23:55:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RZVpR-00074z-KO@dinsdale.python.org>

http://hg.python.org/cpython/rev/2bbe7dc920de
changeset:   73931:2bbe7dc920de
parent:      73930:8d670cb0d64a
parent:      73929:6cd736239b8a
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 10 17:55:31 2011 -0500
summary:
  merge heads

files:
  Doc/library/tarfile.rst  |  22 +++++--
  Lib/ftplib.py            |  25 +-------
  Lib/tarfile.py           |  66 +++++++++++++++++++++--
  Lib/test/test_minidom.py |   2 +-
  Lib/test/test_tarfile.py |  78 ++++++++++++++++++++++++---
  Lib/xml/dom/pulldom.py   |   6 +-
  Misc/NEWS                |   2 +
  7 files changed, 153 insertions(+), 48 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -13,12 +13,12 @@
 --------------
 
 The :mod:`tarfile` module makes it possible to read and write tar
-archives, including those using gzip or bz2 compression.
+archives, including those using gzip, bz2 and lzma compression.
 (:file:`.zip` files can be read and written using the :mod:`zipfile` module.)
 
 Some facts and figures:
 
-* reads and writes :mod:`gzip` and :mod:`bz2` compressed archives.
+* reads and writes :mod:`gzip`, :mod:`bz2` and :mod:`lzma` compressed archives.
 
 * read/write support for the POSIX.1-1988 (ustar) format.
 
@@ -55,6 +55,8 @@
    +------------------+---------------------------------------------+
    | ``'r:bz2'``      | Open for reading with bzip2 compression.    |
    +------------------+---------------------------------------------+
+   | ``'r:xz'``       | Open for reading with lzma compression.     |
+   +------------------+---------------------------------------------+
    | ``'a' or 'a:'``  | Open for appending with no compression. The |
    |                  | file is created if it does not exist.       |
    +------------------+---------------------------------------------+
@@ -64,11 +66,13 @@
    +------------------+---------------------------------------------+
    | ``'w:bz2'``      | Open for bzip2 compressed writing.          |
    +------------------+---------------------------------------------+
+   | ``'w:xz'``       | Open for lzma compressed writing.           |
+   +------------------+---------------------------------------------+
 
-   Note that ``'a:gz'`` or ``'a:bz2'`` is not possible. If *mode* is not suitable
-   to open a certain (compressed) file for reading, :exc:`ReadError` is raised. Use
-   *mode* ``'r'`` to avoid this.  If a compression method is not supported,
-   :exc:`CompressionError` is raised.
+   Note that ``'a:gz'``, ``'a:bz2'`` or ``'a:xz'`` is not possible. If *mode*
+   is not suitable to open a certain (compressed) file for reading,
+   :exc:`ReadError` is raised. Use *mode* ``'r'`` to avoid this.  If a
+   compression method is not supported, :exc:`CompressionError` is raised.
 
    If *fileobj* is specified, it is used as an alternative to a :term:`file object`
    opened in binary mode for *name*. It is supposed to be at position 0.
@@ -99,6 +103,9 @@
    | ``'r|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | reading.                                   |
    +-------------+--------------------------------------------+
+   | ``'r|xz'``  | Open a lzma compressed *stream* for        |
+   |             | reading.                                   |
+   +-------------+--------------------------------------------+
    | ``'w|'``    | Open an uncompressed *stream* for writing. |
    +-------------+--------------------------------------------+
    | ``'w|gz'``  | Open a gzip compressed *stream* for        |
@@ -107,6 +114,9 @@
    | ``'w|bz2'`` | Open a bzip2 compressed *stream* for       |
    |             | writing.                                   |
    +-------------+--------------------------------------------+
+   | ``'w|xz'``  | Open an lzma compressed *stream* for       |
+   |             | writing.                                   |
+   +-------------+--------------------------------------------+
 
 
 .. class:: TarFile
diff --git a/Lib/ftplib.py b/Lib/ftplib.py
--- a/Lib/ftplib.py
+++ b/Lib/ftplib.py
@@ -359,8 +359,7 @@
                 conn.close()
                 raise
         else:
-            sock = self.makeport()
-            try:
+            with self.makeport() as sock:
                 if rest is not None:
                     self.sendcmd("REST %s" % rest)
                 resp = self.sendcmd(cmd)
@@ -372,8 +371,6 @@
                 conn, sockaddr = sock.accept()
                 if self.timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                     conn.settimeout(self.timeout)
-            finally:
-                sock.close()
         if resp[:3] == '150':
             # this is conditional in case we received a 125
             size = parse150(resp)
@@ -753,8 +750,7 @@
 
         def retrbinary(self, cmd, callback, blocksize=8192, rest=None):
             self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
+            with self.transfercmd(cmd, rest) as conn:
                 while 1:
                     data = conn.recv(blocksize)
                     if not data:
@@ -763,8 +759,6 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def retrlines(self, cmd, callback = None):
@@ -772,7 +766,7 @@
             resp = self.sendcmd('TYPE A')
             conn = self.transfercmd(cmd)
             fp = conn.makefile('r', encoding=self.encoding)
-            try:
+            with fp, conn:
                 while 1:
                     line = fp.readline()
                     if self.debugging > 2: print('*retr*', repr(line))
@@ -786,15 +780,11 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                fp.close()
-                conn.close()
             return self.voidresp()
 
         def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
             self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
+            with self.transfercmd(cmd, rest) as conn:
                 while 1:
                     buf = fp.read(blocksize)
                     if not buf: break
@@ -803,14 +793,11 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def storlines(self, cmd, fp, callback=None):
             self.voidcmd('TYPE A')
-            conn = self.transfercmd(cmd)
-            try:
+            with self.transfercmd(cmd) as conn:
                 while 1:
                     buf = fp.readline()
                     if not buf: break
@@ -822,8 +809,6 @@
                 # shutdown ssl layer
                 if isinstance(conn, ssl.SSLSocket):
                     conn.unwrap()
-            finally:
-                conn.close()
             return self.voidresp()
 
         def abort(self):
diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -420,10 +420,11 @@
                 self.crc = zlib.crc32(b"")
                 if mode == "r":
                     self._init_read_gz()
+                    self.exception = zlib.error
                 else:
                     self._init_write_gz()
 
-            if comptype == "bz2":
+            elif comptype == "bz2":
                 try:
                     import bz2
                 except ImportError:
@@ -431,8 +432,25 @@
                 if mode == "r":
                     self.dbuf = b""
                     self.cmp = bz2.BZ2Decompressor()
+                    self.exception = IOError
                 else:
                     self.cmp = bz2.BZ2Compressor()
+
+            elif comptype == "xz":
+                try:
+                    import lzma
+                except ImportError:
+                    raise CompressionError("lzma module is not available")
+                if mode == "r":
+                    self.dbuf = b""
+                    self.cmp = lzma.LZMADecompressor()
+                    self.exception = lzma.LZMAError
+                else:
+                    self.cmp = lzma.LZMACompressor()
+
+            elif comptype != "tar":
+                raise CompressionError("unknown compression type %r" % comptype)
+
         except:
             if not self._extfileobj:
                 self.fileobj.close()
@@ -584,7 +602,7 @@
                 break
             try:
                 buf = self.cmp.decompress(buf)
-            except IOError:
+            except self.exception:
                 raise ReadError("invalid compressed data")
             self.dbuf += buf
             c += len(buf)
@@ -622,11 +640,14 @@
         return self.buf
 
     def getcomptype(self):
-        if self.buf.startswith(b"\037\213\010"):
+        if self.buf.startswith(b"\x1f\x8b\x08"):
             return "gz"
-        if self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
+        elif self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
             return "bz2"
-        return "tar"
+        elif self.buf.startswith((b"\x5d\x00\x00\x80", b"\xfd7zXZ")):
+            return "xz"
+        else:
+            return "tar"
 
     def close(self):
         self.fileobj.close()
@@ -1651,18 +1672,22 @@
            'r:'         open for reading exclusively uncompressed
            'r:gz'       open for reading with gzip compression
            'r:bz2'      open for reading with bzip2 compression
+           'r:xz'       open for reading with lzma compression
            'a' or 'a:'  open for appending, creating the file if necessary
            'w' or 'w:'  open for writing without compression
            'w:gz'       open for writing with gzip compression
            'w:bz2'      open for writing with bzip2 compression
+           'w:xz'       open for writing with lzma compression
 
            'r|*'        open a stream of tar blocks with transparent compression
            'r|'         open an uncompressed stream of tar blocks for reading
            'r|gz'       open a gzip compressed stream of tar blocks
            'r|bz2'      open a bzip2 compressed stream of tar blocks
+           'r|xz'       open an lzma compressed stream of tar blocks
            'w|'         open an uncompressed stream for writing
            'w|gz'       open a gzip compressed stream for writing
            'w|bz2'      open a bzip2 compressed stream for writing
+           'w|xz'       open an lzma compressed stream for writing
         """
 
         if not name and not fileobj:
@@ -1780,11 +1805,40 @@
         t._extfileobj = False
         return t
 
+    @classmethod
+    def xzopen(cls, name, mode="r", fileobj=None, preset=9, **kwargs):
+        """Open lzma compressed tar archive name for reading or writing.
+           Appending is not allowed.
+        """
+        if mode not in ("r", "w"):
+            raise ValueError("mode must be 'r' or 'w'")
+
+        try:
+            import lzma
+        except ImportError:
+            raise CompressionError("lzma module is not available")
+
+        if mode == "r":
+            # LZMAFile complains about a preset argument in read mode.
+            preset = None
+
+        fileobj = lzma.LZMAFile(filename=name if fileobj is None else None,
+                mode=mode, fileobj=fileobj, preset=preset)
+
+        try:
+            t = cls.taropen(name, mode, fileobj, **kwargs)
+        except (lzma.LZMAError, EOFError):
+            fileobj.close()
+            raise ReadError("not an lzma file")
+        t._extfileobj = False
+        return t
+
     # All *open() methods are registered here.
     OPEN_METH = {
         "tar": "taropen",   # uncompressed tar
         "gz":  "gzopen",    # gzip compressed tar
-        "bz2": "bz2open"    # bzip2 compressed tar
+        "bz2": "bz2open",   # bzip2 compressed tar
+        "xz":  "xzopen"     # lzma compressed tar
     }
 
     #--------------------------------------------------------------------------
diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py
--- a/Lib/test/test_minidom.py
+++ b/Lib/test/test_minidom.py
@@ -47,7 +47,7 @@
 
     def checkWholeText(self, node, s):
         t = node.wholeText
-        self.confirm(t == s, "looking for %s, found %s" % (repr(s), repr(t)))
+        self.confirm(t == s, "looking for %r, found %r" % (s, t))
 
     def testParseFromFile(self):
         with open(tstfile) as file:
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -21,6 +21,10 @@
     import bz2
 except ImportError:
     bz2 = None
+try:
+    import lzma
+except ImportError:
+    lzma = None
 
 def md5sum(data):
     return md5(data).hexdigest()
@@ -29,6 +33,7 @@
 tarname = support.findfile("testtar.tar")
 gzipname = os.path.join(TEMPDIR, "testtar.tar.gz")
 bz2name = os.path.join(TEMPDIR, "testtar.tar.bz2")
+xzname = os.path.join(TEMPDIR, "testtar.tar.xz")
 tmpname = os.path.join(TEMPDIR, "tmp.tar")
 
 md5_regtype = "65f477c818ad9e15f7feab0c6d37742f"
@@ -201,13 +206,15 @@
             _open = gzip.GzipFile
         elif self.mode.endswith(":bz2"):
             _open = bz2.BZ2File
+        elif self.mode.endswith(":xz"):
+            _open = lzma.LZMAFile
         else:
-            _open = open
+            _open = io.FileIO
 
         for char in (b'\0', b'a'):
             # Test if EOFHeaderError ('\0') and InvalidHeaderError ('a')
             # are ignored correctly.
-            with _open(tmpname, "wb") as fobj:
+            with _open(tmpname, "w") as fobj:
                 fobj.write(char * 1024)
                 fobj.write(tarfile.TarInfo("foo").tobuf())
 
@@ -222,9 +229,10 @@
 class MiscReadTest(CommonReadTest):
 
     def test_no_name_argument(self):
-        if self.mode.endswith("bz2"):
-            # BZ2File has no name attribute.
-            return
+        if self.mode.endswith(("bz2", "xz")):
+            # BZ2File and LZMAFile have no name attribute.
+            self.skipTest("no name attribute")
+
         with open(self.tarname, "rb") as fobj:
             tar = tarfile.open(fileobj=fobj, mode=self.mode)
             self.assertEqual(tar.name, os.path.abspath(fobj.name))
@@ -265,10 +273,12 @@
             _open = gzip.GzipFile
         elif self.mode.endswith(":bz2"):
             _open = bz2.BZ2File
+        elif self.mode.endswith(":xz"):
+            _open = lzma.LZMAFile
         else:
-            _open = open
-        fobj = _open(self.tarname, "rb")
-        try:
+            _open = io.FileIO
+
+        with _open(self.tarname) as fobj:
             fobj.seek(offset)
 
             # Test if the tarfile starts with the second member.
@@ -281,8 +291,6 @@
             self.assertEqual(tar.extractfile(t).read(), data,
                     "seek back did not work")
             tar.close()
-        finally:
-            fobj.close()
 
     def test_fail_comp(self):
         # For Gzip and Bz2 Tests: fail with a ReadError on an uncompressed file.
@@ -526,6 +534,18 @@
             testfunc(bz2name, "r|*")
             testfunc(bz2name, "r|bz2")
 
+        if lzma:
+            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:xz")
+            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|xz")
+            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r:")
+            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r|")
+
+            testfunc(xzname, "r")
+            testfunc(xzname, "r:*")
+            testfunc(xzname, "r:xz")
+            testfunc(xzname, "r|*")
+            testfunc(xzname, "r|xz")
+
     def test_detect_file(self):
         self._test_modes(self._testfunc_file)
 
@@ -1096,6 +1116,9 @@
             data = dec.decompress(data)
             self.assertTrue(len(dec.unused_data) == 0,
                     "found trailing data")
+        elif self.mode.endswith("xz"):
+            with lzma.LZMAFile(tmpname) as fobj:
+                data = fobj.read()
         else:
             with open(tmpname, "rb") as fobj:
                 data = fobj.read()
@@ -1510,6 +1533,12 @@
         self._create_testtar("w:bz2")
         self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
 
+    def test_append_lzma(self):
+        if lzma is None:
+            self.skipTest("lzma module not available")
+        self._create_testtar("w:xz")
+        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
+
     # Append mode is supposed to fail if the tarfile to append to
     # does not end with a zero block.
     def _test_error(self, data):
@@ -1788,6 +1817,21 @@
         self._test_partial_input("r:bz2")
 
 
+class LzmaMiscReadTest(MiscReadTest):
+    tarname = xzname
+    mode = "r:xz"
+class LzmaUstarReadTest(UstarReadTest):
+    tarname = xzname
+    mode = "r:xz"
+class LzmaStreamReadTest(StreamReadTest):
+    tarname = xzname
+    mode = "r|xz"
+class LzmaWriteTest(WriteTest):
+    mode = "w:xz"
+class LzmaStreamWriteTest(StreamWriteTest):
+    mode = "w|xz"
+
+
 def test_main():
     support.unlink(TEMPDIR)
     os.makedirs(TEMPDIR)
@@ -1850,6 +1894,20 @@
             Bz2PartialReadTest,
         ]
 
+    if lzma:
+        # Create testtar.tar.xz and add lzma-specific tests.
+        support.unlink(xzname)
+        with lzma.LZMAFile(xzname, "w") as tar:
+            tar.write(data)
+
+        tests += [
+            LzmaMiscReadTest,
+            LzmaUstarReadTest,
+            LzmaStreamReadTest,
+            LzmaWriteTest,
+            LzmaStreamWriteTest,
+        ]
+
     try:
         support.run_unittest(*tests)
     finally:
diff --git a/Lib/xml/dom/pulldom.py b/Lib/xml/dom/pulldom.py
--- a/Lib/xml/dom/pulldom.py
+++ b/Lib/xml/dom/pulldom.py
@@ -1,6 +1,5 @@
 import xml.sax
 import xml.sax.handler
-import types
 
 START_ELEMENT = "START_ELEMENT"
 END_ELEMENT = "END_ELEMENT"
@@ -334,10 +333,7 @@
     return DOMEventStream(stream, parser, bufsize)
 
 def parseString(string, parser=None):
-    try:
-        from io import StringIO
-    except ImportError:
-        from io import StringIO
+    from io import StringIO
 
     bufsize = len(string)
     buf = StringIO(string)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -408,6 +408,8 @@
 
 - Alias resource.error to OSError ala PEP 3151.
 
+- Issue #5689: Add support for lzma compression to the tarfile module.
+
 - Issue #13248: Turn 3.2's PendingDeprecationWarning into 3.3's
   DeprecationWarning.  It covers 'cgi.escape', 'importlib.abc.PyLoader',
   'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath',

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Dec 11 05:34:35 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 11 Dec 2011 05:34:35 +0100
Subject: [Python-checkins] Daily reference leaks (2bbe7dc920de): sum=0
Message-ID: <E1RZb7T-0004CV-SQ@ap.vmr.nerim.net>

results for 2bbe7dc920de on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogbSizE7', '-x']

From python-checkins at python.org  Mon Dec 12 01:25:52 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:52 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_error_handling_in_resiz?=
	=?utf8?b?ZV9jb21wYWN0KCk=?=
Message-ID: <E1RZtiK-0001Q9-MW@dinsdale.python.org>

http://hg.python.org/cpython/rev/21d7f5fdd34c
changeset:   73932:21d7f5fdd34c
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 20:04:56 2011 +0100
summary:
  Fix error handling in resize_compact()

files:
  Objects/unicodeobject.c |  20 ++++++++++++--------
  1 files changed, 12 insertions(+), 8 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -630,6 +630,7 @@
     Py_ssize_t struct_size;
     Py_ssize_t new_size;
     int share_wstr;
+    PyObject *new_unicode;
 
     assert(PyUnicode_IS_READY(unicode));
     char_size = PyUnicode_KIND(unicode);
@@ -639,22 +640,25 @@
         struct_size = sizeof(PyCompactUnicodeObject);
     share_wstr = _PyUnicode_SHARE_WSTR(unicode);
 
+    if (length > ((PY_SSIZE_T_MAX - struct_size) / char_size - 1)) {
+        Py_DECREF(unicode);
+        PyErr_NoMemory();
+        return NULL;
+    }
+    new_size = (struct_size + (length + 1) * char_size);
+
     _Py_DEC_REFTOTAL;
     _Py_ForgetReference(unicode);
 
-    if (length > ((PY_SSIZE_T_MAX - struct_size) / char_size - 1)) {
-        PyErr_NoMemory();
-        return NULL;
-    }
-    new_size = (struct_size + (length + 1) * char_size);
-
-    unicode = (PyObject *)PyObject_REALLOC((char *)unicode, new_size);
-    if (unicode == NULL) {
+    new_unicode = (PyObject *)PyObject_REALLOC((char *)unicode, new_size);
+    if (new_unicode == NULL) {
         PyObject_Del(unicode);
         PyErr_NoMemory();
         return NULL;
     }
+    unicode = new_unicode;
     _Py_NewReference(unicode);
+
     _PyUnicode_LENGTH(unicode) = length;
     if (share_wstr) {
         _PyUnicode_WSTR(unicode) = PyUnicode_DATA(unicode);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:53 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Move_the_slowest_UTF-8_deco?=
	=?utf8?q?der_to_its_own_subfunction?=
Message-ID: <E1RZtiL-0001QJ-GK@dinsdale.python.org>

http://hg.python.org/cpython/rev/8815966e881a
changeset:   73933:8815966e881a
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 20:09:03 2011 +0100
summary:
  Move the slowest UTF-8 decoder to its own subfunction

 * Create decode_utf8_errors()
 * Reuse unicode_fromascii()
 * decode_utf8_errors() doesn't refit at the beginning
 * Remove refit_partial_string(), use unicode_adjust_maxchar() instead

files:
  Objects/unicodeobject.c |  230 ++++++++++++---------------
  1 files changed, 100 insertions(+), 130 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -1784,7 +1784,7 @@
 static PyObject*
 unicode_fromascii(const unsigned char* s, Py_ssize_t size)
 {
-    PyObject *res;
+    PyObject *unicode;
 #ifdef Py_DEBUG
     const unsigned char *p;
     const unsigned char *end = s + size;
@@ -1794,11 +1794,12 @@
 #endif
     if (size == 1)
         return get_latin1_char(s[0]);
-    res = PyUnicode_New(size, 127);
-    if (!res)
-        return NULL;
-    memcpy(PyUnicode_1BYTE_DATA(res), s, size);
-    return res;
+    unicode = PyUnicode_New(size, 127);
+    if (!unicode)
+        return NULL;
+    memcpy(PyUnicode_1BYTE_DATA(unicode), s, size);
+    assert(_PyUnicode_CheckConsistency(unicode, 1));
+    return unicode;
 }
 
 static Py_UCS4
@@ -4320,126 +4321,38 @@
     return 65537;
 }
 
-/* Called when we encountered some error that wasn't detected in the original
-   scan, e.g. an encoded surrogate character. The original maxchar computation
-   may have been incorrect, so redo it. */
-static int
-refit_partial_string(PyObject **unicode, int kind, void *data, Py_ssize_t n)
-{
-    PyObject *tmp;
-    Py_ssize_t k;
-    Py_UCS4 maxchar;
-    for (k = 0, maxchar = 0; k < n; k++)
-        maxchar = Py_MAX(maxchar, PyUnicode_READ(kind, data, k));
-    tmp = PyUnicode_New(PyUnicode_GET_LENGTH(*unicode), maxchar);
-    if (tmp == NULL)
-        return -1;
-    PyUnicode_CopyCharacters(tmp, 0, *unicode, 0, n);
-    Py_DECREF(*unicode);
-    *unicode = tmp;
-    return 0;
-}
-
 /* Similar to PyUnicode_WRITE but may attempt to widen and resize the string
-   in case of errors. Implicit parameters: unicode, kind, data, has_errors,
-   onError. Potential resizing overallocates, so the result needs to shrink
-   at the end.
+   in case of errors. Implicit parameters: unicode, kind, data, onError.
+   Potential resizing overallocates, so the result needs to shrink at the end.
 */
-#define WRITE_MAYBE_FAIL(index, value)                                  \
-    do {                                                                \
-        if (has_errors) {                                               \
-            Py_ssize_t pos = index;                                     \
-            if (pos > PyUnicode_GET_LENGTH(unicode) &&                  \
-                unicode_resize(&unicode, pos + pos/8) < 0)              \
-                goto onError;                                           \
-            if (unicode_putchar(&unicode, &pos, value) < 0)             \
-                goto onError;                                           \
-        }                                                               \
-        else                                                            \
-            PyUnicode_WRITE(kind, data, index, value);                  \
+#define WRITE_MAYBE_FAIL(index, value)                              \
+    do {                                                            \
+        Py_ssize_t pos = index;                                     \
+        if (pos > PyUnicode_GET_LENGTH(unicode) &&                  \
+            unicode_resize(&unicode, pos + pos/8) < 0)              \
+            goto onError;                                           \
+        if (unicode_putchar(&unicode, &pos, value) < 0)             \
+            goto onError;                                           \
     } while (0)
 
 PyObject *
-PyUnicode_DecodeUTF8Stateful(const char *s,
-                             Py_ssize_t size,
-                             const char *errors,
-                             Py_ssize_t *consumed)
-{
-    const char *starts = s;
+decode_utf8_errors(const char *starts,
+                   Py_ssize_t size,
+                   const char *errors,
+                   Py_ssize_t *consumed,
+                   const char *s,
+                   PyObject *unicode,
+                   Py_ssize_t i)
+{
     int n;
     int k;
     Py_ssize_t startinpos;
     Py_ssize_t endinpos;
-    const char *e, *aligned_end;
-    PyObject *unicode;
+    const char *e = starts + size;
+    const char *aligned_end;
     const char *errmsg = "";
     PyObject *errorHandler = NULL;
     PyObject *exc = NULL;
-    Py_UCS4 maxchar = 0;
-    Py_ssize_t unicode_size;
-    Py_ssize_t i;
-    int kind;
-    void *data;
-    int has_errors = 0;
-
-    if (size == 0) {
-        if (consumed)
-            *consumed = 0;
-        return (PyObject *)PyUnicode_New(0, 0);
-    }
-    maxchar = utf8_max_char_size_and_char_count(s, size, &unicode_size);
-    /* When the string is ASCII only, just use memcpy and return.
-       unicode_size may be != size if there is an incomplete UTF-8
-       sequence at the end of the ASCII block.  */
-    if (maxchar < 128 && size == unicode_size) {
-        if (consumed)
-            *consumed = size;
-
-        if (size == 1)
-            return get_latin1_char((unsigned char)s[0]);
-
-        unicode = PyUnicode_New(unicode_size, maxchar);
-        if (!unicode)
-            return NULL;
-        Py_MEMCPY(PyUnicode_1BYTE_DATA(unicode), s, unicode_size);
-        assert(_PyUnicode_CheckConsistency(unicode, 1));
-        return unicode;
-    }
-
-    /* In case of errors, maxchar and size computation might be incorrect;
-       code below refits and resizes as necessary. */
-    unicode = PyUnicode_New(unicode_size, maxchar);
-    if (!unicode)
-        return NULL;
-    kind = PyUnicode_KIND(unicode);
-    data = PyUnicode_DATA(unicode);
-
-    /* Unpack UTF-8 encoded data */
-    i = 0;
-    e = s + size;
-    switch (kind) {
-    case PyUnicode_1BYTE_KIND:
-        has_errors = ucs1lib_utf8_try_decode(s, e, (Py_UCS1 *) data, &s, &i);
-        break;
-    case PyUnicode_2BYTE_KIND:
-        has_errors = ucs2lib_utf8_try_decode(s, e, (Py_UCS2 *) data, &s, &i);
-        break;
-    case PyUnicode_4BYTE_KIND:
-        has_errors = ucs4lib_utf8_try_decode(s, e, (Py_UCS4 *) data, &s, &i);
-        break;
-    }
-    if (!has_errors) {
-        /* Ensure the unicode size calculation was correct */
-        assert(i == unicode_size);
-        assert(s == e);
-        if (consumed)
-            *consumed = s-starts;
-        return unicode;
-    }
-    /* Fall through to the generic decoding loop for the rest of
-       the string */
-    if (refit_partial_string(&unicode, kind, data, i) < 0)
-        goto onError;
 
     aligned_end = (const char *) ((size_t) e & ~LONG_PTR_MASK);
 
@@ -4591,11 +4504,6 @@
         continue;
 
       utf8Error:
-        if (!has_errors) {
-            if (refit_partial_string(&unicode, kind, data, i) < 0)
-                goto onError;
-            has_errors = 1;
-        }
         if (unicode_decode_call_errorhandler(
                 errors, &errorHandler,
                 "utf8", errmsg,
@@ -4604,22 +4512,18 @@
             goto onError;
         /* Update data because unicode_decode_call_errorhandler might have
            re-created or resized the unicode object. */
-        data = PyUnicode_DATA(unicode);
-        kind = PyUnicode_KIND(unicode);
         aligned_end = (const char *) ((size_t) e & ~LONG_PTR_MASK);
     }
-    /* Ensure the unicode_size calculation above was correct: */
-    assert(has_errors || i == unicode_size);
-
     if (consumed)
         *consumed = s-starts;
 
     /* Adjust length and ready string when it contained errors and
        is of the old resizable kind. */
-    if (has_errors) {
-        if (PyUnicode_Resize(&unicode, i) < 0)
-            goto onError;
-    }
+    if (unicode_resize(&unicode, i) < 0)
+        goto onError;
+    unicode_adjust_maxchar(&unicode);
+    if (unicode == NULL)
+        goto onError;
 
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
@@ -4629,12 +4533,78 @@
   onError:
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
-    Py_DECREF(unicode);
+    Py_XDECREF(unicode);
     return NULL;
 }
-
 #undef WRITE_MAYBE_FAIL
 
+PyObject *
+PyUnicode_DecodeUTF8Stateful(const char *s,
+                             Py_ssize_t size,
+                             const char *errors,
+                             Py_ssize_t *consumed)
+{
+    Py_UCS4 maxchar = 0;
+    Py_ssize_t unicode_size;
+    int has_errors = 0;
+    PyObject *unicode;
+    int kind;
+    void *data;
+    const char *starts = s;
+    const char *e;
+    Py_ssize_t i;
+
+    if (size == 0) {
+        if (consumed)
+            *consumed = 0;
+        return (PyObject *)PyUnicode_New(0, 0);
+    }
+
+    maxchar = utf8_max_char_size_and_char_count(s, size, &unicode_size);
+
+    /* When the string is ASCII only, just use memcpy and return.
+       unicode_size may be != size if there is an incomplete UTF-8
+       sequence at the end of the ASCII block.  */
+    if (maxchar < 128 && size == unicode_size) {
+        if (consumed)
+            *consumed = size;
+        return unicode_fromascii(s, size);
+    }
+
+    unicode = PyUnicode_New(unicode_size, maxchar);
+    if (!unicode)
+        return NULL;
+    kind = PyUnicode_KIND(unicode);
+    data = PyUnicode_DATA(unicode);
+
+    /* Unpack UTF-8 encoded data */
+    i = 0;
+    e = starts + size;
+    switch (kind) {
+    case PyUnicode_1BYTE_KIND:
+        has_errors = ucs1lib_utf8_try_decode(s, e, (Py_UCS1 *) data, &s, &i);
+        break;
+    case PyUnicode_2BYTE_KIND:
+        has_errors = ucs2lib_utf8_try_decode(s, e, (Py_UCS2 *) data, &s, &i);
+        break;
+    case PyUnicode_4BYTE_KIND:
+        has_errors = ucs4lib_utf8_try_decode(s, e, (Py_UCS4 *) data, &s, &i);
+        break;
+    }
+    if (!has_errors) {
+        /* Ensure the unicode size calculation was correct */
+        assert(i == unicode_size);
+        assert(s == e);
+        if (consumed)
+            *consumed = size;
+        return unicode;
+    }
+
+    /* In case of errors, maxchar and size computation might be incorrect;
+       code below refits and resizes as necessary. */
+    return decode_utf8_errors(starts, size, errors, consumed, s, unicode, i);
+}
+
 #ifdef __APPLE__
 
 /* Simplified UTF-8 decoder using surrogateescape error handler,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:54 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:54 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Use_directly_unicode=5Fempt?=
 =?utf8?q?y_instead_of_PyUnicode=5FNew=280=2C_0=29?=
Message-ID: <E1RZtiM-0001QO-Dj@dinsdale.python.org>

http://hg.python.org/cpython/rev/7d4051e620fe
changeset:   73934:7d4051e620fe
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 21:44:00 2011 +0100
summary:
  Use directly unicode_empty instead of PyUnicode_New(0, 0)

files:
  Objects/unicodeobject.c |  18 ++++++++++++------
  1 files changed, 12 insertions(+), 6 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -2122,8 +2122,10 @@
 PyUnicode_FromWideChar(register const wchar_t *w, Py_ssize_t size)
 {
     if (w == NULL) {
-        if (size == 0)
-            return PyUnicode_New(0, 0);
+        if (size == 0) {
+            Py_INCREF(unicode_empty);
+            return unicode_empty;
+        }
         PyErr_BadInternalCall();
         return NULL;
     }
@@ -4557,7 +4559,8 @@
     if (size == 0) {
         if (consumed)
             *consumed = 0;
-        return (PyObject *)PyUnicode_New(0, 0);
+        Py_INCREF(unicode_empty);
+        return unicode_empty;
     }
 
     maxchar = utf8_max_char_size_and_char_count(s, size, &unicode_size);
@@ -12906,7 +12909,8 @@
         }
 
         if (slicelength <= 0) {
-            return PyUnicode_New(0, 0);
+            Py_INCREF(unicode_empty);
+            return unicode_empty;
         } else if (start == 0 && step == 1 &&
                    slicelength == PyUnicode_GET_LENGTH(self) &&
                    PyUnicode_CheckExact(self)) {
@@ -13582,8 +13586,10 @@
     if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:str",
                                      kwlist, &x, &encoding, &errors))
         return NULL;
-    if (x == NULL)
-        return PyUnicode_New(0, 0);
+    if (x == NULL) {
+        Py_INCREF(unicode_empty);
+        return unicode_empty;
+    }
     if (encoding == NULL && errors == NULL)
         return PyObject_Str(x);
     else

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:55 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:55 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Call_directly_PyUnicode=5FD?=
 =?utf8?q?ecodeUTF8Stateful=28=29_instead_of_PyUnicode=5FDecodeUTF8=28=29?=
Message-ID: <E1RZtiN-0001QS-7H@dinsdale.python.org>

http://hg.python.org/cpython/rev/9538a4e2ec08
changeset:   73935:9538a4e2ec08
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 21:53:09 2011 +0100
summary:
  Call directly PyUnicode_DecodeUTF8Stateful() instead of PyUnicode_DecodeUTF8()

 * Remove micro-optimization from PyUnicode_FromStringAndSize():
   PyUnicode_DecodeUTF8Stateful() has already these optimizations (for size=0
   and one ascii char).
 * Rename utf8_max_char_size_and_char_count() to utf8_scanner(), and remove an
   useless variable

files:
  Objects/unicodeobject.c |  47 ++++++++--------------------
  1 files changed, 14 insertions(+), 33 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -1717,28 +1717,10 @@
                         "Negative size passed to PyUnicode_FromStringAndSize");
         return NULL;
     }
-
-    /* If the Unicode data is known at construction time, we can apply
-       some optimizations which share commonly used objects.
-       Also, this means the input must be UTF-8, so fall back to the
-       UTF-8 decoder at the end. */
-    if (u != NULL) {
-
-        /* Optimization for empty strings */
-        if (size == 0 && unicode_empty != NULL) {
-            Py_INCREF(unicode_empty);
-            return unicode_empty;
-        }
-
-        /* Single characters are shared when using this constructor.
-           Restrict to ASCII, since the input must be UTF-8. */
-        if (size == 1 && (unsigned char)*u < 128)
-            return get_latin1_char((unsigned char)*u);
-
-        return PyUnicode_DecodeUTF8(u, size, NULL);
-    }
-
-    return (PyObject *)_PyUnicode_New(size);
+    if (u != NULL)
+        return PyUnicode_DecodeUTF8Stateful(u, size, NULL, NULL);
+    else
+        return (PyObject *)_PyUnicode_New(size);
 }
 
 PyObject *
@@ -1749,15 +1731,16 @@
         PyErr_SetString(PyExc_OverflowError, "input too long");
         return NULL;
     }
-
-    return PyUnicode_FromStringAndSize(u, size);
+    return PyUnicode_DecodeUTF8Stateful(u, (Py_ssize_t)size, NULL, NULL);
 }
 
 PyObject *
 _PyUnicode_FromId(_Py_Identifier *id)
 {
     if (!id->object) {
-        id->object = PyUnicode_FromString(id->string);
+        id->object = PyUnicode_DecodeUTF8Stateful(id->string,
+                                                  strlen(id->string),
+                                                  NULL, NULL);
         if (!id->object)
             return NULL;
         PyUnicode_InternInPlace(&id->object);
@@ -2443,7 +2426,7 @@
             {
                 /* UTF-8 */
                 const char *s = va_arg(count, const char*);
-                PyObject *str = PyUnicode_DecodeUTF8(s, strlen(s), "replace");
+                PyObject *str = PyUnicode_DecodeUTF8Stateful(s, strlen(s), "replace", NULL);
                 if (!str)
                     goto fail;
                 /* since PyUnicode_DecodeUTF8 returns already flexible
@@ -2482,7 +2465,7 @@
                     *callresult++ = NULL;
                 }
                 else {
-                    str_obj = PyUnicode_DecodeUTF8(str, strlen(str), "replace");
+                    str_obj = PyUnicode_DecodeUTF8Stateful(str, strlen(str), "replace", NULL);
                     if (!str_obj)
                         goto fail;
                     if (PyUnicode_READY(str_obj)) {
@@ -2947,7 +2930,7 @@
     if (normalize_encoding(encoding, lower, sizeof(lower))) {
         if ((strcmp(lower, "utf-8") == 0) ||
             (strcmp(lower, "utf8") == 0))
-            return PyUnicode_DecodeUTF8(s, size, errors);
+            return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
         else if ((strcmp(lower, "latin-1") == 0) ||
                  (strcmp(lower, "latin1") == 0) ||
                  (strcmp(lower, "iso-8859-1") == 0))
@@ -3260,7 +3243,7 @@
 #ifdef HAVE_MBCS
     return PyUnicode_DecodeMBCS(s, size, NULL);
 #elif defined(__APPLE__)
-    return PyUnicode_DecodeUTF8(s, size, "surrogateescape");
+    return PyUnicode_DecodeUTF8Stateful(s, size, "surrogateescape", NULL);
 #else
     PyInterpreterState *interp = PyThreadState_GET()->interp;
     /* Bootstrap check: if the filesystem codec is implemented in Python, we
@@ -4240,11 +4223,9 @@
    PyUnicode_DecodeUTF8Stateful.
    */
 static Py_UCS4
-utf8_max_char_size_and_char_count(const char *s, Py_ssize_t string_size,
-                                  Py_ssize_t *unicode_size)
+utf8_scanner(const unsigned char *p, Py_ssize_t string_size, Py_ssize_t *unicode_size)
 {
     Py_ssize_t char_count = 0;
-    const unsigned char *p = (const unsigned char *)s;
     const unsigned char *end = p + string_size;
     const unsigned char *aligned_end = (const unsigned char *) ((size_t) end & ~LONG_PTR_MASK);
 
@@ -4563,7 +4544,7 @@
         return unicode_empty;
     }
 
-    maxchar = utf8_max_char_size_and_char_count(s, size, &unicode_size);
+    maxchar = utf8_scanner((const unsigned char *)s, size, &unicode_size);
 
     /* When the string is ASCII only, just use memcpy and return.
        unicode_size may be != size if there is an incomplete UTF-8

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:55 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:55 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_unicode=5Ffromascii=28=29_d?=
 =?utf8?q?oesn=27t_check_string_content_twice_in_debug_mode?=
Message-ID: <E1RZtiN-0001QW-W8@dinsdale.python.org>

http://hg.python.org/cpython/rev/d728637f5d78
changeset:   73936:d728637f5d78
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 21:54:30 2011 +0100
summary:
  unicode_fromascii() doesn't check string content twice in debug mode

_PyUnicode_CheckConsistency() also checks string content.

files:
  Objects/unicodeobject.c |  11 ++++-------
  1 files changed, 4 insertions(+), 7 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -1768,15 +1768,12 @@
 unicode_fromascii(const unsigned char* s, Py_ssize_t size)
 {
     PyObject *unicode;
+    if (size == 1) {
 #ifdef Py_DEBUG
-    const unsigned char *p;
-    const unsigned char *end = s + size;
-    for (p=s; p < end; p++) {
-        assert(*p < 128);
-    }
-#endif
-    if (size == 1)
+        assert(s[0] < 128);
+#endif
         return get_latin1_char(s[0]);
+    }
     unicode = PyUnicode_New(size, 127);
     if (!unicode)
         return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:56 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:56 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_fixup=28=29_for_unchang?=
	=?utf8?q?ed_unicode_subtype?=
Message-ID: <E1RZtiO-0001Qa-OZ@dinsdale.python.org>

http://hg.python.org/cpython/rev/7a6ff1275b16
changeset:   73937:7a6ff1275b16
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 22:22:39 2011 +0100
summary:
  Fix fixup() for unchanged unicode subtype

If maxchar_new == 0 and self is a unicode subtype, return u instead of duplicating u.

files:
  Objects/unicodeobject.c |  70 ++++++++++++++--------------
  1 files changed, 35 insertions(+), 35 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -9211,6 +9211,7 @@
 {
     PyObject *u;
     Py_UCS4 maxchar_old, maxchar_new = 0;
+    PyObject *v;
 
     u = PyUnicode_Copy(self);
     if (u == NULL)
@@ -9222,9 +9223,19 @@
        everything is fine.  Otherwise we need to change the string kind
        and re-run the fix function. */
     maxchar_new = fixfct(u);
-    if (maxchar_new == 0)
-        /* do nothing, keep maxchar_new at 0 which means no changes. */;
-    else if (maxchar_new <= 127)
+
+    if (maxchar_new == 0) {
+        /* no changes */;
+        if (PyUnicode_CheckExact(self)) {
+            Py_DECREF(u);
+            Py_INCREF(self);
+            return self;
+        }
+        else
+            return u;
+    }
+
+    if (maxchar_new <= 127)
         maxchar_new = 127;
     else if (maxchar_new <= 255)
         maxchar_new = 255;
@@ -9233,41 +9244,30 @@
     else
         maxchar_new = MAX_UNICODE;
 
-    if (!maxchar_new && PyUnicode_CheckExact(self)) {
-        /* fixfct should return TRUE if it modified the buffer. If
-           FALSE, return a reference to the original buffer instead
-           (to save space, not time) */
-        Py_INCREF(self);
+    if (maxchar_new == maxchar_old)
+        return u;
+
+    /* In case the maximum character changed, we need to
+       convert the string to the new category. */
+    v = PyUnicode_New(PyUnicode_GET_LENGTH(self), maxchar_new);
+    if (v == NULL) {
         Py_DECREF(u);
-        return self;
-    }
-    else if (maxchar_new == maxchar_old) {
-        return u;
+        return NULL;
+    }
+    if (maxchar_new > maxchar_old) {
+        /* If the maxchar increased so that the kind changed, not all
+           characters are representable anymore and we need to fix the
+           string again. This only happens in very few cases. */
+        copy_characters(v, 0, self, 0, PyUnicode_GET_LENGTH(self));
+        maxchar_old = fixfct(v);
+        assert(maxchar_old > 0 && maxchar_old <= maxchar_new);
     }
     else {
-        /* In case the maximum character changed, we need to
-           convert the string to the new category. */
-        PyObject *v = PyUnicode_New(PyUnicode_GET_LENGTH(self), maxchar_new);
-        if (v == NULL) {
-            Py_DECREF(u);
-            return NULL;
-        }
-        if (maxchar_new > maxchar_old) {
-            /* If the maxchar increased so that the kind changed, not all
-               characters are representable anymore and we need to fix the
-               string again. This only happens in very few cases. */
-            copy_characters(v, 0, self, 0, PyUnicode_GET_LENGTH(self));
-            maxchar_old = fixfct(v);
-            assert(maxchar_old > 0 && maxchar_old <= maxchar_new);
-        }
-        else {
-            copy_characters(v, 0, u, 0, PyUnicode_GET_LENGTH(self));
-        }
-
-        Py_DECREF(u);
-        assert(_PyUnicode_CheckConsistency(v, 1));
-        return v;
-    }
+        copy_characters(v, 0, u, 0, PyUnicode_GET_LENGTH(self));
+    }
+    Py_DECREF(u);
+    assert(_PyUnicode_CheckConsistency(v, 1));
+    return v;
 }
 
 static Py_UCS4

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Create_unicode=5Fresult=5Fu?=
	=?utf8?q?nchanged=28=29_subfunction?=
Message-ID: <E1RZtiP-0001Qe-CB@dinsdale.python.org>

http://hg.python.org/cpython/rev/6d498f469377
changeset:   73938:6d498f469377
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 11 22:44:26 2011 +0100
summary:
  Create unicode_result_unchanged() subfunction

files:
  Objects/unicodeobject.c |  139 +++++++++++----------------
  1 files changed, 59 insertions(+), 80 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -487,6 +487,20 @@
         return unicode_result_wchar(unicode);
 }
 
+static PyObject*
+unicode_result_unchanged(PyObject *unicode)
+{
+    if (PyUnicode_CheckExact(unicode)) {
+        if (PyUnicode_READY(unicode) < 0)
+            return NULL;
+        Py_INCREF(unicode);
+        return unicode;
+    }
+    else
+        /* Subtype -- return genuine unicode string with the same value. */
+        return PyUnicode_Copy(unicode);
+}
+
 #ifdef HAVE_MBCS
 static OSVERSIONINFOEX winver;
 #endif
@@ -3563,7 +3577,7 @@
         PyErr_BadArgument();
         return (Py_UCS4)-1;
     }
-    if (index < 0 || index >= _PyUnicode_LENGTH(unicode)) {
+    if (index < 0 || index >= PyUnicode_GET_LENGTH(unicode)) {
         PyErr_SetString(PyExc_IndexError, "string index out of range");
         return (Py_UCS4)-1;
     }
@@ -3577,7 +3591,7 @@
         PyErr_BadArgument();
         return -1;
     }
-    if (index < 0 || index >= _PyUnicode_LENGTH(unicode)) {
+    if (index < 0 || index >= PyUnicode_GET_LENGTH(unicode)) {
         PyErr_SetString(PyExc_IndexError, "string index out of range");
         return -1;
     }
@@ -9675,10 +9689,8 @@
     if (right < 0)
         right = 0;
 
-    if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
+    if (left == 0 && right == 0)
+        return unicode_result_unchanged(self);
 
     if (left > PY_SSIZE_T_MAX - _PyUnicode_LENGTH(self) ||
         right > PY_SSIZE_T_MAX - (left + _PyUnicode_LENGTH(self))) {
@@ -10214,11 +10226,8 @@
         PyMem_FREE(buf1);
     if (release2)
         PyMem_FREE(buf2);
-    if (PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
-    return PyUnicode_Copy(self);
+    return unicode_result_unchanged(self);
+
   error:
     if (srelease && sbuf)
         PyMem_FREE(sbuf);
@@ -10334,15 +10343,13 @@
     if (!PyArg_ParseTuple(args, "n|O&:center", &width, convert_uc, &fillchar))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-
-    if (_PyUnicode_LENGTH(self) >= width && PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
-
-    marg = width - _PyUnicode_LENGTH(self);
+    if (PyUnicode_READY(self) < 0)
+        return NULL;
+
+    if (PyUnicode_GET_LENGTH(self) >= width)
+        return unicode_result_unchanged(self);
+
+    marg = width - PyUnicode_GET_LENGTH(self);
     left = marg / 2 + (marg & width & 1);
 
     return pad(self, left, marg - left, fillchar);
@@ -10851,10 +10858,8 @@
                 line_pos = 0;
         }
     }
-    if (!found && PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
+    if (!found)
+        return unicode_result_unchanged(self);
 
     /* Second pass: create output string and fill it */
     u = PyUnicode_New(j, PyUnicode_MAX_CHAR_VALUE(self));
@@ -11489,18 +11494,16 @@
     Py_ssize_t width;
     Py_UCS4 fillchar = ' ';
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-
     if (!PyArg_ParseTuple(args, "n|O&:ljust", &width, convert_uc, &fillchar))
         return NULL;
 
-    if (_PyUnicode_LENGTH(self) >= width && PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
-
-    return pad(self, 0, width - _PyUnicode_LENGTH(self), fillchar);
+    if (PyUnicode_READY(self) < 0)
+        return NULL;
+
+    if (PyUnicode_GET_LENGTH(self) >= width)
+        return unicode_result_unchanged(self);
+
+    return pad(self, 0, width - PyUnicode_GET_LENGTH(self), fillchar);
 }
 
 PyDoc_STRVAR(lower__doc__,
@@ -11575,14 +11578,7 @@
     end = Py_MIN(end, PyUnicode_GET_LENGTH(self));
 
     if (start == 0 && end == PyUnicode_GET_LENGTH(self))
-    {
-        if (PyUnicode_CheckExact(self)) {
-            Py_INCREF(self);
-            return self;
-        }
-        else
-            return PyUnicode_Copy(self);
-    }
+        return unicode_result_unchanged(self);
 
     length = end - start;
     if (length == 1)
@@ -11723,13 +11719,11 @@
         return unicode_empty;
     }
 
-    if (len == 1 && PyUnicode_CheckExact(str)) {
-        /* no repeat, return original string */
-        Py_INCREF(str);
-        return str;
-    }
-
-    if (PyUnicode_READY(str) == -1)
+    /* no repeat, return original string */
+    if (len == 1)
+        return unicode_result_unchanged(str);
+
+    if (PyUnicode_READY(str) < 0)
         return NULL;
 
     if (PyUnicode_GET_LENGTH(str) > PY_SSIZE_T_MAX / len) {
@@ -12079,15 +12073,13 @@
     if (!PyArg_ParseTuple(args, "n|O&:rjust", &width, convert_uc, &fillchar))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-
-    if (_PyUnicode_LENGTH(self) >= width && PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    }
-
-    return pad(self, width - _PyUnicode_LENGTH(self), 0, fillchar);
+    if (PyUnicode_READY(self) < 0)
+        return NULL;
+
+    if (PyUnicode_GET_LENGTH(self) >= width)
+        return unicode_result_unchanged(self);
+
+    return pad(self, width - PyUnicode_GET_LENGTH(self), 0, fillchar);
 }
 
 PyObject *
@@ -12380,12 +12372,7 @@
 static
 PyObject *unicode_str(PyObject *self)
 {
-    if (PyUnicode_CheckExact(self)) {
-        Py_INCREF(self);
-        return self;
-    } else
-        /* Subtype -- return genuine unicode string with the same value. */
-        return PyUnicode_Copy(self);
+    return unicode_result_unchanged(self);
 }
 
 PyDoc_STRVAR(swapcase__doc__,
@@ -12558,22 +12545,16 @@
     void *data;
     Py_UCS4 chr;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-
     if (!PyArg_ParseTuple(args, "n:zfill", &width))
         return NULL;
 
-    if (PyUnicode_GET_LENGTH(self) >= width) {
-        if (PyUnicode_CheckExact(self)) {
-            Py_INCREF(self);
-            return self;
-        }
-        else
-            return PyUnicode_Copy(self);
-    }
-
-    fill = width - _PyUnicode_LENGTH(self);
+    if (PyUnicode_READY(self) < 0)
+        return NULL;
+
+    if (PyUnicode_GET_LENGTH(self) >= width)
+        return unicode_result_unchanged(self);
+
+    fill = width - PyUnicode_GET_LENGTH(self);
 
     u = pad(self, fill, 0, '0');
 
@@ -12890,10 +12871,8 @@
             Py_INCREF(unicode_empty);
             return unicode_empty;
         } else if (start == 0 && step == 1 &&
-                   slicelength == PyUnicode_GET_LENGTH(self) &&
-                   PyUnicode_CheckExact(self)) {
-            Py_INCREF(self);
-            return self;
+                   slicelength == PyUnicode_GET_LENGTH(self)) {
+            return unicode_result_unchanged(self);
         } else if (step == 1) {
             return PyUnicode_Substring(self,
                                        start, start + slicelength);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:58 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_PyUnicode=5FIS=5FASCII=28?=
 =?utf8?q?=29_macro_ensures_that_the_string_is_ready?=
Message-ID: <E1RZtiQ-0001Qi-3Z@dinsdale.python.org>

http://hg.python.org/cpython/rev/d36c4ec5342b
changeset:   73939:d36c4ec5342b
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 01:24:20 2011 +0100
summary:
  PyUnicode_IS_ASCII() macro ensures that the string is ready

It has no sense to check if a not ready string is ASCII or not.

files:
  Include/unicodeobject.h |  12 +++++++-----
  1 files changed, 7 insertions(+), 5 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -424,10 +424,12 @@
 #define SSTATE_INTERNED_IMMORTAL 2
 
 /* Return true if the string contains only ASCII characters, or 0 if not. The
-   string may be compact (PyUnicode_IS_COMPACT_ASCII) or not. No type checks
-   or Ready calls are performed. */
-#define PyUnicode_IS_ASCII(op)                 \
-    (((PyASCIIObject*)op)->state.ascii)
+   string may be compact (PyUnicode_IS_COMPACT_ASCII) or not, but must be
+   ready. */
+#define PyUnicode_IS_ASCII(op)                   \
+    (assert(PyUnicode_Check(op)),                \
+     assert(PyUnicode_IS_READY(op)),             \
+     ((PyASCIIObject*)op)->state.ascii)
 
 /* Return true if the string is compact or 0 if not.
    No type checks or Ready calls are performed. */
@@ -437,7 +439,7 @@
 /* Return true if the string is a compact ASCII string (use PyASCIIObject
    structure), or 0 if not.  No type checks or Ready calls are performed. */
 #define PyUnicode_IS_COMPACT_ASCII(op)                 \
-    (PyUnicode_IS_ASCII(op) && PyUnicode_IS_COMPACT(op))
+    (((PyASCIIObject*)op)->state.ascii && PyUnicode_IS_COMPACT(op))
 
 enum PyUnicode_Kind {
 /* String contains only wstr byte characters.  This is only possible

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:58 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Rewrite_PyUnicode=5FAppend?=
 =?utf8?q?=28=29=3B_unicode=5Fmodifiable=28=29_is_more_strict?=
Message-ID: <E1RZtiQ-0001Qm-SK@dinsdale.python.org>

http://hg.python.org/cpython/rev/561f5e86fa47
changeset:   73940:561f5e86fa47
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 00:01:39 2011 +0100
summary:
  Rewrite PyUnicode_Append(); unicode_modifiable() is more strict

 * Rename unicode_resizable() to unicode_modifiable()
 * Rename _PyUnicode_Dirty() to unicode_check_modifiable() to make it clear
   that the function is private
 * Inline PyUnicode_Concat() and unicode_append_inplace() in PyUnicode_Append()
   to simplify the code
 * unicode_modifiable() return 0 if the hash has been computed or if the string
   is not an exact unicode string
 * Remove _PyUnicode_DIRTY(): no need to reset the hash anymore, because if the
   hash has already been computed, you cannot modify a string inplace anymore
 * PyUnicode_Concat() checks for integer overflow

files:
  Objects/unicodeobject.c |  197 ++++++++++++++-------------
  1 files changed, 101 insertions(+), 96 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -165,9 +165,6 @@
             *_to++ = (to_type) *_iter++;                \
     } while (0)
 
-/* The Unicode string has been modified: reset the hash */
-#define _PyUnicode_DIRTY(op) do { _PyUnicode_HASH(op) = -1; } while (0)
-
 /* This dictionary holds all interned unicode strings.  Note that references
    to strings in this dictionary are *not* counted in the string's ob_refcnt.
    When the interned string reaches a refcnt of 0 the string deallocation
@@ -226,6 +223,8 @@
     PyObject *to, Py_ssize_t to_start,
     PyObject *from, Py_ssize_t from_start,
     Py_ssize_t how_many);
+static int unicode_modifiable(PyObject *unicode);
+
 
 static PyObject *
 unicode_fromascii(const unsigned char *s, Py_ssize_t size);
@@ -645,10 +644,11 @@
     Py_ssize_t new_size;
     int share_wstr;
     PyObject *new_unicode;
-
     assert(PyUnicode_IS_READY(unicode));
+    assert(PyUnicode_IS_COMPACT(unicode));
+
     char_size = PyUnicode_KIND(unicode);
-    if (PyUnicode_IS_COMPACT_ASCII(unicode))
+    if (PyUnicode_IS_ASCII(unicode))
         struct_size = sizeof(PyASCIIObject);
     else
         struct_size = sizeof(PyCompactUnicodeObject);
@@ -676,7 +676,7 @@
     _PyUnicode_LENGTH(unicode) = length;
     if (share_wstr) {
         _PyUnicode_WSTR(unicode) = PyUnicode_DATA(unicode);
-        if (!PyUnicode_IS_COMPACT_ASCII(unicode))
+        if (!PyUnicode_IS_ASCII(unicode))
             _PyUnicode_WSTR_LENGTH(unicode) = length;
     }
     PyUnicode_WRITE(PyUnicode_KIND(unicode), PyUnicode_DATA(unicode),
@@ -691,8 +691,6 @@
     assert(!PyUnicode_IS_COMPACT(unicode));
     assert(Py_REFCNT(unicode) == 1);
 
-    _PyUnicode_DIRTY(unicode);
-
     if (PyUnicode_IS_READY(unicode)) {
         Py_ssize_t char_size;
         Py_ssize_t new_size;
@@ -1115,15 +1113,13 @@
 #endif
 
 static int
-_PyUnicode_Dirty(PyObject *unicode)
-{
-    assert(_PyUnicode_CHECK(unicode));
-    if (Py_REFCNT(unicode) != 1) {
+unicode_check_modifiable(PyObject *unicode)
+{
+    if (!unicode_modifiable(unicode)) {
         PyErr_SetString(PyExc_SystemError,
-                        "Cannot modify a string having more than 1 reference");
+                        "Cannot modify a string currently used");
         return -1;
     }
-    _PyUnicode_DIRTY(unicode);
     return 0;
 }
 
@@ -1289,7 +1285,7 @@
     if (how_many == 0)
         return 0;
 
-    if (_PyUnicode_Dirty(to))
+    if (unicode_check_modifiable(to))
         return -1;
 
     err = _copy_characters(to, to_start, from, from_start, how_many, 1);
@@ -1537,12 +1533,17 @@
 #endif
 
 static int
-unicode_resizable(PyObject *unicode)
-{
+unicode_modifiable(PyObject *unicode)
+{
+    assert(_PyUnicode_CHECK(unicode));
     if (Py_REFCNT(unicode) != 1)
         return 0;
+    if (_PyUnicode_HASH(unicode) != -1)
+        return 0;
     if (PyUnicode_CHECK_INTERNED(unicode))
         return 0;
+    if (!PyUnicode_CheckExact(unicode))
+        return 0;
 #ifdef Py_DEBUG
     /* singleton refcount is greater than 1 */
     assert(!unicode_is_singleton(unicode));
@@ -1577,7 +1578,7 @@
         return 0;
     }
 
-    if (!unicode_resizable(unicode)) {
+    if (!unicode_modifiable(unicode)) {
         PyObject *copy = resize_copy(unicode, length);
         if (copy == NULL)
             return -1;
@@ -3591,11 +3592,12 @@
         PyErr_BadArgument();
         return -1;
     }
+    assert(PyUnicode_IS_READY(unicode));
     if (index < 0 || index >= PyUnicode_GET_LENGTH(unicode)) {
         PyErr_SetString(PyExc_IndexError, "string index out of range");
         return -1;
     }
-    if (_PyUnicode_Dirty(unicode))
+    if (unicode_check_modifiable(unicode))
         return -1;
     PyUnicode_WRITE(PyUnicode_KIND(unicode), PyUnicode_DATA(unicode),
                     index, ch);
@@ -10566,6 +10568,7 @@
 {
     PyObject *u = NULL, *v = NULL, *w;
     Py_UCS4 maxchar, maxchar2;
+    Py_ssize_t u_len, v_len, new_len;
 
     /* Coerce the two arguments */
     u = PyUnicode_FromObject(left);
@@ -10585,18 +10588,25 @@
         return v;
     }
 
+    u_len = PyUnicode_GET_LENGTH(u);
+    v_len = PyUnicode_GET_LENGTH(v);
+    if (u_len > PY_SSIZE_T_MAX - v_len) {
+        PyErr_SetString(PyExc_OverflowError,
+                        "strings are too large to concat");
+        goto onError;
+    }
+    new_len = u_len + v_len;
+
     maxchar = PyUnicode_MAX_CHAR_VALUE(u);
     maxchar2 = PyUnicode_MAX_CHAR_VALUE(v);
     maxchar = Py_MAX(maxchar, maxchar2);
 
     /* Concat the two Unicode strings */
-    w = PyUnicode_New(
-        PyUnicode_GET_LENGTH(u) + PyUnicode_GET_LENGTH(v),
-        maxchar);
+    w = PyUnicode_New(new_len, maxchar);
     if (w == NULL)
         goto onError;
-    copy_characters(w, 0, u, 0, PyUnicode_GET_LENGTH(u));
-    copy_characters(w, PyUnicode_GET_LENGTH(u), v, 0, PyUnicode_GET_LENGTH(v));
+    copy_characters(w, 0, u, 0, u_len);
+    copy_characters(w, u_len, v, 0, v_len);
     Py_DECREF(u);
     Py_DECREF(v);
     assert(_PyUnicode_CheckConsistency(w, 1));
@@ -10608,15 +10618,41 @@
     return NULL;
 }
 
-static void
-unicode_append_inplace(PyObject **p_left, PyObject *right)
-{
+void
+PyUnicode_Append(PyObject **p_left, PyObject *right)
+{
+    PyObject *left, *res;
+    Py_UCS4 maxchar, maxchar2;
     Py_ssize_t left_len, right_len, new_len;
 
-    assert(PyUnicode_IS_READY(*p_left));
-    assert(PyUnicode_IS_READY(right));
-
-    left_len = PyUnicode_GET_LENGTH(*p_left);
+    if (p_left == NULL) {
+        if (!PyErr_Occurred())
+            PyErr_BadInternalCall();
+        return;
+    }
+    left = *p_left;
+    if (right == NULL || !PyUnicode_Check(left)) {
+        if (!PyErr_Occurred())
+            PyErr_BadInternalCall();
+        goto error;
+    }
+
+    if (PyUnicode_READY(left))
+        goto error;
+    if (PyUnicode_READY(right))
+        goto error;
+
+    /* Shortcuts */
+    if (left == unicode_empty) {
+        Py_DECREF(left);
+        Py_INCREF(right);
+        *p_left = right;
+        return;
+    }
+    if (right == unicode_empty)
+        return;
+
+    left_len = PyUnicode_GET_LENGTH(left);
     right_len = PyUnicode_GET_LENGTH(right);
     if (left_len > PY_SSIZE_T_MAX - right_len) {
         PyErr_SetString(PyExc_OverflowError,
@@ -10625,78 +10661,47 @@
     }
     new_len = left_len + right_len;
 
-    /* Now we own the last reference to 'left', so we can resize it
-     * in-place.
-     */
-    if (unicode_resize(p_left, new_len) != 0) {
-        /* XXX if _PyUnicode_Resize() fails, 'left' has been
-         * deallocated so it cannot be put back into
-         * 'variable'.  The MemoryError is raised when there
-         * is no value in 'variable', which might (very
-         * remotely) be a cause of incompatibilities.
-         */
-        goto error;
-    }
-    /* copy 'right' into the newly allocated area of 'left' */
-    copy_characters(*p_left, left_len, right, 0, right_len);
-    _PyUnicode_DIRTY(*p_left);
-    return;
-
-error:
-    Py_DECREF(*p_left);
-    *p_left = NULL;
-}
-
-void
-PyUnicode_Append(PyObject **p_left, PyObject *right)
-{
-    PyObject *left, *res;
-
-    if (p_left == NULL) {
-        if (!PyErr_Occurred())
-            PyErr_BadInternalCall();
-        return;
-    }
-    left = *p_left;
-    if (right == NULL || !PyUnicode_Check(left)) {
-        if (!PyErr_Occurred())
-            PyErr_BadInternalCall();
-        goto error;
-    }
-
-    if (PyUnicode_READY(left))
-        goto error;
-    if (PyUnicode_READY(right))
-        goto error;
-
-    if (PyUnicode_CheckExact(left) && left != unicode_empty
-        && PyUnicode_CheckExact(right) && right != unicode_empty
-        && unicode_resizable(left)
-        && (_PyUnicode_KIND(right) <= _PyUnicode_KIND(left)
-            || _PyUnicode_WSTR(left) != NULL))
-    {
+    if (unicode_modifiable(left)
+        && PyUnicode_CheckExact(right)
+        && PyUnicode_KIND(right) <= PyUnicode_KIND(left)
         /* Don't resize for ascii += latin1. Convert ascii to latin1 requires
            to change the structure size, but characters are stored just after
            the structure, and so it requires to move all characters which is
            not so different than duplicating the string. */
-        if (!(PyUnicode_IS_ASCII(left) && !PyUnicode_IS_ASCII(right)))
-        {
-            unicode_append_inplace(p_left, right);
-            assert(p_left == NULL || _PyUnicode_CheckConsistency(*p_left, 1));
-            return;
-        }
-    }
-
-    res = PyUnicode_Concat(left, right);
-    if (res == NULL)
-        goto error;
-    Py_DECREF(left);
-    *p_left = res;
+        && !(PyUnicode_IS_ASCII(left) && !PyUnicode_IS_ASCII(right)))
+    {
+        /* append inplace */
+        if (unicode_resize(p_left, new_len) != 0) {
+            /* XXX if _PyUnicode_Resize() fails, 'left' has been
+             * deallocated so it cannot be put back into
+             * 'variable'.  The MemoryError is raised when there
+             * is no value in 'variable', which might (very
+             * remotely) be a cause of incompatibilities.
+             */
+            goto error;
+        }
+        /* copy 'right' into the newly allocated area of 'left' */
+        copy_characters(*p_left, left_len, right, 0, right_len);
+    }
+    else {
+        maxchar = PyUnicode_MAX_CHAR_VALUE(left);
+        maxchar2 = PyUnicode_MAX_CHAR_VALUE(right);
+        maxchar = Py_MAX(maxchar, maxchar2);
+
+        /* Concat the two Unicode strings */
+        res = PyUnicode_New(new_len, maxchar);
+        if (res == NULL)
+            goto error;
+        copy_characters(res, 0, left, 0, left_len);
+        copy_characters(res, left_len, right, 0, right_len);
+        Py_DECREF(left);
+        *p_left = res;
+    }
+    assert(_PyUnicode_CheckConsistency(*p_left, 1));
     return;
 
 error:
-    Py_DECREF(*p_left);
-    *p_left = NULL;
+    Py_CLEAR(*p_left);
 }
 
 void

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:25:59 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:25:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_resize=5Fcopy=28=29_now_sup?=
	=?utf8?q?ports_legacy_ready_strings?=
Message-ID: <E1RZtiR-0001Qr-LX@dinsdale.python.org>

http://hg.python.org/cpython/rev/0eea55925d1e
changeset:   73941:0eea55925d1e
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 00:13:42 2011 +0100
summary:
  resize_copy() now supports legacy ready strings

files:
  Include/unicodeobject.h |   4 ++++
  Objects/unicodeobject.c |  28 +++++++++++++++-------------
  2 files changed, 19 insertions(+), 13 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -220,6 +220,7 @@
        - compact ascii:
 
          * structure = PyASCIIObject
+         * test: PyUnicode_IS_COMPACT_ASCII(op)
          * kind = PyUnicode_1BYTE_KIND
          * compact = 1
          * ascii = 1
@@ -231,6 +232,7 @@
        - compact:
 
          * structure = PyCompactUnicodeObject
+         * test: PyUnicode_IS_ASCII(op) && !PyUnicode_IS_COMPACT(op)
          * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
            PyUnicode_4BYTE_KIND
          * compact = 1
@@ -247,6 +249,7 @@
        - legacy string, not ready:
 
          * structure = PyUnicodeObject
+         * test: kind == PyUnicode_WCHAR_KIND
          * length = 0 (use wstr_length)
          * hash = -1
          * kind = PyUnicode_WCHAR_KIND
@@ -262,6 +265,7 @@
        - legacy string, ready:
 
          * structure = PyUnicodeObject structure
+         * test: !PyUnicode_IS_COMPACT(op) && kind != PyUnicode_WCHAR_KIND
          * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
            PyUnicode_4BYTE_KIND
          * compact = 0
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -688,20 +688,26 @@
 resize_inplace(PyObject *unicode, Py_ssize_t length)
 {
     wchar_t *wstr;
+    Py_ssize_t new_size;
     assert(!PyUnicode_IS_COMPACT(unicode));
     assert(Py_REFCNT(unicode) == 1);
 
     if (PyUnicode_IS_READY(unicode)) {
         Py_ssize_t char_size;
-        Py_ssize_t new_size;
         int share_wstr, share_utf8;
         void *data;
 
         data = _PyUnicode_DATA_ANY(unicode);
-        assert(data != NULL);
         char_size = PyUnicode_KIND(unicode);
         share_wstr = _PyUnicode_SHARE_WSTR(unicode);
         share_utf8 = _PyUnicode_SHARE_UTF8(unicode);
+
+        if (length > (PY_SSIZE_T_MAX / char_size - 1)) {
+            PyErr_NoMemory();
+            return -1;
+        }
+        new_size = (length + 1) * char_size;
+
         if (!share_utf8 && _PyUnicode_HAS_UTF8_MEMORY(unicode))
         {
             PyObject_DEL(_PyUnicode_UTF8(unicode));
@@ -709,12 +715,6 @@
             _PyUnicode_UTF8_LENGTH(unicode) = 0;
         }
 
-        if (length > (PY_SSIZE_T_MAX / char_size - 1)) {
-            PyErr_NoMemory();
-            return -1;
-        }
-        new_size = (length + 1) * char_size;
-
         data = (PyObject *)PyObject_REALLOC(data, new_size);
         if (data == NULL) {
             PyErr_NoMemory();
@@ -743,8 +743,9 @@
         PyErr_NoMemory();
         return -1;
     }
+    new_size = sizeof(wchar_t) * (length + 1);
     wstr =  _PyUnicode_WSTR(unicode);
-    wstr = PyObject_REALLOC(wstr, sizeof(wchar_t) * (length + 1));
+    wstr = PyObject_REALLOC(wstr, new_size);
     if (!wstr) {
         PyErr_NoMemory();
         return -1;
@@ -760,9 +761,11 @@
 resize_copy(PyObject *unicode, Py_ssize_t length)
 {
     Py_ssize_t copy_length;
-    if (PyUnicode_IS_COMPACT(unicode)) {
+    if (_PyUnicode_KIND(unicode) != PyUnicode_WCHAR_KIND) {
         PyObject *copy;
-        assert(PyUnicode_IS_READY(unicode));
+
+        if (PyUnicode_READY(unicode) < 0)
+            return NULL;
 
         copy = PyUnicode_New(length, PyUnicode_MAX_CHAR_VALUE(unicode));
         if (copy == NULL)
@@ -774,8 +777,7 @@
     }
     else {
         PyObject *w;
-        assert(_PyUnicode_WSTR(unicode) != NULL);
-        assert(_PyUnicode_DATA_ANY(unicode) == NULL);
+
         w = (PyObject*)_PyUnicode_New(length);
         if (w == NULL)
             return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 01:51:31 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 01:51:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Make_PyUnicode=5FCopy=28=29?=
 =?utf8?b?IHByaXZhdGUgPT4gX1B5VW5pY29kZV9Db3B5KCk=?=
Message-ID: <E1RZu79-0002rn-A8@dinsdale.python.org>

http://hg.python.org/cpython/rev/9c76bb654809
changeset:   73942:9c76bb654809
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 01:53:47 2011 +0100
summary:
  Make PyUnicode_Copy() private => _PyUnicode_Copy()

Undocument the function.

Make also decode_utf8_errors() as private (static).

files:
  Doc/c-api/unicode.rst   |   7 -------
  Doc/whatsnew/3.3.rst    |   4 ++--
  Include/unicodeobject.h |   4 +++-
  Objects/codeobject.c    |   2 +-
  Objects/unicodeobject.c |  12 ++++++------
  5 files changed, 12 insertions(+), 17 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -386,13 +386,6 @@
    .. versionadded:: 3.3
 
 
-.. c:function:: PyObject* PyUnicode_Copy(PyObject *unicode)
-
-   Get a new copy of a Unicode object.
-
-   .. versionadded:: 3.3
-
-
 .. c:function:: PyObject* PyUnicode_FromKindAndData(int kind, const void *buffer, \
                                                     Py_ssize_t size)
 
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -769,8 +769,8 @@
  * :c:macro:`PyUnicode_GET_DATA_SIZE`: use
    ``PyUnicode_GET_LENGTH(str) * PyUnicode_KIND(str)`` (only work on ready
    strings)
- * :c:func:`PyUnicode_AsUnicodeCopy`: use :c:func:`PyUnicode_AsUCS4Copy`,
-   :c:func:`PyUnicode_AsWideCharString` or :c:func:`PyUnicode_Copy`
+ * :c:func:`PyUnicode_AsUnicodeCopy`: use :c:func:`PyUnicode_AsUCS4Copy` or
+   :c:func:`PyUnicode_AsWideCharString`
 
 Functions and macros manipulating Py_UNICODE* strings:
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -616,9 +616,11 @@
 #endif
 
 /* Get a copy of a Unicode string. */
-PyAPI_FUNC(PyObject*) PyUnicode_Copy(
+#ifndef Py_LIMITED_API
+PyAPI_FUNC(PyObject*) _PyUnicode_Copy(
     PyObject *unicode
     );
+#endif
 
 /* Copy character from one unicode object into another, this function performs
    character conversion when necessary and falls back to memcpy if possible.
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -249,7 +249,7 @@
             return NULL;
         }
         else {
-            item = PyUnicode_Copy(item);
+            item = _PyUnicode_Copy(item);
             if (item == NULL) {
                 Py_DECREF(newtuple);
                 return NULL;
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -497,7 +497,7 @@
     }
     else
         /* Subtype -- return genuine unicode string with the same value. */
-        return PyUnicode_Copy(unicode);
+        return _PyUnicode_Copy(unicode);
 }
 
 #ifdef HAVE_MBCS
@@ -1961,7 +1961,7 @@
 }
 
 PyObject*
-PyUnicode_Copy(PyObject *unicode)
+_PyUnicode_Copy(PyObject *unicode)
 {
     Py_ssize_t length;
     PyObject *copy;
@@ -2832,7 +2832,7 @@
     if (PyUnicode_Check(obj)) {
         /* For a Unicode subtype that's not a Unicode object,
            return a true Unicode object with the same data. */
-        return PyUnicode_Copy(obj);
+        return _PyUnicode_Copy(obj);
     }
     PyErr_Format(PyExc_TypeError,
                  "Can't convert '%.100s' object to str implicitly",
@@ -4333,7 +4333,7 @@
             goto onError;                                           \
     } while (0)
 
-PyObject *
+static PyObject *
 decode_utf8_errors(const char *starts,
                    Py_ssize_t size,
                    const char *errors,
@@ -9231,7 +9231,7 @@
     Py_UCS4 maxchar_old, maxchar_new = 0;
     PyObject *v;
 
-    u = PyUnicode_Copy(self);
+    u = _PyUnicode_Copy(self);
     if (u == NULL)
         return NULL;
     maxchar_old = PyUnicode_MAX_CHAR_VALUE(u);
@@ -12753,7 +12753,7 @@
 static PyObject *
 unicode_getnewargs(PyObject *v)
 {
-    PyObject *copy = PyUnicode_Copy(v);
+    PyObject *copy = _PyUnicode_Copy(v);
     if (!copy)
         return NULL;
     return Py_BuildValue("(N)", copy);

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Dec 12 05:35:25 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 12 Dec 2011 05:35:25 +0100
Subject: [Python-checkins] Daily reference leaks (9c76bb654809): sum=0
Message-ID: <E1RZxbp-0003ov-EO@ap.vmr.nerim.net>

results for 9c76bb654809 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog64t0OT', '-x']

From python-checkins at python.org  Mon Dec 12 05:54:38 2011
From: python-checkins at python.org (meador.inge)
Date: Mon, 12 Dec 2011 05:54:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313544=3A_Add_=5F?=
 =?utf8?q?=5Fqualname=5F=5F_to_functools=2EWRAPPER=5FASSIGNMENTS=2E?=
Message-ID: <E1RZxuQ-0004Fe-VA@dinsdale.python.org>

http://hg.python.org/cpython/rev/963e98f5ad31
changeset:   73943:963e98f5ad31
user:        Meador Inge <meadori at gmail.com>
date:        Sun Dec 11 22:37:31 2011 -0600
summary:
  Issue #13544: Add __qualname__ to functools.WRAPPER_ASSIGNMENTS.

Patch by Filip Gruszczy?ski.

files:
  Lib/functools.py           |   3 ++-
  Lib/test/test_functools.py |  12 +++++++++---
  2 files changed, 11 insertions(+), 4 deletions(-)


diff --git a/Lib/functools.py b/Lib/functools.py
--- a/Lib/functools.py
+++ b/Lib/functools.py
@@ -21,7 +21,8 @@
 # update_wrapper() and wraps() are tools to help write
 # wrapper functions that can handle naive introspection
 
-WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__doc__', '__annotations__')
+WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
+                       '__annotations__')
 WRAPPER_UPDATES = ('__dict__',)
 def update_wrapper(wrapper,
                    wrapped,
diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -226,6 +226,7 @@
         self.check_wrapper(wrapper, f)
         self.assertIs(wrapper.__wrapped__, f)
         self.assertEqual(wrapper.__name__, 'f')
+        self.assertEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.attr, 'This is also a test')
         self.assertEqual(wrapper.__annotations__['a'], 'This is a new annotation')
         self.assertNotIn('b', wrapper.__annotations__)
@@ -246,6 +247,7 @@
         functools.update_wrapper(wrapper, f, (), ())
         self.check_wrapper(wrapper, f, (), ())
         self.assertEqual(wrapper.__name__, 'wrapper')
+        self.assertNotEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.__doc__, None)
         self.assertEqual(wrapper.__annotations__, {})
         self.assertFalse(hasattr(wrapper, 'attr'))
@@ -263,6 +265,7 @@
         functools.update_wrapper(wrapper, f, assign, update)
         self.check_wrapper(wrapper, f, assign, update)
         self.assertEqual(wrapper.__name__, 'wrapper')
+        self.assertNotEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.__doc__, None)
         self.assertEqual(wrapper.attr, 'This is a different test')
         self.assertEqual(wrapper.dict_attr, f.dict_attr)
@@ -309,17 +312,18 @@
         def wrapper():
             pass
         self.check_wrapper(wrapper, f)
-        return wrapper
+        return wrapper, f
 
     def test_default_update(self):
-        wrapper = self._default_update()
+        wrapper, f = self._default_update()
         self.assertEqual(wrapper.__name__, 'f')
+        self.assertEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.attr, 'This is also a test')
 
     @unittest.skipIf(not sys.flags.optimize <= 1,
                      "Docstrings are omitted with -O2 and above")
     def test_default_update_doc(self):
-        wrapper = self._default_update()
+        wrapper, _ = self._default_update()
         self.assertEqual(wrapper.__doc__, 'This is a test')
 
     def test_no_update(self):
@@ -332,6 +336,7 @@
             pass
         self.check_wrapper(wrapper, f, (), ())
         self.assertEqual(wrapper.__name__, 'wrapper')
+        self.assertNotEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.__doc__, None)
         self.assertFalse(hasattr(wrapper, 'attr'))
 
@@ -351,6 +356,7 @@
             pass
         self.check_wrapper(wrapper, f, assign, update)
         self.assertEqual(wrapper.__name__, 'wrapper')
+        self.assertNotEqual(wrapper.__qualname__, f.__qualname__)
         self.assertEqual(wrapper.__doc__, None)
         self.assertEqual(wrapper.attr, 'This is a different test')
         self.assertEqual(wrapper.dict_attr, f.dict_attr)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 07:31:21 2011
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 12 Dec 2011 07:31:21 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTcz?=
 =?utf8?q?=3A_The_csv=2Ewriter_now_uses_the_repr=28=29_for_floats_rather_t?=
 =?utf8?q?han_str=28=29=2E?=
Message-ID: <E1RZzQ1-0003VH-No@dinsdale.python.org>

http://hg.python.org/cpython/rev/bf7329190ca6
changeset:   73944:bf7329190ca6
branch:      2.7
parent:      73922:fd5f43737546
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Dec 11 22:31:09 2011 -0800
summary:
  Issue #13573: The csv.writer now uses the repr() for floats rather than str().

files:
  Lib/test/test_csv.py |  16 ++++++++++++++--
  Misc/NEWS            |   3 +++
  Modules/_csv.c       |   6 +++++-
  3 files changed, 22 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -207,6 +207,18 @@
             fileobj.close()
             os.unlink(name)
 
+    def test_write_float(self):
+        # Issue 13573: loss of precision because csv.writer
+        # uses str() for floats instead of repr()
+        orig_row = [1.234567890123, 1.0/7.0, 'abc']
+        f = StringIO()
+        c = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
+        c.writerow(orig_row)
+        f.seek(0)
+        c = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
+        new_row = next(c)
+        self.assertEqual(orig_row, new_row)
+
     def _read_test(self, input, expect, **kwargs):
         reader = csv.reader(input, **kwargs)
         result = list(reader)
@@ -784,7 +796,7 @@
         try:
             writer = csv.writer(fileobj, dialect="excel")
             writer.writerow(a)
-            expected = ",".join([str(i) for i in a])+"\r\n"
+            expected = ",".join([repr(i) for i in a])+"\r\n"
             fileobj.seek(0)
             self.assertEqual(fileobj.read(), expected)
         finally:
@@ -800,7 +812,7 @@
         try:
             writer = csv.writer(fileobj, dialect="excel")
             writer.writerow(a)
-            expected = ",".join([str(i) for i in a])+"\r\n"
+            expected = ",".join([repr(i) for i in a])+"\r\n"
             fileobj.seek(0)
             self.assertEqual(fileobj.read(), expected)
         finally:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -85,6 +85,9 @@
 - tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
   other than 900k.
 
+- Issue #13573: The csv.writer now uses the repr() for floats rather than str().
+  This allows floats to round-trip without loss of precision.
+
 - Issue #13439: Fix many errors in turtle docstrings.
 
 - Issue #12856: Ensure child processes do not inherit the parent's random
diff --git a/Modules/_csv.c b/Modules/_csv.c
--- a/Modules/_csv.c
+++ b/Modules/_csv.c
@@ -1184,7 +1184,11 @@
         else {
             PyObject *str;
 
-            str = PyObject_Str(field);
+            if (PyFloat_Check(field)) {
+                str = PyObject_Repr(field);
+            } else {
+                str = PyObject_Str(field);
+            }
             Py_DECREF(field);
             if (str == NULL)
                 return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 10:27:33 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Mon, 12 Dec 2011 10:27:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Add_missing_versionchanged_?=
	=?utf8?q?information=2E?=
Message-ID: <E1Ra2AX-00038B-1H@dinsdale.python.org>

http://hg.python.org/cpython/rev/e9a57c3ef635
changeset:   73945:e9a57c3ef635
parent:      73943:963e98f5ad31
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Mon Dec 12 10:22:56 2011 +0100
summary:
  Add missing versionchanged information.

files:
  Doc/library/tarfile.rst |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -32,6 +32,9 @@
   character devices and block devices and is able to acquire and restore file
   information like timestamp, access permissions and owner.
 
+.. versionchanged:: 3.3
+   Added support for :mod:`lzma` compression.
+
 
 .. function:: open(name=None, mode='r', fileobj=None, bufsize=10240, \*\*kwargs)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 13:09:06 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 13:09:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_PyUnicode=5FResize=28?=
 =?utf8?q?=29_for_compact_string=3A_leave_the_string_unchanged_on_error?=
Message-ID: <E1Ra4gs-00014h-RE@dinsdale.python.org>

http://hg.python.org/cpython/rev/61a2cae5308c
changeset:   73946:61a2cae5308c
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 13:08:33 2011 +0100
summary:
  Fix PyUnicode_Resize() for compact string: leave the string unchanged on error

Fix also PyUnicode_Resize() doc

files:
  Include/unicodeobject.h |  13 ++++-------
  Objects/unicodeobject.c |  31 +++++++++-------------------
  2 files changed, 15 insertions(+), 29 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -768,18 +768,15 @@
 PyAPI_FUNC(Py_UNICODE) PyUnicode_GetMax(void);
 #endif
 
-/* Resize an Unicode object allocated by the legacy API (e.g.
-   PyUnicode_FromUnicode). Unicode objects allocated by the new API (e.g.
-   PyUnicode_New) cannot be resized by this function.
-
-   The length is a number of characters (and not the number of Py_UNICODE characters).
+/* Resize an Unicode object. The length is the number of characters, except
+   if the kind of the string is PyUnicode_WCHAR_KIND: in this case, the length
+   is the number of Py_UNICODE characters.
 
    *unicode is modified to point to the new (resized) object and 0
    returned on success.
 
-   If the refcount on the object is 1, the function resizes the string in
-   place, which is usually faster than allocating a new string (and copy
-   characters).
+   Try to resize the string in place (which is usually faster than allocating
+   a new string and copy characters), or create a new string.
 
    Error handling is implemented as follows: an exception is set, -1
    is returned and *unicode left untouched. */
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -655,7 +655,6 @@
     share_wstr = _PyUnicode_SHARE_WSTR(unicode);
 
     if (length > ((PY_SSIZE_T_MAX - struct_size) / char_size - 1)) {
-        Py_DECREF(unicode);
         PyErr_NoMemory();
         return NULL;
     }
@@ -666,7 +665,7 @@
 
     new_unicode = (PyObject *)PyObject_REALLOC((char *)unicode, new_size);
     if (new_unicode == NULL) {
-        PyObject_Del(unicode);
+        _Py_NewReference(unicode);
         PyErr_NoMemory();
         return NULL;
     }
@@ -834,8 +833,9 @@
     new_size = sizeof(Py_UNICODE) * ((size_t)length + 1);
     _PyUnicode_WSTR(unicode) = (Py_UNICODE*) PyObject_MALLOC(new_size);
     if (!_PyUnicode_WSTR(unicode)) {
+        Py_DECREF(unicode);
         PyErr_NoMemory();
-        goto onError;
+        return NULL;
     }
 
     /* Initialize the first element to guard against cases where
@@ -860,13 +860,6 @@
     _PyUnicode_UTF8_LENGTH(unicode) = 0;
     assert(_PyUnicode_CheckConsistency((PyObject *)unicode, 0));
     return unicode;
-
-  onError:
-    /* XXX UNREF/NEWREF interface should be more symmetrical */
-    _Py_DEC_REFTOTAL;
-    _Py_ForgetReference((PyObject *)unicode);
-    PyObject_Del(unicode);
-    return NULL;
 }
 
 static const char*
@@ -1506,15 +1499,10 @@
         PyObject_DEL(_PyUnicode_WSTR(unicode));
     if (_PyUnicode_HAS_UTF8_MEMORY(unicode))
         PyObject_DEL(_PyUnicode_UTF8(unicode));
-
-    if (PyUnicode_IS_COMPACT(unicode)) {
-        Py_TYPE(unicode)->tp_free(unicode);
-    }
-    else {
-        if (_PyUnicode_DATA_ANY(unicode))
-            PyObject_DEL(_PyUnicode_DATA_ANY(unicode));
-        Py_TYPE(unicode)->tp_free(unicode);
-    }
+    if (!PyUnicode_IS_COMPACT(unicode) && _PyUnicode_DATA_ANY(unicode))
+        PyObject_DEL(_PyUnicode_DATA_ANY(unicode));
+
+    Py_TYPE(unicode)->tp_free(unicode);
 }
 
 #ifdef Py_DEBUG
@@ -1590,9 +1578,10 @@
     }
 
     if (PyUnicode_IS_COMPACT(unicode)) {
-        *p_unicode = resize_compact(unicode, length);
-        if (*p_unicode == NULL)
+        PyObject *new_unicode = resize_compact(unicode, length);
+        if (new_unicode == NULL)
             return -1;
+        *p_unicode = new_unicode;
         assert(_PyUnicode_CheckConsistency(*p_unicode, 0));
         return 0;
     }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 13:21:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 12 Dec 2011 13:21:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_PyUnicode=5FResize=28=29=3A?=
 =?utf8?q?_warn_about_canonical_representation?=
Message-ID: <E1Ra4tJ-00027I-QC@dinsdale.python.org>

http://hg.python.org/cpython/rev/3ad7d01acbf4
changeset:   73947:3ad7d01acbf4
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 12 13:24:15 2011 +0100
summary:
  PyUnicode_Resize(): warn about canonical representation

Call also directly unicode_resize() in unicodeobject.c

files:
  Include/unicodeobject.h |   5 ++++-
  Objects/unicodeobject.c |  25 +++++++++++++------------
  2 files changed, 17 insertions(+), 13 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -779,7 +779,10 @@
    a new string and copy characters), or create a new string.
 
    Error handling is implemented as follows: an exception is set, -1
-   is returned and *unicode left untouched. */
+   is returned and *unicode left untouched.
+
+   WARNING: The function doesn't check string content, the result may not be a
+            string in canonical representation. */
 
 PyAPI_FUNC(int) PyUnicode_Resize(
     PyObject **unicode,         /* Pointer to the Unicode object */
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -5040,7 +5040,7 @@
         *consumed = (const char *)q-starts;
 
     /* Adjust length */
-    if (PyUnicode_Resize(&unicode, outpos) < 0)
+    if (unicode_resize(&unicode, outpos) < 0)
         goto onError;
 
     Py_XDECREF(errorHandler);
@@ -5404,7 +5404,7 @@
         *consumed = (const char *)q-starts;
 
     /* Adjust length */
-    if (PyUnicode_Resize(&unicode, outpos) < 0)
+    if (unicode_resize(&unicode, outpos) < 0)
         goto onError;
 
     Py_XDECREF(errorHandler);
@@ -5824,7 +5824,7 @@
     }
 #undef WRITECHAR
 
-    if (PyUnicode_Resize(&v, i) < 0)
+    if (unicode_resize(&v, i) < 0)
         goto onError;
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
@@ -6086,7 +6086,7 @@
       nextByte:
         ;
     }
-    if (PyUnicode_Resize(&v, outpos) < 0)
+    if (unicode_resize(&v, outpos) < 0)
         goto onError;
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
@@ -6273,7 +6273,7 @@
             goto onError;
     }
 
-    if (PyUnicode_Resize(&v, outpos) < 0)
+    if (unicode_resize(&v, outpos) < 0)
         goto onError;
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
@@ -6727,7 +6727,7 @@
             data = PyUnicode_DATA(v);
         }
     }
-    if (PyUnicode_Resize(&v, outpos) < 0)
+    if (unicode_resize(&v, outpos) < 0)
         goto onError;
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);
@@ -6874,7 +6874,7 @@
     else {
         /* Extend unicode object */
         Py_ssize_t n = PyUnicode_GET_SIZE(*v);
-        if (PyUnicode_Resize(v, n + outsize) < 0)
+        if (unicode_resize(v, n + outsize) < 0)
             return -1;
         out = PyUnicode_AS_UNICODE(*v) + n;
     }
@@ -6958,7 +6958,7 @@
             PyErr_NoMemory();
             goto error;
         }
-        if (PyUnicode_Resize(v, n + size * Py_ARRAY_LENGTH(buffer)) < 0)
+        if (unicode_resize(v, n + size * Py_ARRAY_LENGTH(buffer)) < 0)
             goto error;
         startout = PyUnicode_AS_UNICODE(*v) + n;
     }
@@ -7017,7 +7017,7 @@
     /* Extend unicode object */
     outsize = out - startout;
     assert(outsize <= PyUnicode_WSTR_LENGTH(*v));
-    if (PyUnicode_Resize(v, outsize) < 0)
+    if (unicode_resize(v, outsize) < 0)
         goto error;
     ret = size;
 
@@ -7664,8 +7664,9 @@
                             (targetsize << 2);
                         extrachars += needed;
                         /* XXX overflow detection missing */
-                        if (PyUnicode_Resize(&v,
-                                             PyUnicode_GET_LENGTH(v) + needed) < 0) {
+                        if (unicode_resize(&v,
+                                           PyUnicode_GET_LENGTH(v) + needed) < 0)
+                        {
                             Py_DECREF(x);
                             goto onError;
                         }
@@ -7689,7 +7690,7 @@
             ++s;
         }
     }
-    if (PyUnicode_Resize(&v, outpos) < 0)
+    if (unicode_resize(&v, outpos) < 0)
         goto onError;
     Py_XDECREF(errorHandler);
     Py_XDECREF(exc);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 13:47:40 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 12 Dec 2011 13:47:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313577=3A_various_k?=
 =?utf8?q?inds_of_descriptors_now_have_a_=5F=5Fqualname=5F=5F_attribute=2E?=
Message-ID: <E1Ra5IC-0004Ls-Bl@dinsdale.python.org>

http://hg.python.org/cpython/rev/24238e89f938
changeset:   73948:24238e89f938
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 12 13:47:25 2011 +0100
summary:
  Issue #13577: various kinds of descriptors now have a __qualname__ attribute.
Patch by sbt.

files:
  Include/descrobject.h  |   1 +
  Lib/test/test_descr.py |  18 ++++++++++
  Lib/test/test_sys.py   |   8 ++--
  Objects/descrobject.c  |  51 ++++++++++++++++++++++++++++++
  4 files changed, 74 insertions(+), 4 deletions(-)


diff --git a/Include/descrobject.h b/Include/descrobject.h
--- a/Include/descrobject.h
+++ b/Include/descrobject.h
@@ -42,6 +42,7 @@
     PyObject_HEAD
     PyTypeObject *d_type;
     PyObject *d_name;
+    PyObject *d_qualname;
 } PyDescrObject;
 
 #define PyDescr_COMMON PyDescrObject d_common
diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py
--- a/Lib/test/test_descr.py
+++ b/Lib/test/test_descr.py
@@ -4442,6 +4442,24 @@
         self.assertIn("can't delete X.__doc__", str(cm.exception))
         self.assertEqual(X.__doc__, "banana")
 
+    def test_qualname(self):
+        descriptors = [str.lower, complex.real, float.real, int.__add__]
+        types = ['method', 'member', 'getset', 'wrapper']
+
+        # make sure we have an example of each type of descriptor
+        for d, n in zip(descriptors, types):
+            self.assertEqual(type(d).__name__, n + '_descriptor')
+
+        for d in descriptors:
+            qualname = d.__objclass__.__qualname__ + '.' + d.__name__
+            self.assertEqual(d.__qualname__, qualname)
+
+        self.assertEqual(str.lower.__qualname__, 'str.lower')
+        self.assertEqual(complex.real.__qualname__, 'complex.real')
+        self.assertEqual(float.real.__qualname__, 'float.real')
+        self.assertEqual(int.__add__.__qualname__, 'int.__add__')
+
+
 class DictProxyTests(unittest.TestCase):
     def setUp(self):
         class C(object):
diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py
--- a/Lib/test/test_sys.py
+++ b/Lib/test/test_sys.py
@@ -670,17 +670,17 @@
         # complex
         check(complex(0,1), size(h + '2d'))
         # method_descriptor (descriptor object)
-        check(str.lower, size(h + '2PP'))
+        check(str.lower, size(h + '3PP'))
         # classmethod_descriptor (descriptor object)
         # XXX
         # member_descriptor (descriptor object)
         import datetime
-        check(datetime.timedelta.days, size(h + '2PP'))
+        check(datetime.timedelta.days, size(h + '3PP'))
         # getset_descriptor (descriptor object)
         import collections
-        check(collections.defaultdict.default_factory, size(h + '2PP'))
+        check(collections.defaultdict.default_factory, size(h + '3PP'))
         # wrapper_descriptor (descriptor object)
-        check(int.__add__, size(h + '2P2P'))
+        check(int.__add__, size(h + '3P2P'))
         # method-wrapper (descriptor object)
         check({}.__iter__, size(h + '2P'))
         # dict
diff --git a/Objects/descrobject.c b/Objects/descrobject.c
--- a/Objects/descrobject.c
+++ b/Objects/descrobject.c
@@ -9,6 +9,7 @@
     _PyObject_GC_UNTRACK(descr);
     Py_XDECREF(descr->d_type);
     Py_XDECREF(descr->d_name);
+    Py_XDECREF(descr->d_qualname);
     PyObject_GC_Del(descr);
 }
 
@@ -321,6 +322,44 @@
     return PyUnicode_FromString(descr->d_method->ml_doc);
 }
 
+static PyObject *
+calculate_qualname(PyDescrObject *descr)
+{
+    PyObject *type_qualname, *res;
+    _Py_IDENTIFIER(__qualname__);
+
+    if (descr->d_name == NULL || !PyUnicode_Check(descr->d_name)) {
+        PyErr_SetString(PyExc_TypeError,
+                        "<descriptor>.__name__ is not a unicode object");
+        return NULL;
+    }
+
+    type_qualname = _PyObject_GetAttrId((PyObject *)descr->d_type,
+                                        &PyId___qualname__);
+    if (type_qualname == NULL)
+        return NULL;
+
+    if (!PyUnicode_Check(type_qualname)) {
+        PyErr_SetString(PyExc_TypeError, "<descriptor>.__objclass__."
+                        "__qualname__ is not a unicode object");
+        Py_XDECREF(type_qualname);
+        return NULL;
+    }
+
+    res = PyUnicode_FromFormat("%S.%S", type_qualname, descr->d_name);
+    Py_DECREF(type_qualname);
+    return res;
+}
+
+static PyObject *
+descr_get_qualname(PyDescrObject *descr)
+{
+    if (descr->d_qualname == NULL)
+        descr->d_qualname = calculate_qualname(descr);
+    Py_XINCREF(descr->d_qualname);
+    return descr->d_qualname;
+}
+
 static PyMemberDef descr_members[] = {
     {"__objclass__", T_OBJECT, offsetof(PyDescrObject, d_type), READONLY},
     {"__name__", T_OBJECT, offsetof(PyDescrObject, d_name), READONLY},
@@ -329,6 +368,7 @@
 
 static PyGetSetDef method_getset[] = {
     {"__doc__", (getter)method_get_doc},
+    {"__qualname__", (getter)descr_get_qualname},
     {0}
 };
 
@@ -344,6 +384,7 @@
 
 static PyGetSetDef member_getset[] = {
     {"__doc__", (getter)member_get_doc},
+    {"__qualname__", (getter)descr_get_qualname},
     {0}
 };
 
@@ -359,6 +400,7 @@
 
 static PyGetSetDef getset_getset[] = {
     {"__doc__", (getter)getset_get_doc},
+    {"__qualname__", (getter)descr_get_qualname},
     {0}
 };
 
@@ -374,6 +416,7 @@
 
 static PyGetSetDef wrapperdescr_getset[] = {
     {"__doc__", (getter)wrapperdescr_get_doc},
+    {"__qualname__", (getter)descr_get_qualname},
     {0}
 };
 
@@ -585,6 +628,7 @@
             Py_DECREF(descr);
             descr = NULL;
         }
+        descr->d_qualname = NULL;
     }
     return descr;
 }
@@ -987,9 +1031,16 @@
     }
 }
 
+static PyObject *
+wrapper_qualname(wrapperobject *wp)
+{
+    return descr_get_qualname((PyDescrObject *)wp->descr);
+}
+
 static PyGetSetDef wrapper_getsets[] = {
     {"__objclass__", (getter)wrapper_objclass},
     {"__name__", (getter)wrapper_name},
+    {"__qualname__", (getter)wrapper_qualname},
     {"__doc__", (getter)wrapper_doc},
     {0}
 };

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 18:55:51 2011
From: python-checkins at python.org (florent.xicluna)
Date: Mon, 12 Dec 2011 18:55:51 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313575=3A_there_is_?=
	=?utf8?q?only_one_class_type=2E?=
Message-ID: <E1RaA6R-0000gm-I5@dinsdale.python.org>

http://hg.python.org/cpython/rev/021e5bb297d1
changeset:   73949:021e5bb297d1
user:        Florent Xicluna <florent.xicluna at gmail.com>
date:        Mon Dec 12 18:54:29 2011 +0100
summary:
  Issue #13575: there is only one class type.

files:
  Doc/howto/descriptor.rst |  12 +------
  Lib/pickle.py            |   2 +-
  Lib/pickletools.py       |   2 +
  Modules/gc_weakref.txt   |   6 +-
  Objects/typeobject.c     |  31 +++++++------------
  Python/errors.c          |   2 +-
  Python/pythonrun.c       |   7 +---
  Tools/gdb/libpython.py   |  44 +--------------------------
  8 files changed, 25 insertions(+), 81 deletions(-)


diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst
--- a/Doc/howto/descriptor.rst
+++ b/Doc/howto/descriptor.rst
@@ -36,9 +36,7 @@
 looked-up value is an object defining one of the descriptor methods, then Python
 may override the default behavior and invoke the descriptor method instead.
 Where this occurs in the precedence chain depends on which descriptor methods
-were defined.  Note that descriptors are only invoked for new style objects or
-classes (a class is new style if it inherits from :class:`object` or
-:class:`type`).
+were defined.
 
 Descriptors are a powerful, general purpose protocol.  They are the mechanism
 behind properties, methods, static methods, class methods, and :func:`super()`.
@@ -89,8 +87,6 @@
 is invoked according to the precedence rules listed below.
 
 The details of invocation depend on whether ``obj`` is an object or a class.
-Either way, descriptors only work for new style objects and classes.  A class is
-new style if it is a subclass of :class:`object`.
 
 For objects, the machinery is in :meth:`object.__getattribute__` which
 transforms ``b.x`` into ``type(b).__dict__['x'].__get__(b, type(b))``.  The
@@ -115,7 +111,6 @@
 
 * descriptors are invoked by the :meth:`__getattribute__` method
 * overriding :meth:`__getattribute__` prevents automatic descriptor calls
-* :meth:`__getattribute__` is only available with new style classes and objects
 * :meth:`object.__getattribute__` and :meth:`type.__getattribute__` make
   different calls to :meth:`__get__`.
 * data descriptors always override instance dictionaries.
@@ -128,10 +123,7 @@
 ``m`` is returned unchanged.  If not in the dictionary, ``m`` reverts to a
 search using :meth:`object.__getattribute__`.
 
-Note, in Python 2.2, ``super(B, obj).m()`` would only invoke :meth:`__get__` if
-``m`` was a data descriptor.  In Python 2.3, non-data descriptors also get
-invoked unless an old-style class is involved.  The implementation details are
-in :c:func:`super_getattro()` in
+The implementation details are in :c:func:`super_getattro()` in
 `Objects/typeobject.c <http://svn.python.org/view/python/trunk/Objects/typeobject.c?view=markup>`_
 and a pure Python equivalent can be found in `Guido's Tutorial`_.
 
diff --git a/Lib/pickle.py b/Lib/pickle.py
--- a/Lib/pickle.py
+++ b/Lib/pickle.py
@@ -375,7 +375,7 @@
             # allowing protocol 0 and 1 to work normally.  For this to
             # work, the function returned by __reduce__ should be
             # called __newobj__, and its first argument should be a
-            # new-style class.  The implementation for __newobj__
+            # class.  The implementation for __newobj__
             # should be as follows, although pickle has no way to
             # verify this:
             #
diff --git a/Lib/pickletools.py b/Lib/pickletools.py
--- a/Lib/pickletools.py
+++ b/Lib/pickletools.py
@@ -1639,6 +1639,8 @@
       is pushed on the stack.
 
       NOTE:  checks for __safe_for_unpickling__ went away in Python 2.3.
+      NOTE:  the distinction between old-style and new-style classes does
+             not make sense in Python 3.
       """),
 
     I(name='OBJ',
diff --git a/Modules/gc_weakref.txt b/Modules/gc_weakref.txt
--- a/Modules/gc_weakref.txt
+++ b/Modules/gc_weakref.txt
@@ -15,8 +15,8 @@
 historically common bug), tp_clear empties an instance's __dict__, and
 "impossible" AttributeErrors result.  At worst, tp_clear leaves behind an
 insane object at the C level, and segfaults result (historically, most
-often by setting a new-style class's mro pointer to NULL, after which
-attribute lookups performed by the class can segfault).
+often by setting a class's mro pointer to NULL, after which attribute
+lookups performed by the class can segfault).
 
 OTOH, it's OK to run Python-level code that can't access unreachable
 objects, and sometimes that's necessary.  The chief example is the callback
@@ -119,7 +119,7 @@
 it took quite a while to dream up failing test cases.  Zope3 saw segfaults
 during shutdown, during the second call of gc in Py_Finalize, after most
 modules had been torn down.  That creates many trash cycles (esp. those
-involving new-style classes), making the problem much more likely.  Once you
+involving classes), making the problem much more likely.  Once you
 know what's required to provoke the problem, though, it's easy to create
 tests that segfault before shutdown.
 
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -100,15 +100,13 @@
 static void
 type_mro_modified(PyTypeObject *type, PyObject *bases) {
     /*
-       Check that all base classes or elements of the mro of type are
+       Check that all base classes or elements of the MRO of type are
        able to be cached.  This function is called after the base
        classes or mro of the type are altered.
 
        Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
-       inherits from an old-style class, either directly or if it
-       appears in the MRO of a new-style class.  No support either for
-       custom MROs that include types that are not officially super
-       types.
+       has a custom MRO that includes a type which is not officially
+       super type.
 
        Called from mro_internal, which will subsequently be called on
        each subclass when their mro is recursively updated.
@@ -124,11 +122,7 @@
         PyObject *b = PyTuple_GET_ITEM(bases, i);
         PyTypeObject *cls;
 
-        if (!PyType_Check(b) ) {
-            clear = 1;
-            break;
-        }
-
+        assert(PyType_Check(b));
         cls = (PyTypeObject *)b;
 
         if (!PyType_HasFeature(cls, Py_TPFLAGS_HAVE_VERSION_TAG) ||
@@ -488,7 +482,7 @@
         if (!PyType_Check(ob)) {
             PyErr_Format(
                 PyExc_TypeError,
-    "%s.__bases__ must be tuple of old- or new-style classes, not '%s'",
+    "%s.__bases__ must be tuple of classes, not '%s'",
                             type->tp_name, Py_TYPE(ob)->tp_name);
                     return -1;
         }
@@ -1619,7 +1613,7 @@
     type->tp_mro = tuple;
 
     type_mro_modified(type, type->tp_mro);
-    /* corner case: the old-style super class might have been hidden
+    /* corner case: the super class might have been hidden
        from the custom MRO */
     type_mro_modified(type, type->tp_bases);
 
@@ -1676,9 +1670,8 @@
             return NULL;
         }
     }
-    if (base == NULL)
-        PyErr_SetString(PyExc_TypeError,
-            "a new-style class can't have only classic bases");
+    assert (base != NULL);
+
     return base;
 }
 
@@ -3196,7 +3189,7 @@
     }
     if (!PyType_Check(value)) {
         PyErr_Format(PyExc_TypeError,
-          "__class__ must be set to new-style class, not '%s' object",
+          "__class__ must be set to a class, not '%s' object",
           Py_TYPE(value)->tp_name);
         return -1;
     }
@@ -3811,8 +3804,8 @@
            that the extension type's own factory function ensures).
            Heap types, of course, are under our control, so they do
            inherit tp_new; static extension types that specify some
-           other built-in type as the default are considered
-           new-style-aware so they also inherit object.__new__. */
+           other built-in type as the default also
+           inherit object.__new__. */
         if (base != &PyBaseObject_Type ||
             (type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
             if (type->tp_new == NULL)
@@ -6352,7 +6345,7 @@
 {
     /* Check that a super() call makes sense.  Return a type object.
 
-       obj can be a new-style class, or an instance of one:
+       obj can be a class, or an instance of one:
 
        - If it is a class, it must be a subclass of 'type'.      This case is
          used for class methods; the return value is obj.
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -665,7 +665,7 @@
         if (bases == NULL)
             goto failure;
     }
-    /* Create a real new-style class. */
+    /* Create a real class. */
     result = PyObject_CallFunction((PyObject *)&PyType_Type, "sOO",
                                    dot+1, bases, dict);
   failure:
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -474,7 +474,7 @@
     flush_std_files();
 
     /* Collect final garbage.  This disposes of cycles created by
-     * new-style class definitions, for example.
+     * class definitions, for example.
      * XXX This is disabled because it caused too many problems.  If
      * XXX a __del__ or weakref callback triggers here, Python code has
      * XXX a hard time running, because even the sys module has been
@@ -1348,11 +1348,6 @@
     _Py_IDENTIFIER(offset);
     _Py_IDENTIFIER(text);
 
-    /* old style errors */
-    if (PyTuple_Check(err))
-        return PyArg_ParseTuple(err, "O(ziiz)", message, filename,
-                                lineno, offset, text);
-
     /* new style errors.  `err' is an instance */
 
     if (! (v = _PyObject_GetAttrId(err, &PyId_msg)))
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -402,7 +402,7 @@
 
 
 def _write_instance_repr(out, visited, name, pyop_attrdict, address):
-    '''Shared code for use by old-style and new-style classes:
+    '''Shared code for use by all classes:
     write a representation to file-like object "out"'''
     out.write('<')
     out.write(name)
@@ -481,7 +481,7 @@
 
     def proxyval(self, visited):
         '''
-        Support for new-style classes.
+        Support for classes.
 
         Currently we just locate the dictionary using a transliteration to
         python of _PyObject_GetDictPtr, ignoring descriptors
@@ -498,7 +498,7 @@
             attr_dict = {}
         tp_name = self.safe_tp_name()
 
-        # New-style class:
+        # Class:
         return InstanceProxy(tp_name, attr_dict, long(self._gdbval))
 
     def write_repr(self, out, visited):
@@ -670,44 +670,6 @@
             pyop_value.write_repr(out, visited)
         out.write('}')
 
-class PyInstanceObjectPtr(PyObjectPtr):
-    _typename = 'PyInstanceObject'
-
-    def proxyval(self, visited):
-        # Guard against infinite loops:
-        if self.as_address() in visited:
-            return ProxyAlreadyVisited('<...>')
-        visited.add(self.as_address())
-
-        # Get name of class:
-        in_class = self.pyop_field('in_class')
-        cl_name = in_class.pyop_field('cl_name').proxyval(visited)
-
-        # Get dictionary of instance attributes:
-        in_dict = self.pyop_field('in_dict').proxyval(visited)
-
-        # Old-style class:
-        return InstanceProxy(cl_name, in_dict, long(self._gdbval))
-
-    def write_repr(self, out, visited):
-        # Guard against infinite loops:
-        if self.as_address() in visited:
-            out.write('<...>')
-            return
-        visited.add(self.as_address())
-
-        # Old-style class:
-
-        # Get name of class:
-        in_class = self.pyop_field('in_class')
-        cl_name = in_class.pyop_field('cl_name').proxyval(visited)
-
-        # Get dictionary of instance attributes:
-        pyop_in_dict = self.pyop_field('in_dict')
-
-        _write_instance_repr(out, visited,
-                             cl_name, pyop_in_dict, self.as_address())
-
 class PyListObjectPtr(PyObjectPtr):
     _typename = 'PyListObject'
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 12 19:18:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 12 Dec 2011 19:18:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_test=5Fgdb_failure?=
Message-ID: <E1RaASV-0002le-J9@dinsdale.python.org>

http://hg.python.org/cpython/rev/6f097ff9ac04
changeset:   73950:6f097ff9ac04
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 12 19:18:24 2011 +0100
summary:
  Fix test_gdb failure

files:
  Tools/gdb/libpython.py |  1 -
  1 files changed, 0 insertions(+), 1 deletions(-)


diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -328,7 +328,6 @@
 
         name_map = {'bool': PyBoolObjectPtr,
                     'classobj': PyClassObjectPtr,
-                    'instance': PyInstanceObjectPtr,
                     'NoneType': PyNoneStructPtr,
                     'frame': PyFrameObjectPtr,
                     'set' : PySetObjectPtr,

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Dec 13 05:34:19 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 13 Dec 2011 05:34:19 +0100
Subject: [Python-checkins] Daily reference leaks (6f097ff9ac04): sum=0
Message-ID: <E1RaK4J-0004NB-MY@ap.vmr.nerim.net>

results for 6f097ff9ac04 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogq7CYkf', '-x']

From python-checkins at python.org  Tue Dec 13 14:38:29 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:38:29 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNTQ5OiBpbXBy?=
 =?utf8?q?ove_tutorial_section_about_listcomps=2E?=
Message-ID: <E1RaSYv-0000um-EB@dinsdale.python.org>

http://hg.python.org/cpython/rev/132158b287d7
changeset:   73951:132158b287d7
branch:      2.7
parent:      73944:bf7329190ca6
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 14:50:21 2011 +0200
summary:
  #13549: improve tutorial section about listcomps.

files:
  Doc/tutorial/datastructures.rst |  167 +++++++++++++------
  1 files changed, 109 insertions(+), 58 deletions(-)


diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -235,89 +235,139 @@
 List Comprehensions
 -------------------
 
-List comprehensions provide a concise way to create lists without resorting to
-use of :func:`map`, :func:`filter` and/or :keyword:`lambda`. The resulting list
-definition tends often to be clearer than lists built using those constructs.
-Each list comprehension consists of an expression followed by a :keyword:`for`
-clause, then zero or more :keyword:`for` or :keyword:`if` clauses.  The result
-will be a list resulting from evaluating the expression in the context of the
-:keyword:`for` and :keyword:`if` clauses which follow it.  If the expression
-would evaluate to a tuple, it must be parenthesized. ::
+List comprehensions provide a concise way to create lists.
+Common applications are to make new lists where each element is the result of
+some operations applied to each member of another sequence or iterable, or to
+create a subsequence of those elements that satisfy a certain condition.
 
+For example, assume we want to create a list of squares, like::
+
+   >>> squares = []
+   >>> for x in range(10):
+   ...     squares.append(x**2)
+   ...
+   >>> squares
+   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
+
+We can obtain the same result with::
+
+   squares = [x**2 for x in range(10)]
+
+This is also equivalent to ``squares = map(lambda x: x**2, range(10))``,
+but it's more concise and readable.
+
+A list comprehension consists of brackets containing an expression followed
+by a :keyword:`for` clause, then zero or more :keyword:`for` or :keyword:`if`
+clauses.  The result will be a new list resulting from evaluating the expression
+in the context of the :keyword:`for` and :keyword:`if` clauses which follow it.
+For example, this listcomp combines the elements of two lists if they are not
+equal::
+
+   >>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
+
+and it's equivalent to:
+
+   >>> combs = []
+   >>> for x in [1,2,3]:
+   ...     for y in [3,1,4]:
+   ...         if x != y:
+   ...             combs.append((x, y))
+   ...
+   >>> combs
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
+
+Note how the order of the :keyword:`for` and :keyword:`if` statements is the
+same in both these snippets.
+
+If the expression is a tuple (e.g. the ``(x, y)`` in the previous example),
+it must be parenthesized. ::
+
+   >>> vec = [-4, -2, 0, 2, 4]
+   >>> # create a new list with the values doubled
+   >>> [x*2 for x in vec]
+   [-8, -4, 0, 4, 8]
+   >>> # filter the list to exclude negative numbers
+   >>> [x for x in vec if x >= 0]
+   [0, 2, 4]
+   >>> # apply a function to all the elements
+   >>> [abs(x) for x in vec]
+   [4, 2, 0, 2, 4]
+   >>> # call a method on each element
    >>> freshfruit = ['  banana', '  loganberry ', 'passion fruit  ']
    >>> [weapon.strip() for weapon in freshfruit]
    ['banana', 'loganberry', 'passion fruit']
-   >>> vec = [2, 4, 6]
-   >>> [3*x for x in vec]
-   [6, 12, 18]
-   >>> [3*x for x in vec if x > 3]
-   [12, 18]
-   >>> [3*x for x in vec if x < 2]
-   []
-   >>> [[x,x**2] for x in vec]
-   [[2, 4], [4, 16], [6, 36]]
-   >>> [x, x**2 for x in vec]  # error - parens required for tuples
-     File "<stdin>", line 1, in ?
-       [x, x**2 for x in vec]
+   >>> # create a list of 2-tuples like (number, square)
+   >>> [(x, x**2) for x in range(6)]
+   [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
+   >>> # the tuple must be parenthesized, otherwise an error is raised
+   >>> [x, x**2 for x in range(6)]
+     File "<stdin>", line 1
+       [x, x**2 for x in range(6)]
                   ^
    SyntaxError: invalid syntax
-   >>> [(x, x**2) for x in vec]
-   [(2, 4), (4, 16), (6, 36)]
-   >>> vec1 = [2, 4, 6]
-   >>> vec2 = [4, 3, -9]
-   >>> [x*y for x in vec1 for y in vec2]
-   [8, 6, -18, 16, 12, -36, 24, 18, -54]
-   >>> [x+y for x in vec1 for y in vec2]
-   [6, 5, -7, 8, 7, -5, 10, 9, -3]
-   >>> [vec1[i]*vec2[i] for i in range(len(vec1))]
-   [8, 12, -54]
+   >>> # flatten a list using a listcomp with two 'for'
+   >>> vec = [[1,2,3], [4,5,6], [7,8,9]]
+   >>> [num for elem in vec for num in elem]
+   [1, 2, 3, 4, 5, 6, 7, 8, 9]
 
-List comprehensions are much more flexible than :func:`map` and can be applied
-to complex expressions and nested functions::
+List comprehensions can contain complex expressions and nested functions::
 
-   >>> [str(round(355/113.0, i)) for i in range(1,6)]
+   >>> from math import pi
+   >>> [str(round(pi, i)) for i in range(1, 6)]
    ['3.1', '3.14', '3.142', '3.1416', '3.14159']
 
 
 Nested List Comprehensions
---------------------------
+''''''''''''''''''''''''''
 
-If you've got the stomach for it, list comprehensions can be nested. They are a
-powerful tool but -- like all powerful tools -- they need to be used carefully,
-if at all.
+The initial expression in a list comprehension can be any arbitrary expression,
+including another list comprehension.
 
-Consider the following example of a 3x3 matrix held as a list containing three
-lists, one list per row::
+Consider the following example of a 3x4 matrix implemented as a list of
+3 lists of length 4::
 
-    >>> mat = [
-    ...        [1, 2, 3],
-    ...        [4, 5, 6],
-    ...        [7, 8, 9],
-    ...       ]
+   >>> matrix = [
+   ...     [1, 2, 3, 4],
+   ...     [5, 6, 7, 8],
+   ...     [9, 10, 11, 12],
+   ... ]
 
-Now, if you wanted to swap rows and columns, you could use a list
-comprehension::
+The following list comprehension will transpose rows and columns::
 
-    >>> print [[row[i] for row in mat] for i in [0, 1, 2]]
-    [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
+   >>> [[row[i] for row in matrix] for i in range(4)]
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-Special care has to be taken for the *nested* list comprehension:
+As we saw in the previous section, the nested listcomp is evaluated in
+the context of the :keyword:`for` that follows it, so this example is
+equivalent to::
 
-    To avoid apprehension when nesting list comprehensions, read from right to
-    left.
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     transposed.append([row[i] for row in matrix])
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-A more verbose version of this snippet shows the flow explicitly::
+which, in turn, is the same as::
 
-    for i in [0, 1, 2]:
-        for row in mat:
-            print row[i],
-        print
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     # the following 3 lines implement the nested listcomp
+   ...     transposed_row = []
+   ...     for row in matrix:
+   ...         transposed_row.append(row[i])
+   ...     transposed.append(transposed_row)
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-In real world, you should prefer built-in functions to complex flow statements.
+
+In the real world, you should prefer built-in functions to complex flow statements.
 The :func:`zip` function would do a great job for this use case::
 
-    >>> zip(*mat)
-    [(1, 4, 7), (2, 5, 8), (3, 6, 9)]
+   >>> zip(*matrix)
+   [(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
 
 See :ref:`tut-unpacking-arguments` for details on the asterisk in this line.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 14:38:30 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:38:30 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNTQ5OiBpbXBy?=
 =?utf8?q?ove_tutorial_section_about_listcomps=2E?=
Message-ID: <E1RaSYw-0000ut-6b@dinsdale.python.org>

http://hg.python.org/cpython/rev/ad5c70296c7b
changeset:   73952:ad5c70296c7b
branch:      3.2
parent:      73925:6040c248a199
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 15:36:19 2011 +0200
summary:
  #13549: improve tutorial section about listcomps.

files:
  Doc/tutorial/datastructures.rst |  166 +++++++++++--------
  1 files changed, 98 insertions(+), 68 deletions(-)


diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -163,107 +163,137 @@
 List Comprehensions
 -------------------
 
-List comprehensions provide a concise way to create lists from sequences.
-Common applications are to make lists where each element is the result of
-some operations applied to each member of the sequence, or to create a
-subsequence of those elements that satisfy a certain condition.
+List comprehensions provide a concise way to create lists.
+Common applications are to make new lists where each element is the result of
+some operations applied to each member of another sequence or iterable, or to
+create a subsequence of those elements that satisfy a certain condition.
+
+For example, assume we want to create a list of squares, like::
+
+   >>> squares = []
+   >>> for x in range(10):
+   ...     squares.append(x**2)
+   ...
+   >>> squares
+   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
+
+We can obtain the same result with::
+
+   squares = [x**2 for x in range(10)]
+
+This is also equivalent to ``squares = map(lambda x: x**2, range(10))``,
+but it's more concise and readable.
 
 A list comprehension consists of brackets containing an expression followed
 by a :keyword:`for` clause, then zero or more :keyword:`for` or :keyword:`if`
-clauses.  The result will be a list resulting from evaluating the expression in
-the context of the :keyword:`for` and :keyword:`if` clauses which follow it.  If
-the expression would evaluate to a tuple, it must be parenthesized.
+clauses.  The result will be a new list resulting from evaluating the expression
+in the context of the :keyword:`for` and :keyword:`if` clauses which follow it.
+For example, this listcomp combines the elements of two lists if they are not
+equal::
 
-Here we take a list of numbers and return a list of three times each number::
+   >>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
 
-   >>> vec = [2, 4, 6]
-   >>> [3*x for x in vec]
-   [6, 12, 18]
+and it's equivalent to::
 
-Now we get a little fancier::
+   >>> combs = []
+   >>> for x in [1,2,3]:
+   ...     for y in [3,1,4]:
+   ...         if x != y:
+   ...             combs.append((x, y))
+   ...
+   >>> combs
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
 
-   >>> [[x, x**2] for x in vec]
-   [[2, 4], [4, 16], [6, 36]]
+Note how the order of the :keyword:`for` and :keyword:`if` statements is the
+same in both these snippets.
 
-Here we apply a method call to each item in a sequence::
+If the expression is a tuple (e.g. the ``(x, y)`` in the previous example),
+it must be parenthesized. ::
 
+   >>> vec = [-4, -2, 0, 2, 4]
+   >>> # create a new list with the values doubled
+   >>> [x*2 for x in vec]
+   [-8, -4, 0, 4, 8]
+   >>> # filter the list to exclude negative numbers
+   >>> [x for x in vec if x >= 0]
+   [0, 2, 4]
+   >>> # apply a function to all the elements
+   >>> [abs(x) for x in vec]
+   [4, 2, 0, 2, 4]
+   >>> # call a method on each element
    >>> freshfruit = ['  banana', '  loganberry ', 'passion fruit  ']
    >>> [weapon.strip() for weapon in freshfruit]
    ['banana', 'loganberry', 'passion fruit']
-
-Using the :keyword:`if` clause we can filter the stream::
-
-   >>> [3*x for x in vec if x > 3]
-   [12, 18]
-   >>> [3*x for x in vec if x < 2]
-   []
-
-Tuples can often be created without their parentheses, but not here::
-
-   >>> [x, x**2 for x in vec]  # error - parens required for tuples
+   >>> # create a list of 2-tuples like (number, square)
+   >>> [(x, x**2) for x in range(6)]
+   [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
+   >>> # the tuple must be parenthesized, otherwise an error is raised
+   >>> [x, x**2 for x in range(6)]
      File "<stdin>", line 1, in ?
-       [x, x**2 for x in vec]
+       [x, x**2 for x in range(6)]
                   ^
    SyntaxError: invalid syntax
-   >>> [(x, x**2) for x in vec]
-   [(2, 4), (4, 16), (6, 36)]
+   >>> # flatten a list using a listcomp with two 'for'
+   >>> vec = [[1,2,3], [4,5,6], [7,8,9]]
+   >>> [num for elem in vec for num in elem]
+   [1, 2, 3, 4, 5, 6, 7, 8, 9]
 
-Here are some nested for loops and other fancy behavior::
+List comprehensions can contain complex expressions and nested functions::
 
-   >>> vec1 = [2, 4, 6]
-   >>> vec2 = [4, 3, -9]
-   >>> [x*y for x in vec1 for y in vec2]
-   [8, 6, -18, 16, 12, -36, 24, 18, -54]
-   >>> [x+y for x in vec1 for y in vec2]
-   [6, 5, -7, 8, 7, -5, 10, 9, -3]
-   >>> [vec1[i]*vec2[i] for i in range(len(vec1))]
-   [8, 12, -54]
-
-List comprehensions can be applied to complex expressions and nested functions::
-
-   >>> [str(round(355/113, i)) for i in range(1, 6)]
+   >>> from math import pi
+   >>> [str(round(pi, i)) for i in range(1, 6)]
    ['3.1', '3.14', '3.142', '3.1416', '3.14159']
 
-
 Nested List Comprehensions
 --------------------------
 
-If you've got the stomach for it, list comprehensions can be nested. They are a
-powerful tool but -- like all powerful tools -- they need to be used carefully,
-if at all.
+The initial expression in a list comprehension can be any arbitrary expression,
+including another list comprehension.
 
-Consider the following example of a 3x3 matrix held as a list containing three
-lists, one list per row::
+Consider the following example of a 3x4 matrix implemented as a list of
+3 lists of length 4::
 
-    >>> mat = [
-    ...        [1, 2, 3],
-    ...        [4, 5, 6],
-    ...        [7, 8, 9],
-    ...       ]
+   >>> matrix = [
+   ...     [1, 2, 3, 4],
+   ...     [5, 6, 7, 8],
+   ...     [9, 10, 11, 12],
+   ... ]
 
-Now, if you wanted to swap rows and columns, you could use a list
-comprehension::
+The following list comprehension will transpose rows and columns::
 
-    >>> print([[row[i] for row in mat] for i in [0, 1, 2]])
-    [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
+   >>> [[row[i] for row in matrix] for i in range(4)]
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-Special care has to be taken for the *nested* list comprehension:
+As we saw in the previous section, the nested listcomp is evaluated in
+the context of the :keyword:`for` that follows it, so this example is
+equivalent to::
 
-    To avoid apprehension when nesting list comprehensions, read from right to
-    left.
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     transposed.append([row[i] for row in matrix])
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-A more verbose version of this snippet shows the flow explicitly::
+which, in turn, is the same as::
 
-    for i in [0, 1, 2]:
-        for row in mat:
-            print(row[i], end="")
-        print()
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     # the following 3 lines implement the nested listcomp
+   ...     transposed_row = []
+   ...     for row in matrix:
+   ...         transposed_row.append(row[i])
+   ...     transposed.append(transposed_row)
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-In real world, you should prefer built-in functions to complex flow statements.
+In the real world, you should prefer built-in functions to complex flow statements.
 The :func:`zip` function would do a great job for this use case::
 
-    >>> list(zip(*mat))
-    [(1, 4, 7), (2, 5, 8), (3, 6, 9)]
+   >>> zip(*matrix)
+   [(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
 
 See :ref:`tut-unpacking-arguments` for details on the asterisk in this line.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 14:38:33 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:38:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313549=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RaSYz-0000vo-GJ@dinsdale.python.org>

http://hg.python.org/cpython/rev/37094a1454ab
changeset:   73953:37094a1454ab
parent:      73950:6f097ff9ac04
parent:      73952:ad5c70296c7b
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 15:38:13 2011 +0200
summary:
  #13549: merge with 3.2.

files:
  Doc/tutorial/datastructures.rst |  166 +++++++++++--------
  1 files changed, 98 insertions(+), 68 deletions(-)


diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -168,107 +168,137 @@
 List Comprehensions
 -------------------
 
-List comprehensions provide a concise way to create lists from sequences.
-Common applications are to make lists where each element is the result of
-some operations applied to each member of the sequence, or to create a
-subsequence of those elements that satisfy a certain condition.
+List comprehensions provide a concise way to create lists.
+Common applications are to make new lists where each element is the result of
+some operations applied to each member of another sequence or iterable, or to
+create a subsequence of those elements that satisfy a certain condition.
+
+For example, assume we want to create a list of squares, like::
+
+   >>> squares = []
+   >>> for x in range(10):
+   ...     squares.append(x**2)
+   ...
+   >>> squares
+   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
+
+We can obtain the same result with::
+
+   squares = [x**2 for x in range(10)]
+
+This is also equivalent to ``squares = map(lambda x: x**2, range(10))``,
+but it's more concise and readable.
 
 A list comprehension consists of brackets containing an expression followed
 by a :keyword:`for` clause, then zero or more :keyword:`for` or :keyword:`if`
-clauses.  The result will be a list resulting from evaluating the expression in
-the context of the :keyword:`for` and :keyword:`if` clauses which follow it.  If
-the expression would evaluate to a tuple, it must be parenthesized.
+clauses.  The result will be a new list resulting from evaluating the expression
+in the context of the :keyword:`for` and :keyword:`if` clauses which follow it.
+For example, this listcomp combines the elements of two lists if they are not
+equal::
 
-Here we take a list of numbers and return a list of three times each number::
+   >>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
 
-   >>> vec = [2, 4, 6]
-   >>> [3*x for x in vec]
-   [6, 12, 18]
+and it's equivalent to::
 
-Now we get a little fancier::
+   >>> combs = []
+   >>> for x in [1,2,3]:
+   ...     for y in [3,1,4]:
+   ...         if x != y:
+   ...             combs.append((x, y))
+   ...
+   >>> combs
+   [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
 
-   >>> [[x, x**2] for x in vec]
-   [[2, 4], [4, 16], [6, 36]]
+Note how the order of the :keyword:`for` and :keyword:`if` statements is the
+same in both these snippets.
 
-Here we apply a method call to each item in a sequence::
+If the expression is a tuple (e.g. the ``(x, y)`` in the previous example),
+it must be parenthesized. ::
 
+   >>> vec = [-4, -2, 0, 2, 4]
+   >>> # create a new list with the values doubled
+   >>> [x*2 for x in vec]
+   [-8, -4, 0, 4, 8]
+   >>> # filter the list to exclude negative numbers
+   >>> [x for x in vec if x >= 0]
+   [0, 2, 4]
+   >>> # apply a function to all the elements
+   >>> [abs(x) for x in vec]
+   [4, 2, 0, 2, 4]
+   >>> # call a method on each element
    >>> freshfruit = ['  banana', '  loganberry ', 'passion fruit  ']
    >>> [weapon.strip() for weapon in freshfruit]
    ['banana', 'loganberry', 'passion fruit']
-
-Using the :keyword:`if` clause we can filter the stream::
-
-   >>> [3*x for x in vec if x > 3]
-   [12, 18]
-   >>> [3*x for x in vec if x < 2]
-   []
-
-Tuples can often be created without their parentheses, but not here::
-
-   >>> [x, x**2 for x in vec]  # error - parens required for tuples
+   >>> # create a list of 2-tuples like (number, square)
+   >>> [(x, x**2) for x in range(6)]
+   [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
+   >>> # the tuple must be parenthesized, otherwise an error is raised
+   >>> [x, x**2 for x in range(6)]
      File "<stdin>", line 1, in ?
-       [x, x**2 for x in vec]
+       [x, x**2 for x in range(6)]
                   ^
    SyntaxError: invalid syntax
-   >>> [(x, x**2) for x in vec]
-   [(2, 4), (4, 16), (6, 36)]
+   >>> # flatten a list using a listcomp with two 'for'
+   >>> vec = [[1,2,3], [4,5,6], [7,8,9]]
+   >>> [num for elem in vec for num in elem]
+   [1, 2, 3, 4, 5, 6, 7, 8, 9]
 
-Here are some nested for loops and other fancy behavior::
+List comprehensions can contain complex expressions and nested functions::
 
-   >>> vec1 = [2, 4, 6]
-   >>> vec2 = [4, 3, -9]
-   >>> [x*y for x in vec1 for y in vec2]
-   [8, 6, -18, 16, 12, -36, 24, 18, -54]
-   >>> [x+y for x in vec1 for y in vec2]
-   [6, 5, -7, 8, 7, -5, 10, 9, -3]
-   >>> [vec1[i]*vec2[i] for i in range(len(vec1))]
-   [8, 12, -54]
-
-List comprehensions can be applied to complex expressions and nested functions::
-
-   >>> [str(round(355/113, i)) for i in range(1, 6)]
+   >>> from math import pi
+   >>> [str(round(pi, i)) for i in range(1, 6)]
    ['3.1', '3.14', '3.142', '3.1416', '3.14159']
 
-
 Nested List Comprehensions
 --------------------------
 
-If you've got the stomach for it, list comprehensions can be nested. They are a
-powerful tool but -- like all powerful tools -- they need to be used carefully,
-if at all.
+The initial expression in a list comprehension can be any arbitrary expression,
+including another list comprehension.
 
-Consider the following example of a 3x3 matrix held as a list containing three
-lists, one list per row::
+Consider the following example of a 3x4 matrix implemented as a list of
+3 lists of length 4::
 
-    >>> mat = [
-    ...        [1, 2, 3],
-    ...        [4, 5, 6],
-    ...        [7, 8, 9],
-    ...       ]
+   >>> matrix = [
+   ...     [1, 2, 3, 4],
+   ...     [5, 6, 7, 8],
+   ...     [9, 10, 11, 12],
+   ... ]
 
-Now, if you wanted to swap rows and columns, you could use a list
-comprehension::
+The following list comprehension will transpose rows and columns::
 
-    >>> print([[row[i] for row in mat] for i in [0, 1, 2]])
-    [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
+   >>> [[row[i] for row in matrix] for i in range(4)]
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-Special care has to be taken for the *nested* list comprehension:
+As we saw in the previous section, the nested listcomp is evaluated in
+the context of the :keyword:`for` that follows it, so this example is
+equivalent to::
 
-    To avoid apprehension when nesting list comprehensions, read from right to
-    left.
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     transposed.append([row[i] for row in matrix])
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-A more verbose version of this snippet shows the flow explicitly::
+which, in turn, is the same as::
 
-    for i in [0, 1, 2]:
-        for row in mat:
-            print(row[i], end="")
-        print()
+   >>> transposed = []
+   >>> for i in range(4):
+   ...     # the following 3 lines implement the nested listcomp
+   ...     transposed_row = []
+   ...     for row in matrix:
+   ...         transposed_row.append(row[i])
+   ...     transposed.append(transposed_row)
+   ...
+   >>> transposed
+   [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
 
-In real world, you should prefer built-in functions to complex flow statements.
+In the real world, you should prefer built-in functions to complex flow statements.
 The :func:`zip` function would do a great job for this use case::
 
-    >>> list(zip(*mat))
-    [(1, 4, 7), (2, 5, 8), (3, 6, 9)]
+   >>> zip(*matrix)
+   [(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
 
 See :ref:`tut-unpacking-arguments` for details on the asterisk in this line.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 14:54:05 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:54:05 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzY1NzA6IGNsYXJp?=
 =?utf8?q?fy_tutorial_section_about_keyword_arguments=2E?=
Message-ID: <E1RaSo1-0002Wy-VR@dinsdale.python.org>

http://hg.python.org/cpython/rev/44ca4264dc88
changeset:   73955:44ca4264dc88
branch:      3.2
parent:      73952:ad5c70296c7b
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 15:49:22 2011 +0200
summary:
  #6570: clarify tutorial section about keyword arguments.

files:
  Doc/tutorial/controlflow.rst |  39 +++++++++++++----------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst
--- a/Doc/tutorial/controlflow.rst
+++ b/Doc/tutorial/controlflow.rst
@@ -412,8 +412,8 @@
 Keyword Arguments
 -----------------
 
-Functions can also be called using keyword arguments of the form ``keyword =
-value``.  For instance, the following function::
+Functions can also be called using :term:`keyword arguments <keyword argument>`
+of the form ``kwarg=value``.  For instance, the following function::
 
    def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
        print("-- This parrot wouldn't", action, end=' ')
@@ -421,26 +421,31 @@
        print("-- Lovely plumage, the", type)
        print("-- It's", state, "!")
 
-could be called in any of the following ways::
+accepts one required argument (``voltage``) and three optional arguments
+(``state``, ``action``, and ``type``).  This function can be called in any
+of the following ways::
 
-   parrot(1000)
-   parrot(action = 'VOOOOOM', voltage = 1000000)
-   parrot('a thousand', state = 'pushing up the daisies')
-   parrot('a million', 'bereft of life', 'jump')
+   parrot(1000)                                          # 1 positional argument
+   parrot(voltage=1000)                                  # 1 keyword argument
+   parrot(voltage=1000000, action='VOOOOOM')             # 2 keyword arguments
+   parrot(action='VOOOOOM', voltage=1000000)             # 2 keyword arguments
+   parrot('a million', 'bereft of life', 'jump')         # 3 positional arguments
+   parrot('a thousand', state='pushing up the daisies')  # 1 positional, 1 keyword
 
-but the following calls would all be invalid::
+but all the following calls would be invalid::
 
    parrot()                     # required argument missing
-   parrot(voltage=5.0, 'dead')  # non-keyword argument following keyword
-   parrot(110, voltage=220)     # duplicate value for argument
-   parrot(actor='John Cleese')  # unknown keyword
+   parrot(voltage=5.0, 'dead')  # non-keyword argument after a keyword argument
+   parrot(110, voltage=220)     # duplicate value for the same argument
+   parrot(actor='John Cleese')  # unknown keyword argument
 
-In general, an argument list must have any positional arguments followed by any
-keyword arguments, where the keywords must be chosen from the formal parameter
-names.  It's not important whether a formal parameter has a default value or
-not.  No argument may receive a value more than once --- formal parameter names
-corresponding to positional arguments cannot be used as keywords in the same
-calls. Here's an example that fails due to this restriction::
+In a function call, keyword arguments must follow positional arguments.
+All the keyword arguments passed must match one of the arguments
+accepted by the function (e.g. ``actor`` is not a valid argument for the
+``parrot`` function), and their order is not important.  This also includes
+non-optional arguments (e.g. ``parrot(voltage=1000)`` is valid too).
+No argument may receive a value more than once.
+Here's an example that fails due to this restriction::
 
    >>> def function(a):
    ...     pass

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 14:54:06 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:54:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?b?OiAjNjU3MDogbWVyZ2Ugd2l0aCAzLjIu?=
Message-ID: <E1RaSo2-0002X2-OP@dinsdale.python.org>

http://hg.python.org/cpython/rev/207408428242
changeset:   73956:207408428242
parent:      73953:37094a1454ab
parent:      73955:44ca4264dc88
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 15:53:47 2011 +0200
summary:
  #6570: merge with 3.2.

files:
  Doc/tutorial/controlflow.rst |  39 +++++++++++++----------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst
--- a/Doc/tutorial/controlflow.rst
+++ b/Doc/tutorial/controlflow.rst
@@ -412,8 +412,8 @@
 Keyword Arguments
 -----------------
 
-Functions can also be called using keyword arguments of the form ``keyword =
-value``.  For instance, the following function::
+Functions can also be called using :term:`keyword arguments <keyword argument>`
+of the form ``kwarg=value``.  For instance, the following function::
 
    def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
        print("-- This parrot wouldn't", action, end=' ')
@@ -421,26 +421,31 @@
        print("-- Lovely plumage, the", type)
        print("-- It's", state, "!")
 
-could be called in any of the following ways::
+accepts one required argument (``voltage``) and three optional arguments
+(``state``, ``action``, and ``type``).  This function can be called in any
+of the following ways::
 
-   parrot(1000)
-   parrot(action = 'VOOOOOM', voltage = 1000000)
-   parrot('a thousand', state = 'pushing up the daisies')
-   parrot('a million', 'bereft of life', 'jump')
+   parrot(1000)                                          # 1 positional argument
+   parrot(voltage=1000)                                  # 1 keyword argument
+   parrot(voltage=1000000, action='VOOOOOM')             # 2 keyword arguments
+   parrot(action='VOOOOOM', voltage=1000000)             # 2 keyword arguments
+   parrot('a million', 'bereft of life', 'jump')         # 3 positional arguments
+   parrot('a thousand', state='pushing up the daisies')  # 1 positional, 1 keyword
 
-but the following calls would all be invalid::
+but all the following calls would be invalid::
 
    parrot()                     # required argument missing
-   parrot(voltage=5.0, 'dead')  # non-keyword argument following keyword
-   parrot(110, voltage=220)     # duplicate value for argument
-   parrot(actor='John Cleese')  # unknown keyword
+   parrot(voltage=5.0, 'dead')  # non-keyword argument after a keyword argument
+   parrot(110, voltage=220)     # duplicate value for the same argument
+   parrot(actor='John Cleese')  # unknown keyword argument
 
-In general, an argument list must have any positional arguments followed by any
-keyword arguments, where the keywords must be chosen from the formal parameter
-names.  It's not important whether a formal parameter has a default value or
-not.  No argument may receive a value more than once --- formal parameter names
-corresponding to positional arguments cannot be used as keywords in the same
-calls. Here's an example that fails due to this restriction::
+In a function call, keyword arguments must follow positional arguments.
+All the keyword arguments passed must match one of the arguments
+accepted by the function (e.g. ``actor`` is not a valid argument for the
+``parrot`` function), and their order is not important.  This also includes
+non-optional arguments (e.g. ``parrot(voltage=1000)`` is valid too).
+No argument may receive a value more than once.
+Here's an example that fails due to this restriction::
 
    >>> def function(a):
    ...     pass

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 14:54:05 2011
From: python-checkins at python.org (ezio.melotti)
Date: Tue, 13 Dec 2011 14:54:05 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzY1NzA6IGNsYXJp?=
 =?utf8?q?fy_tutorial_section_about_keyword_arguments=2E?=
Message-ID: <E1RaSo1-0002Wv-7X@dinsdale.python.org>

http://hg.python.org/cpython/rev/d60856651139
changeset:   73954:d60856651139
branch:      2.7
parent:      73951:132158b287d7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Tue Dec 13 15:49:22 2011 +0200
summary:
  #6570: clarify tutorial section about keyword arguments.

files:
  Doc/tutorial/controlflow.rst |  39 +++++++++++++----------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst
--- a/Doc/tutorial/controlflow.rst
+++ b/Doc/tutorial/controlflow.rst
@@ -383,8 +383,8 @@
 Keyword Arguments
 -----------------
 
-Functions can also be called using keyword arguments of the form ``keyword =
-value``.  For instance, the following function::
+Functions can also be called using :term:`keyword arguments <keyword argument>`
+of the form ``kwarg=value``.  For instance, the following function::
 
    def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
        print "-- This parrot wouldn't", action,
@@ -392,26 +392,31 @@
        print "-- Lovely plumage, the", type
        print "-- It's", state, "!"
 
-could be called in any of the following ways::
+accepts one required argument (``voltage``) and three optional arguments
+(``state``, ``action``, and ``type``).  This function can be called in any
+of the following ways::
 
-   parrot(1000)
-   parrot(action = 'VOOOOOM', voltage = 1000000)
-   parrot('a thousand', state = 'pushing up the daisies')
-   parrot('a million', 'bereft of life', 'jump')
+   parrot(1000)                                          # 1 positional argument
+   parrot(voltage=1000)                                  # 1 keyword argument
+   parrot(voltage=1000000, action='VOOOOOM')             # 2 keyword arguments
+   parrot(action='VOOOOOM', voltage=1000000)             # 2 keyword arguments
+   parrot('a million', 'bereft of life', 'jump')         # 3 positional arguments
+   parrot('a thousand', state='pushing up the daisies')  # 1 positional, 1 keyword
 
-but the following calls would all be invalid::
+but all the following calls would be invalid::
 
    parrot()                     # required argument missing
-   parrot(voltage=5.0, 'dead')  # non-keyword argument following keyword
-   parrot(110, voltage=220)     # duplicate value for argument
-   parrot(actor='John Cleese')  # unknown keyword
+   parrot(voltage=5.0, 'dead')  # non-keyword argument after a keyword argument
+   parrot(110, voltage=220)     # duplicate value for the same argument
+   parrot(actor='John Cleese')  # unknown keyword argument
 
-In general, an argument list must have any positional arguments followed by any
-keyword arguments, where the keywords must be chosen from the formal parameter
-names.  It's not important whether a formal parameter has a default value or
-not.  No argument may receive a value more than once --- formal parameter names
-corresponding to positional arguments cannot be used as keywords in the same
-calls. Here's an example that fails due to this restriction::
+In a function call, keyword arguments must follow positional arguments.
+All the keyword arguments passed must match one of the arguments
+accepted by the function (e.g. ``actor`` is not a valid argument for the
+``parrot`` function), and their order is not important.  This also includes
+non-optional arguments (e.g. ``parrot(voltage=1000)`` is valid too).
+No argument may receive a value more than once.
+Here's an example that fails due to this restriction::
 
    >>> def function(a):
    ...     pass

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 19:23:20 2011
From: python-checkins at python.org (alexandre.vassalotti)
Date: Tue, 13 Dec 2011 19:23:20 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTA1?=
 =?utf8?q?=3A_Make_pickling_of_bytes_object_compatible_with_Python_2=2E?=
Message-ID: <E1RaX0a-0006rV-4v@dinsdale.python.org>

http://hg.python.org/cpython/rev/14695b4825dc
changeset:   73957:14695b4825dc
branch:      3.2
parent:      73955:44ca4264dc88
user:        Alexandre Vassalotti <alexandre at peadrop.com>
date:        Tue Dec 13 13:08:09 2011 -0500
summary:
  Issue #13505: Make pickling of bytes object compatible with Python 2.

Initial patch by sbt.

files:
  Lib/pickle.py            |   6 ++-
  Lib/pickletools.py       |  70 ++++++++++++---------------
  Lib/test/pickletester.py |  12 +++-
  Misc/NEWS                |   3 +
  Modules/_pickle.c        |  60 ++++++++++++++++++-----
  5 files changed, 94 insertions(+), 57 deletions(-)


diff --git a/Lib/pickle.py b/Lib/pickle.py
--- a/Lib/pickle.py
+++ b/Lib/pickle.py
@@ -487,7 +487,11 @@
 
     def save_bytes(self, obj, pack=struct.pack):
         if self.proto < 3:
-            self.save_reduce(bytes, (list(obj),), obj=obj)
+            if len(obj) == 0:
+                self.save_reduce(bytes, (), obj=obj)
+            else:
+                self.save_reduce(codecs.encode,
+                                 (str(obj, 'latin1'), 'latin1'), obj=obj)
             return
         n = len(obj)
         if n < 256:
diff --git a/Lib/pickletools.py b/Lib/pickletools.py
--- a/Lib/pickletools.py
+++ b/Lib/pickletools.py
@@ -2083,27 +2083,22 @@
    29: (    MARK
    30: d        DICT       (MARK at 29)
    31: p    PUT        2
-   34: c    GLOBAL     '__builtin__ bytes'
-   53: p    PUT        3
-   56: (    MARK
-   57: (        MARK
-   58: l            LIST       (MARK at 57)
+   34: c    GLOBAL     '_codecs encode'
+   50: p    PUT        3
+   53: (    MARK
+   54: V        UNICODE    'abc'
    59: p        PUT        4
-   62: L        LONG       97
-   67: a        APPEND
-   68: L        LONG       98
-   73: a        APPEND
-   74: L        LONG       99
-   79: a        APPEND
-   80: t        TUPLE      (MARK at 56)
-   81: p    PUT        5
-   84: R    REDUCE
-   85: p    PUT        6
-   88: V    UNICODE    'def'
-   93: p    PUT        7
-   96: s    SETITEM
-   97: a    APPEND
-   98: .    STOP
+   62: V        UNICODE    'latin1'
+   70: p        PUT        5
+   73: t        TUPLE      (MARK at 53)
+   74: p    PUT        6
+   77: R    REDUCE
+   78: p    PUT        7
+   81: V    UNICODE    'def'
+   86: p    PUT        8
+   89: s    SETITEM
+   90: a    APPEND
+   91: .    STOP
 highest protocol among opcodes = 0
 
 Try again with a "binary" pickle.
@@ -2122,25 +2117,22 @@
    14: q        BINPUT     1
    16: }        EMPTY_DICT
    17: q        BINPUT     2
-   19: c        GLOBAL     '__builtin__ bytes'
-   38: q        BINPUT     3
-   40: (        MARK
-   41: ]            EMPTY_LIST
-   42: q            BINPUT     4
-   44: (            MARK
-   45: K                BININT1    97
-   47: K                BININT1    98
-   49: K                BININT1    99
-   51: e                APPENDS    (MARK at 44)
-   52: t            TUPLE      (MARK at 40)
-   53: q        BINPUT     5
-   55: R        REDUCE
-   56: q        BINPUT     6
-   58: X        BINUNICODE 'def'
-   66: q        BINPUT     7
-   68: s        SETITEM
-   69: e        APPENDS    (MARK at 3)
-   70: .    STOP
+   19: c        GLOBAL     '_codecs encode'
+   35: q        BINPUT     3
+   37: (        MARK
+   38: X            BINUNICODE 'abc'
+   46: q            BINPUT     4
+   48: X            BINUNICODE 'latin1'
+   59: q            BINPUT     5
+   61: t            TUPLE      (MARK at 37)
+   62: q        BINPUT     6
+   64: R        REDUCE
+   65: q        BINPUT     7
+   67: X        BINUNICODE 'def'
+   75: q        BINPUT     8
+   77: s        SETITEM
+   78: e        APPENDS    (MARK at 3)
+   79: .    STOP
 highest protocol among opcodes = 1
 
 Exercise the INST/OBJ/BUILD family.
diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py
--- a/Lib/test/pickletester.py
+++ b/Lib/test/pickletester.py
@@ -636,9 +636,15 @@
 
     def test_bytes(self):
         for proto in protocols:
-            for u in b'', b'xyz', b'xyz'*100:
-                p = self.dumps(u)
-                self.assertEqual(self.loads(p), u)
+            for s in b'', b'xyz', b'xyz'*100:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
+            for s in [bytes([i]) for i in range(256)]:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
+            for s in [bytes([i, i]) for i in range(256)]:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
 
     def test_ints(self):
         import sys
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #13505: Pickle bytes objects in a way that is compatible with
+  Python 2 when using protocols <= 2.
+
 - Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
   given by Campbell Barton).
 
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -369,7 +369,7 @@
     char *errors;               /* Name of errors handling scheme to used when
                                    decoding strings. The default value is
                                    "strict". */
-    Py_ssize_t *marks;                 /* Mark stack, used for unpickling container
+    Py_ssize_t *marks;          /* Mark stack, used for unpickling container
                                    objects. */
     Py_ssize_t num_marks;       /* Number of marks in the mark stack. */
     Py_ssize_t marks_size;      /* Current allocated size of the mark stack. */
@@ -1700,26 +1700,58 @@
     if (self->proto < 3) {
         /* Older pickle protocols do not have an opcode for pickling bytes
            objects. Therefore, we need to fake the copy protocol (i.e.,
-           the __reduce__ method) to permit bytes object unpickling. */
+           the __reduce__ method) to permit bytes object unpickling.
+
+           Here we use a hack to be compatible with Python 2. Since in Python
+           2 'bytes' is just an alias for 'str' (which has different
+           parameters than the actual bytes object), we use codecs.encode
+           to create the appropriate 'str' object when unpickled using
+           Python 2 *and* the appropriate 'bytes' object when unpickled
+           using Python 3. Again this is a hack and we don't need to do this
+           with newer protocols. */
+        static PyObject *codecs_encode = NULL;
         PyObject *reduce_value = NULL;
-        PyObject *bytelist = NULL;
         int status;
 
-        bytelist = PySequence_List(obj);
-        if (bytelist == NULL)
+        if (codecs_encode == NULL) {
+            PyObject *codecs_module = PyImport_ImportModule("codecs");
+            if (codecs_module == NULL) {
+                return -1;
+            }
+            codecs_encode = PyObject_GetAttrString(codecs_module, "encode");
+            Py_DECREF(codecs_module);
+            if (codecs_encode == NULL) {
+                return -1;
+            }
+        }
+
+        if (PyBytes_GET_SIZE(obj) == 0) {
+            reduce_value = Py_BuildValue("(O())", (PyObject*)&PyBytes_Type);
+        }
+        else {
+            static PyObject *latin1 = NULL;
+            PyObject *unicode_str =
+                PyUnicode_DecodeLatin1(PyBytes_AS_STRING(obj),
+                                       PyBytes_GET_SIZE(obj),
+                                       "strict");
+            if (unicode_str == NULL)
+                return -1;
+            if (latin1 == NULL) {
+                latin1 = PyUnicode_InternFromString("latin1");
+                if (latin1 == NULL)
+                    return -1;
+            }
+            reduce_value = Py_BuildValue("(O(OO))",
+                                         codecs_encode, unicode_str, latin1);
+            Py_DECREF(unicode_str);
+        }
+
+        if (reduce_value == NULL)
             return -1;
 
-        reduce_value = Py_BuildValue("(O(O))", (PyObject *)&PyBytes_Type,
-                                     bytelist);
-        if (reduce_value == NULL) {
-            Py_DECREF(bytelist);
-            return -1;
-        }
-
         /* save_reduce() will memoize the object automatically. */
         status = save_reduce(self, reduce_value, obj);
         Py_DECREF(reduce_value);
-        Py_DECREF(bytelist);
         return status;
     }
     else {
@@ -1727,7 +1759,7 @@
         char header[5];
         Py_ssize_t len;
 
-        size = PyBytes_Size(obj);
+        size = PyBytes_GET_SIZE(obj);
         if (size < 0)
             return -1;
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 13 19:23:25 2011
From: python-checkins at python.org (alexandre.vassalotti)
Date: Tue, 13 Dec 2011 19:23:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_3=2E2?=
Message-ID: <E1RaX0f-0006s1-SF@dinsdale.python.org>

http://hg.python.org/cpython/rev/98df29d51e12
changeset:   73958:98df29d51e12
parent:      73956:207408428242
parent:      73957:14695b4825dc
user:        Alexandre Vassalotti <alexandre at peadrop.com>
date:        Tue Dec 13 13:22:42 2011 -0500
summary:
  Merge 3.2

files:
  Lib/pickle.py            |   6 ++-
  Lib/pickletools.py       |  70 ++++++++++++---------------
  Lib/test/pickletester.py |  12 +++-
  Misc/NEWS                |   3 +
  Modules/_pickle.c        |  60 ++++++++++++++++++-----
  5 files changed, 94 insertions(+), 57 deletions(-)


diff --git a/Lib/pickle.py b/Lib/pickle.py
--- a/Lib/pickle.py
+++ b/Lib/pickle.py
@@ -485,7 +485,11 @@
 
     def save_bytes(self, obj, pack=struct.pack):
         if self.proto < 3:
-            self.save_reduce(bytes, (list(obj),), obj=obj)
+            if len(obj) == 0:
+                self.save_reduce(bytes, (), obj=obj)
+            else:
+                self.save_reduce(codecs.encode,
+                                 (str(obj, 'latin1'), 'latin1'), obj=obj)
             return
         n = len(obj)
         if n < 256:
diff --git a/Lib/pickletools.py b/Lib/pickletools.py
--- a/Lib/pickletools.py
+++ b/Lib/pickletools.py
@@ -2082,27 +2082,22 @@
    29: (    MARK
    30: d        DICT       (MARK at 29)
    31: p    PUT        2
-   34: c    GLOBAL     '__builtin__ bytes'
-   53: p    PUT        3
-   56: (    MARK
-   57: (        MARK
-   58: l            LIST       (MARK at 57)
+   34: c    GLOBAL     '_codecs encode'
+   50: p    PUT        3
+   53: (    MARK
+   54: V        UNICODE    'abc'
    59: p        PUT        4
-   62: L        LONG       97
-   67: a        APPEND
-   68: L        LONG       98
-   73: a        APPEND
-   74: L        LONG       99
-   79: a        APPEND
-   80: t        TUPLE      (MARK at 56)
-   81: p    PUT        5
-   84: R    REDUCE
-   85: p    PUT        6
-   88: V    UNICODE    'def'
-   93: p    PUT        7
-   96: s    SETITEM
-   97: a    APPEND
-   98: .    STOP
+   62: V        UNICODE    'latin1'
+   70: p        PUT        5
+   73: t        TUPLE      (MARK at 53)
+   74: p    PUT        6
+   77: R    REDUCE
+   78: p    PUT        7
+   81: V    UNICODE    'def'
+   86: p    PUT        8
+   89: s    SETITEM
+   90: a    APPEND
+   91: .    STOP
 highest protocol among opcodes = 0
 
 Try again with a "binary" pickle.
@@ -2121,25 +2116,22 @@
    14: q        BINPUT     1
    16: }        EMPTY_DICT
    17: q        BINPUT     2
-   19: c        GLOBAL     '__builtin__ bytes'
-   38: q        BINPUT     3
-   40: (        MARK
-   41: ]            EMPTY_LIST
-   42: q            BINPUT     4
-   44: (            MARK
-   45: K                BININT1    97
-   47: K                BININT1    98
-   49: K                BININT1    99
-   51: e                APPENDS    (MARK at 44)
-   52: t            TUPLE      (MARK at 40)
-   53: q        BINPUT     5
-   55: R        REDUCE
-   56: q        BINPUT     6
-   58: X        BINUNICODE 'def'
-   66: q        BINPUT     7
-   68: s        SETITEM
-   69: e        APPENDS    (MARK at 3)
-   70: .    STOP
+   19: c        GLOBAL     '_codecs encode'
+   35: q        BINPUT     3
+   37: (        MARK
+   38: X            BINUNICODE 'abc'
+   46: q            BINPUT     4
+   48: X            BINUNICODE 'latin1'
+   59: q            BINPUT     5
+   61: t            TUPLE      (MARK at 37)
+   62: q        BINPUT     6
+   64: R        REDUCE
+   65: q        BINPUT     7
+   67: X        BINUNICODE 'def'
+   75: q        BINPUT     8
+   77: s        SETITEM
+   78: e        APPENDS    (MARK at 3)
+   79: .    STOP
 highest protocol among opcodes = 1
 
 Exercise the INST/OBJ/BUILD family.
diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py
--- a/Lib/test/pickletester.py
+++ b/Lib/test/pickletester.py
@@ -637,9 +637,15 @@
 
     def test_bytes(self):
         for proto in protocols:
-            for u in b'', b'xyz', b'xyz'*100:
-                p = self.dumps(u)
-                self.assertEqual(self.loads(p), u)
+            for s in b'', b'xyz', b'xyz'*100:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
+            for s in [bytes([i]) for i in range(256)]:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
+            for s in [bytes([i, i]) for i in range(256)]:
+                p = self.dumps(s)
+                self.assertEqual(self.loads(p), s)
 
     def test_ints(self):
         import sys
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #13505: Pickle bytes objects in a way that is compatible with
+  Python 2 when using protocols <= 2.
+
 - Issue #11147: Fix an unused argument in _Py_ANNOTATE_MEMORY_ORDER.  (Fix
   given by Campbell Barton).
 
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -369,7 +369,7 @@
     char *errors;               /* Name of errors handling scheme to used when
                                    decoding strings. The default value is
                                    "strict". */
-    Py_ssize_t *marks;                 /* Mark stack, used for unpickling container
+    Py_ssize_t *marks;          /* Mark stack, used for unpickling container
                                    objects. */
     Py_ssize_t num_marks;       /* Number of marks in the mark stack. */
     Py_ssize_t marks_size;      /* Current allocated size of the mark stack. */
@@ -1708,26 +1708,58 @@
     if (self->proto < 3) {
         /* Older pickle protocols do not have an opcode for pickling bytes
            objects. Therefore, we need to fake the copy protocol (i.e.,
-           the __reduce__ method) to permit bytes object unpickling. */
+           the __reduce__ method) to permit bytes object unpickling.
+
+           Here we use a hack to be compatible with Python 2. Since in Python
+           2 'bytes' is just an alias for 'str' (which has different
+           parameters than the actual bytes object), we use codecs.encode
+           to create the appropriate 'str' object when unpickled using
+           Python 2 *and* the appropriate 'bytes' object when unpickled
+           using Python 3. Again this is a hack and we don't need to do this
+           with newer protocols. */
+        static PyObject *codecs_encode = NULL;
         PyObject *reduce_value = NULL;
-        PyObject *bytelist = NULL;
         int status;
 
-        bytelist = PySequence_List(obj);
-        if (bytelist == NULL)
+        if (codecs_encode == NULL) {
+            PyObject *codecs_module = PyImport_ImportModule("codecs");
+            if (codecs_module == NULL) {
+                return -1;
+            }
+            codecs_encode = PyObject_GetAttrString(codecs_module, "encode");
+            Py_DECREF(codecs_module);
+            if (codecs_encode == NULL) {
+                return -1;
+            }
+        }
+
+        if (PyBytes_GET_SIZE(obj) == 0) {
+            reduce_value = Py_BuildValue("(O())", (PyObject*)&PyBytes_Type);
+        }
+        else {
+            static PyObject *latin1 = NULL;
+            PyObject *unicode_str =
+                PyUnicode_DecodeLatin1(PyBytes_AS_STRING(obj),
+                                       PyBytes_GET_SIZE(obj),
+                                       "strict");
+            if (unicode_str == NULL)
+                return -1;
+            if (latin1 == NULL) {
+                latin1 = PyUnicode_InternFromString("latin1");
+                if (latin1 == NULL)
+                    return -1;
+            }
+            reduce_value = Py_BuildValue("(O(OO))",
+                                         codecs_encode, unicode_str, latin1);
+            Py_DECREF(unicode_str);
+        }
+
+        if (reduce_value == NULL)
             return -1;
 
-        reduce_value = Py_BuildValue("(O(O))", (PyObject *)&PyBytes_Type,
-                                     bytelist);
-        if (reduce_value == NULL) {
-            Py_DECREF(bytelist);
-            return -1;
-        }
-
         /* save_reduce() will memoize the object automatically. */
         status = save_reduce(self, reduce_value, obj);
         Py_DECREF(reduce_value);
-        Py_DECREF(bytelist);
         return status;
     }
     else {
@@ -1735,7 +1767,7 @@
         char header[5];
         Py_ssize_t len;
 
-        size = PyBytes_Size(obj);
+        size = PyBytes_GET_SIZE(obj);
         if (size < 0)
             return -1;
 

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Wed Dec 14 05:31:27 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 14 Dec 2011 05:31:27 +0100
Subject: [Python-checkins] Daily reference leaks (98df29d51e12): sum=0
Message-ID: <E1RagV5-0004xl-PU@ap.vmr.nerim.net>

results for 98df29d51e12 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogKR3zCb', '-x']

From python-checkins at python.org  Wed Dec 14 13:34:40 2011
From: python-checkins at python.org (giampaolo.rodola)
Date: Wed, 14 Dec 2011 13:34:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_=238684=3A_make_sched?=
 =?utf8?q?=2Escheduler_class_thread-safe?=
Message-ID: <E1Rao2i-0004ff-Ry@dinsdale.python.org>

http://hg.python.org/cpython/rev/f5aed0dba844
changeset:   73959:f5aed0dba844
user:        Giampaolo Rodola' <g.rodola at gmail.com>
date:        Wed Dec 14 13:34:26 2011 +0100
summary:
  Fix #8684: make sched.scheduler class thread-safe

files:
  Doc/library/sched.rst |  30 +-------------
  Doc/whatsnew/3.3.rst  |   4 ++
  Lib/sched.py          |  62 +++++++++++++++++-------------
  Misc/NEWS             |   3 +
  4 files changed, 45 insertions(+), 54 deletions(-)


diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst
--- a/Doc/library/sched.rst
+++ b/Doc/library/sched.rst
@@ -27,6 +27,9 @@
 
    .. versionchanged:: 3.3
       *timefunc* and *delayfunc* parameters are optional.
+   .. versionchanged:: 3.3
+      :class:`scheduler` class can be safely used in multi-threaded
+      environments.
 
 Example::
 
@@ -47,33 +50,6 @@
    From print_time 930343700.273
    930343700.276
 
-In multi-threaded environments, the :class:`scheduler` class has limitations
-with respect to thread-safety, inability to insert a new task before
-the one currently pending in a running scheduler, and holding up the main
-thread until the event queue is empty.  Instead, the preferred approach
-is to use the :class:`threading.Timer` class instead.
-
-Example::
-
-    >>> import time
-    >>> from threading import Timer
-    >>> def print_time():
-    ...     print("From print_time", time.time())
-    ...
-    >>> def print_some_times():
-    ...     print(time.time())
-    ...     Timer(5, print_time, ()).start()
-    ...     Timer(10, print_time, ()).start()
-    ...     time.sleep(11)  # sleep while time-delay events execute
-    ...     print(time.time())
-    ...
-    >>> print_some_times()
-    930343690.257
-    From print_time 930343695.274
-    From print_time 930343700.273
-    930343701.301
-
-
 .. _scheduler-objects:
 
 Scheduler Objects
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -662,6 +662,10 @@
 sched
 -----
 
+* :class:`~sched.scheduler` class can now be safely used in multi-threaded
+  environments.  (Contributed by Josiah Carlson and Giampaolo Rodol? in
+  :issue:`8684`)
+
 * *timefunc* and *delayfunct* parameters of :class:`~sched.scheduler` class
   constructor are now optional and defaults to :func:`time.time` and
   :func:`time.sleep` respectively.  (Contributed by Chris Clark in
diff --git a/Lib/sched.py b/Lib/sched.py
--- a/Lib/sched.py
+++ b/Lib/sched.py
@@ -30,6 +30,7 @@
 
 import time
 import heapq
+import threading
 from collections import namedtuple
 
 __all__ = ["scheduler"]
@@ -48,6 +49,7 @@
         """Initialize a new instance, passing the time and delay
         functions"""
         self._queue = []
+        self._lock = threading.RLock()
         self.timefunc = timefunc
         self.delayfunc = delayfunc
 
@@ -58,9 +60,10 @@
         if necessary.
 
         """
-        event = Event(time, priority, action, argument, kwargs)
-        heapq.heappush(self._queue, event)
-        return event # The ID
+        with self._lock:
+            event = Event(time, priority, action, argument, kwargs)
+            heapq.heappush(self._queue, event)
+            return event # The ID
 
     def enter(self, delay, priority, action, argument=[], kwargs={}):
         """A variant that specifies the time as a relative time.
@@ -68,8 +71,9 @@
         This is actually the more commonly used interface.
 
         """
-        time = self.timefunc() + delay
-        return self.enterabs(time, priority, action, argument, kwargs)
+        with self._lock:
+            time = self.timefunc() + delay
+            return self.enterabs(time, priority, action, argument, kwargs)
 
     def cancel(self, event):
         """Remove an event from the queue.
@@ -78,12 +82,14 @@
         If the event is not in the queue, this raises ValueError.
 
         """
-        self._queue.remove(event)
-        heapq.heapify(self._queue)
+        with self._lock:
+            self._queue.remove(event)
+            heapq.heapify(self._queue)
 
     def empty(self):
         """Check whether the queue is empty."""
-        return not self._queue
+        with self._lock:
+            return not self._queue
 
     def run(self):
         """Execute events until the queue is empty.
@@ -108,24 +114,25 @@
         """
         # localize variable access to minimize overhead
         # and to improve thread safety
-        q = self._queue
-        delayfunc = self.delayfunc
-        timefunc = self.timefunc
-        pop = heapq.heappop
-        while q:
-            time, priority, action, argument, kwargs = checked_event = q[0]
-            now = timefunc()
-            if now < time:
-                delayfunc(time - now)
-            else:
-                event = pop(q)
-                # Verify that the event was not removed or altered
-                # by another thread after we last looked at q[0].
-                if event is checked_event:
-                    action(*argument, **kwargs)
-                    delayfunc(0)   # Let other threads run
+        with self._lock:
+            q = self._queue
+            delayfunc = self.delayfunc
+            timefunc = self.timefunc
+            pop = heapq.heappop
+            while q:
+                time, priority, action, argument, kwargs = checked_event = q[0]
+                now = timefunc()
+                if now < time:
+                    delayfunc(time - now)
                 else:
-                    heapq.heappush(q, event)
+                    event = pop(q)
+                    # Verify that the event was not removed or altered
+                    # by another thread after we last looked at q[0].
+                    if event is checked_event:
+                        action(*argument, **kwargs)
+                        delayfunc(0)   # Let other threads run
+                    else:
+                        heapq.heappush(q, event)
 
     @property
     def queue(self):
@@ -138,5 +145,6 @@
         # Use heapq to sort the queue rather than using 'sorted(self._queue)'.
         # With heapq, two events scheduled at the same time will show in
         # the actual order they would be retrieved.
-        events = self._queue[:]
-        return map(heapq.heappop, [events]*len(events))
+        with self._lock:
+            events = self._queue[:]
+            return map(heapq.heappop, [events]*len(events))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,6 +409,9 @@
 Library
 -------
 
+- Issue #8684 sched.scheduler class can be safely used in multi-threaded
+  environments.
+
 - Alias resource.error to OSError ala PEP 3151.
 
 - Issue #5689: Add support for lzma compression to the tarfile module.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 14:38:50 2011
From: python-checkins at python.org (giampaolo.rodola)
Date: Wed, 14 Dec 2011 14:38:50 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_=2313449=3A_add_=27bloc?=
 =?utf8?q?king=27_parameter_to_sched=2Escheduler=2Erun=28=29_so_that_the?=
Message-ID: <E1Rap2o-0002tK-Gj@dinsdale.python.org>

http://hg.python.org/cpython/rev/2975618965c0
changeset:   73960:2975618965c0
user:        Giampaolo Rodola' <g.rodola at gmail.com>
date:        Wed Dec 14 14:38:45 2011 +0100
summary:
  Fix #13449: add 'blocking' parameter to sched.scheduler.run() so that the scheduler can be used in non-blocking applications

files:
  Doc/library/sched.rst  |  10 ++++++++--
  Doc/whatsnew/3.3.rst   |   6 ++++++
  Lib/sched.py           |   6 +++++-
  Lib/test/test_sched.py |  10 ++++++++++
  Misc/NEWS              |   6 +++++-
  5 files changed, 34 insertions(+), 4 deletions(-)


diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst
--- a/Doc/library/sched.rst
+++ b/Doc/library/sched.rst
@@ -102,12 +102,15 @@
    Return true if the event queue is empty.
 
 
-.. method:: scheduler.run()
+.. method:: scheduler.run(blocking=True)
 
-   Run all scheduled events. This function will wait  (using the :func:`delayfunc`
+   Run all scheduled events. This method will wait  (using the :func:`delayfunc`
    function passed to the constructor) for the next event, then execute it and so
    on until there are no more scheduled events.
 
+   If *blocking* is False executes the scheduled events due to expire soonest
+   (if any) and then return.
+
    Either *action* or *delayfunc* can raise an exception.  In either case, the
    scheduler will maintain a consistent state and propagate the exception.  If an
    exception is raised by *action*, the event will not be attempted in future calls
@@ -118,6 +121,9 @@
    the calling code is responsible for canceling  events which are no longer
    pertinent.
 
+   .. versionadded:: 3.3
+      *blocking* parameter was added.
+
 .. attribute:: scheduler.queue
 
    Read-only attribute returning a list of upcoming events in the order they
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -662,6 +662,12 @@
 sched
 -----
 
+* :meth:`~sched.scheduler.run` now accepts a *blocking* parameter which when
+  set to False makes the method execute the scheduled events due to expire
+  soonest (if any) and then return immediately.
+  This is useful in case you want to use the :class:`~sched.scheduler` in
+  non-blocking applications.  (Contributed by Giampaolo Rodol? in :issue:`13449`)
+
 * :class:`~sched.scheduler` class can now be safely used in multi-threaded
   environments.  (Contributed by Josiah Carlson and Giampaolo Rodol? in
   :issue:`8684`)
diff --git a/Lib/sched.py b/Lib/sched.py
--- a/Lib/sched.py
+++ b/Lib/sched.py
@@ -91,8 +91,10 @@
         with self._lock:
             return not self._queue
 
-    def run(self):
+    def run(self, blocking=True):
         """Execute events until the queue is empty.
+        If blocking is False executes the scheduled events due to
+        expire soonest (if any) and then return.
 
         When there is a positive delay until the first event, the
         delay function is called and the event is left in the queue;
@@ -123,6 +125,8 @@
                 time, priority, action, argument, kwargs = checked_event = q[0]
                 now = timefunc()
                 if now < time:
+                    if not blocking:
+                        return
                     delayfunc(time - now)
                 else:
                     event = pop(q)
diff --git a/Lib/test/test_sched.py b/Lib/test/test_sched.py
--- a/Lib/test/test_sched.py
+++ b/Lib/test/test_sched.py
@@ -86,6 +86,16 @@
         scheduler.run()
         self.assertEqual(flag, [None])
 
+    def test_run_non_blocking(self):
+        l = []
+        fun = lambda x: l.append(x)
+        scheduler = sched.scheduler(time.time, time.sleep)
+        for x in [10, 9, 8, 7, 6]:
+            scheduler.enter(x, 1, fun, (x,))
+        scheduler.run(blocking=False)
+        self.assertEqual(l, [])
+
+
 def test_main():
     support.run_unittest(TestCase)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,8 +409,12 @@
 Library
 -------
 
+- Issue #13449 sched.scheduler.run() method has a new "blocking" parameter which
+  when set to False makes run() execute the scheduled events due to expire
+  soonest (if any) and then return.  Patch by Giampaolo Rodol?.
+
 - Issue #8684 sched.scheduler class can be safely used in multi-threaded
-  environments.
+  environments.  Patch by Josiah Carlson and Giampaolo Rodol?.
 
 - Alias resource.error to OSError ala PEP 3151.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 18:41:22 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 18:41:22 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzQwMjg6?=
 =?utf8?q?_Make_multiprocessing_build_on_SunOS=2E?=
Message-ID: <E1RaspW-00021S-5n@dinsdale.python.org>

http://hg.python.org/cpython/rev/8c658b625475
changeset:   73961:8c658b625475
branch:      2.7
parent:      73954:d60856651139
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 18:35:55 2011 +0100
summary:
  Issue #4028: Make multiprocessing build on SunOS.

files:
  Modules/_multiprocessing/multiprocessing.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_multiprocessing/multiprocessing.c b/Modules/_multiprocessing/multiprocessing.c
--- a/Modules/_multiprocessing/multiprocessing.c
+++ b/Modules/_multiprocessing/multiprocessing.c
@@ -8,7 +8,7 @@
 
 #include "multiprocessing.h"
 
-#ifdef SCM_RIGHTS
+#if (defined(CMSG_LEN) && defined(SCM_RIGHTS))
     #define HAVE_FD_TRANSFER 1
 #else
     #define HAVE_FD_TRANSFER 0

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 18:41:23 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 18:41:23 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzQwMjg6?=
 =?utf8?q?_Make_multiprocessing_build_on_SunOS=2E?=
Message-ID: <E1RaspX-00021Y-3q@dinsdale.python.org>

http://hg.python.org/cpython/rev/49e82c885d6b
changeset:   73962:49e82c885d6b
branch:      3.2
parent:      73957:14695b4825dc
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 18:39:09 2011 +0100
summary:
  Issue #4028: Make multiprocessing build on SunOS.

files:
  Modules/_multiprocessing/multiprocessing.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_multiprocessing/multiprocessing.c b/Modules/_multiprocessing/multiprocessing.c
--- a/Modules/_multiprocessing/multiprocessing.c
+++ b/Modules/_multiprocessing/multiprocessing.c
@@ -8,7 +8,7 @@
 
 #include "multiprocessing.h"
 
-#ifdef SCM_RIGHTS
+#if (defined(CMSG_LEN) && defined(SCM_RIGHTS))
     #define HAVE_FD_TRANSFER 1
 #else
     #define HAVE_FD_TRANSFER 0

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 18:41:24 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 18:41:24 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Null_merge_-_Issue_=234028=3A_Make_multiprocessing_build_on_?=
 =?utf8?q?SunOS=2E?=
Message-ID: <E1RaspY-00021c-6U@dinsdale.python.org>

http://hg.python.org/cpython/rev/4a27b5ab2710
changeset:   73963:4a27b5ab2710
parent:      73960:2975618965c0
parent:      73962:49e82c885d6b
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 18:40:35 2011 +0100
summary:
  Null merge - Issue #4028: Make multiprocessing build on SunOS.

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 19:31:00 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 19:31:00 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Fix_a_race_condition_in_test=5Fpoplib=2E?=
Message-ID: <E1RatbY-0001S7-Qi@dinsdale.python.org>

http://hg.python.org/cpython/rev/fbcaeb4a8654
changeset:   73964:fbcaeb4a8654
branch:      2.7
parent:      73961:8c658b625475
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 19:28:08 2011 +0100
summary:
  Issue #13453: Fix a race condition in test_poplib.

files:
  Lib/test/test_poplib.py |  22 ++++++++++++----------
  1 files changed, 12 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py
--- a/Lib/test/test_poplib.py
+++ b/Lib/test/test_poplib.py
@@ -281,32 +281,34 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(60)  # Safety net. Look issue 11812
         self.port = test_support.bind_port(self.sock)
-        threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
-        time.sleep(.1)
+        self.thread = threading.Thread(target=self.server, args=(self.evt,self.sock))
+        self.thread.setDaemon(True)
+        self.thread.start()
+        self.evt.wait()
 
     def tearDown(self):
-        self.evt.wait()
+        self.thread.join()
+        del self.thread  # Clear out any dangling Thread objects.
 
     def server(self, evt, serv):
         serv.listen(5)
+        evt.set()
         try:
             conn, addr = serv.accept()
+            conn.send("+ Hola mundo\n")
+            conn.close()
         except socket.timeout:
             pass
-        else:
-            conn.send("+ Hola mundo\n")
-            conn.close()
         finally:
             serv.close()
-            evt.set()
 
     def testTimeoutDefault(self):
         self.assertTrue(socket.getdefaulttimeout() is None)
         socket.setdefaulttimeout(30)
         try:
-            pop = poplib.POP3("localhost", self.port)
+            pop = poplib.POP3(HOST, self.port)
         finally:
             socket.setdefaulttimeout(None)
         self.assertEqual(pop.sock.gettimeout(), 30)
@@ -323,7 +325,7 @@
         pop.sock.close()
 
     def testTimeoutValue(self):
-        pop = poplib.POP3("localhost", self.port, timeout=30)
+        pop = poplib.POP3(HOST, self.port, timeout=30)
         self.assertEqual(pop.sock.gettimeout(), 30)
         pop.sock.close()
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 19:31:01 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 19:31:01 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Fix_a_race_condition_in_test=5Fpoplib=2E?=
Message-ID: <E1RatbZ-0001SH-Jn@dinsdale.python.org>

http://hg.python.org/cpython/rev/e497a3ed9beb
changeset:   73965:e497a3ed9beb
branch:      3.2
parent:      73962:49e82c885d6b
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 19:28:56 2011 +0100
summary:
  Issue #13453: Fix a race condition in test_poplib.

files:
  Lib/test/test_poplib.py |  22 ++++++++++++----------
  1 files changed, 12 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py
--- a/Lib/test/test_poplib.py
+++ b/Lib/test/test_poplib.py
@@ -309,32 +309,34 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(60)  # Safety net. Look issue 11812
         self.port = test_support.bind_port(self.sock)
-        threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
-        time.sleep(.1)
+        self.thread = threading.Thread(target=self.server, args=(self.evt,self.sock))
+        self.thread.setDaemon(True)
+        self.thread.start()
+        self.evt.wait()
 
     def tearDown(self):
-        self.evt.wait()
+        self.thread.join()
+        del self.thread  # Clear out any dangling Thread objects.
 
     def server(self, evt, serv):
         serv.listen(5)
+        evt.set()
         try:
             conn, addr = serv.accept()
+            conn.send(b"+ Hola mundo\n")
+            conn.close()
         except socket.timeout:
             pass
-        else:
-            conn.send(b"+ Hola mundo\n")
-            conn.close()
         finally:
             serv.close()
-            evt.set()
 
     def testTimeoutDefault(self):
         self.assertTrue(socket.getdefaulttimeout() is None)
         socket.setdefaulttimeout(30)
         try:
-            pop = poplib.POP3("localhost", self.port)
+            pop = poplib.POP3(HOST, self.port)
         finally:
             socket.setdefaulttimeout(None)
         self.assertEqual(pop.sock.gettimeout(), 30)
@@ -351,7 +353,7 @@
         pop.sock.close()
 
     def testTimeoutValue(self):
-        pop = poplib.POP3("localhost", self.port, timeout=30)
+        pop = poplib.POP3(HOST, self.port, timeout=30)
         self.assertEqual(pop.sock.gettimeout(), 30)
         pop.sock.close()
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 14 19:31:02 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Wed, 14 Dec 2011 19:31:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313453=3A_Fix_a_race_condition_in_test=5Fpoplib=2E?=
Message-ID: <E1Ratba-0001SS-BK@dinsdale.python.org>

http://hg.python.org/cpython/rev/1eae8154c109
changeset:   73966:1eae8154c109
parent:      73963:4a27b5ab2710
parent:      73965:e497a3ed9beb
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Wed Dec 14 19:29:56 2011 +0100
summary:
  Issue #13453: Fix a race condition in test_poplib.

files:
  Lib/test/test_poplib.py |  22 ++++++++++++----------
  1 files changed, 12 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py
--- a/Lib/test/test_poplib.py
+++ b/Lib/test/test_poplib.py
@@ -320,32 +320,34 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(60)  # Safety net. Look issue 11812
         self.port = test_support.bind_port(self.sock)
-        threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
-        time.sleep(.1)
+        self.thread = threading.Thread(target=self.server, args=(self.evt,self.sock))
+        self.thread.setDaemon(True)
+        self.thread.start()
+        self.evt.wait()
 
     def tearDown(self):
-        self.evt.wait()
+        self.thread.join()
+        del self.thread  # Clear out any dangling Thread objects.
 
     def server(self, evt, serv):
         serv.listen(5)
+        evt.set()
         try:
             conn, addr = serv.accept()
+            conn.send(b"+ Hola mundo\n")
+            conn.close()
         except socket.timeout:
             pass
-        else:
-            conn.send(b"+ Hola mundo\n")
-            conn.close()
         finally:
             serv.close()
-            evt.set()
 
     def testTimeoutDefault(self):
         self.assertTrue(socket.getdefaulttimeout() is None)
         socket.setdefaulttimeout(30)
         try:
-            pop = poplib.POP3("localhost", self.port)
+            pop = poplib.POP3(HOST, self.port)
         finally:
             socket.setdefaulttimeout(None)
         self.assertEqual(pop.sock.gettimeout(), 30)
@@ -362,7 +364,7 @@
         pop.sock.close()
 
     def testTimeoutValue(self):
-        pop = poplib.POP3("localhost", self.port, timeout=30)
+        pop = poplib.POP3(HOST, self.port, timeout=30)
         self.assertEqual(pop.sock.gettimeout(), 30)
         pop.sock.close()
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:29 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:29 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzQ2MjU6?=
 =?utf8?q?_If_IDLE_cannot_write_to_its_recent_file_or_breakpoint?=
Message-ID: <E1Ray5l-0001jI-1N@dinsdale.python.org>

http://hg.python.org/cpython/rev/7857d1f4ce79
changeset:   73967:7857d1f4ce79
branch:      2.7
parent:      73964:fbcaeb4a8654
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 14:57:43 2011 -0800
summary:
  Issue #4625: If IDLE cannot write to its recent file or breakpoint
files, display a message popup and continue rather than crash.
(original patch by Roger Serwy)

files:
  Lib/idlelib/EditorWindow.py |  13 ++++++++---
  Lib/idlelib/PyShell.py      |  28 ++++++++++++++++--------
  2 files changed, 27 insertions(+), 14 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -796,11 +796,16 @@
         rf_list = [path for path in rf_list if path not in bad_paths]
         ulchars = "1234567890ABCDEFGHIJK"
         rf_list = rf_list[0:len(ulchars)]
-        rf_file = open(self.recent_files_path, 'w')
         try:
-            rf_file.writelines(rf_list)
-        finally:
-            rf_file.close()
+            with open(self.recent_files_path, 'w') as rf_file:
+                rf_file.writelines(rf_list)
+        except IOError as err:
+            if not getattr(self.root, "recentfilelist_error_displayed", False):
+                self.root.recentfilelist_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update Recent Files list:\n%s'
+                        % str(err),
+                    parent=self.text)
         # for each edit window instance, construct the recent files menu
         for instance in self.top.instance_dict.keys():
             menu = instance.recent_files_menu
diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -207,18 +207,26 @@
         breaks = self.breakpoints
         filename = self.io.filename
         try:
-            lines = open(self.breakpointPath,"r").readlines()
+            with open(self.breakpointPath,"r") as old_file:
+                lines = old_file.readlines()
         except IOError:
             lines = []
-        new_file = open(self.breakpointPath,"w")
-        for line in lines:
-            if not line.startswith(filename + '='):
-                new_file.write(line)
-        self.update_breakpoints()
-        breaks = self.breakpoints
-        if breaks:
-            new_file.write(filename + '=' + str(breaks) + '\n')
-        new_file.close()
+        try:
+            with open(self.breakpointPath,"w") as new_file:
+                for line in lines:
+                    if not line.startswith(filename + '='):
+                        new_file.write(line)
+                self.update_breakpoints()
+                breaks = self.breakpoints
+                if breaks:
+                    new_file.write(filename + '=' + str(breaks) + '\n')
+        except IOError as err:
+            if not getattr(self.root, "breakpoint_error_displayed", False):
+                self.root.breakpoint_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update breakpoint list:\n%s'
+                        % str(err),
+                    parent=self.text)
 
     def restore_file_breaks(self):
         self.text.update()   # this enables setting "BREAK" tags to be visible

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:29 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:29 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzQ2MjU6?=
 =?utf8?q?_If_IDLE_cannot_write_to_its_recent_file_or_breakpoint?=
Message-ID: <E1Ray5l-0001jX-Sn@dinsdale.python.org>

http://hg.python.org/cpython/rev/26e3e542d20d
changeset:   73968:26e3e542d20d
branch:      3.2
parent:      73965:e497a3ed9beb
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 14:58:24 2011 -0800
summary:
  Issue #4625: If IDLE cannot write to its recent file or breakpoint
files, display a message popup and continue rather than crash.
(original patch by Roger Serwy)

files:
  Lib/idlelib/EditorWindow.py |  15 ++++++++++-----
  Lib/idlelib/PyShell.py      |  24 ++++++++++++++++--------
  2 files changed, 26 insertions(+), 13 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -799,12 +799,17 @@
         rf_list = [path for path in rf_list if path not in bad_paths]
         ulchars = "1234567890ABCDEFGHIJK"
         rf_list = rf_list[0:len(ulchars)]
-        rf_file = open(self.recent_files_path, 'w',
-                        encoding='utf_8', errors='replace')
         try:
-            rf_file.writelines(rf_list)
-        finally:
-            rf_file.close()
+            with open(self.recent_files_path, 'w',
+                        encoding='utf_8', errors='replace') as rf_file:
+                rf_file.writelines(rf_list)
+        except IOError as err:
+            if not getattr(self.root, "recentfilelist_error_displayed", False):
+                self.root.recentfilelist_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update Recent Files list:\n%s'
+                        % str(err),
+                    parent=self.text)
         # for each edit window instance, construct the recent files menu
         for instance in self.top.instance_dict:
             menu = instance.recent_files_menu
diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -206,14 +206,22 @@
                 lines = fp.readlines()
         except IOError:
             lines = []
-        with open(self.breakpointPath, "w") as new_file:
-            for line in lines:
-                if not line.startswith(filename + '='):
-                    new_file.write(line)
-            self.update_breakpoints()
-            breaks = self.breakpoints
-            if breaks:
-                new_file.write(filename + '=' + str(breaks) + '\n')
+        try:
+            with open(self.breakpointPath, "w") as new_file:
+                for line in lines:
+                    if not line.startswith(filename + '='):
+                        new_file.write(line)
+                self.update_breakpoints()
+                breaks = self.breakpoints
+                if breaks:
+                    new_file.write(filename + '=' + str(breaks) + '\n')
+        except IOError as err:
+            if not getattr(self.root, "breakpoint_error_displayed", False):
+                self.root.breakpoint_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update breakpoint list:\n%s'
+                        % str(err),
+                    parent=self.text)
 
     def restore_file_breaks(self):
         self.text.update()   # this enables setting "BREAK" tags to be visible

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:30 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:30 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=234625=3A_If_IDLE_cannot_write_to_its_recent_file_or_?=
 =?utf8?q?breakpoint?=
Message-ID: <E1Ray5m-0001jm-Kx@dinsdale.python.org>

http://hg.python.org/cpython/rev/ba832d42cd9e
changeset:   73969:ba832d42cd9e
parent:      73966:1eae8154c109
parent:      73968:26e3e542d20d
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 15:03:31 2011 -0800
summary:
  Issue #4625: If IDLE cannot write to its recent file or breakpoint
files, display a message popup and continue rather than crash.
(original patch by Roger Serwy)

files:
  Lib/idlelib/EditorWindow.py |  15 ++++++++++-----
  Lib/idlelib/PyShell.py      |  24 ++++++++++++++++--------
  2 files changed, 26 insertions(+), 13 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -799,12 +799,17 @@
         rf_list = [path for path in rf_list if path not in bad_paths]
         ulchars = "1234567890ABCDEFGHIJK"
         rf_list = rf_list[0:len(ulchars)]
-        rf_file = open(self.recent_files_path, 'w',
-                        encoding='utf_8', errors='replace')
         try:
-            rf_file.writelines(rf_list)
-        finally:
-            rf_file.close()
+            with open(self.recent_files_path, 'w',
+                        encoding='utf_8', errors='replace') as rf_file:
+                rf_file.writelines(rf_list)
+        except IOError as err:
+            if not getattr(self.root, "recentfilelist_error_displayed", False):
+                self.root.recentfilelist_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update Recent Files list:\n%s'
+                        % str(err),
+                    parent=self.text)
         # for each edit window instance, construct the recent files menu
         for instance in self.top.instance_dict:
             menu = instance.recent_files_menu
diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -206,14 +206,22 @@
                 lines = fp.readlines()
         except IOError:
             lines = []
-        with open(self.breakpointPath, "w") as new_file:
-            for line in lines:
-                if not line.startswith(filename + '='):
-                    new_file.write(line)
-            self.update_breakpoints()
-            breaks = self.breakpoints
-            if breaks:
-                new_file.write(filename + '=' + str(breaks) + '\n')
+        try:
+            with open(self.breakpointPath, "w") as new_file:
+                for line in lines:
+                    if not line.startswith(filename + '='):
+                        new_file.write(line)
+                self.update_breakpoints()
+                breaks = self.breakpoints
+                if breaks:
+                    new_file.write(filename + '=' + str(breaks) + '\n')
+        except IOError as err:
+            if not getattr(self.root, "breakpoint_error_displayed", False):
+                self.root.breakpoint_error_displayed = True
+                tkMessageBox.showerror(title='IDLE Error',
+                    message='Unable to update breakpoint list:\n%s'
+                        % str(err),
+                    parent=self.text)
 
     def restore_file_breaks(self):
         self.text.update()   # this enables setting "BREAK" tags to be visible

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:31 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:31 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzQ2MjU6?=
	=?utf8?q?_add_NEWS_entry=2E?=
Message-ID: <E1Ray5n-0001kV-EI@dinsdale.python.org>

http://hg.python.org/cpython/rev/f6ca22bbf138
changeset:   73970:f6ca22bbf138
branch:      2.7
parent:      73967:7857d1f4ce79
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 15:05:42 2011 -0800
summary:
  Issue #4625: add NEWS entry.

files:
  Misc/NEWS |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -82,6 +82,10 @@
 Library
 -------
 
+- Issue #4625: If IDLE cannot write to its recent file or breakpoint
+  files, display a message popup and continue rather than crash.
+  (original patch by Roger Serwy)
+
 - tarfile.py: Correctly detect bzip2 compressed streams with blocksizes
   other than 900k.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:32 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:32 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzQ2MjU6?=
	=?utf8?q?_add_NEWS_entry=2E?=
Message-ID: <E1Ray5o-0001kl-7E@dinsdale.python.org>

http://hg.python.org/cpython/rev/f1fe411bfd6b
changeset:   73971:f1fe411bfd6b
branch:      3.2
parent:      73968:26e3e542d20d
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 15:06:50 2011 -0800
summary:
  Issue #4625: add NEWS entry.

files:
  Misc/NEWS |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -93,6 +93,10 @@
 Library
 -------
 
+- Issue #4625: If IDLE cannot write to its recent file or breakpoint
+  files, display a message popup and continue rather than crash.
+  (original patch by Roger Serwy)
+
 - Issue #5905: time.strftime() is now using the locale encoding, instead of
   UTF-8, if the wcsftime() function is not available.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 00:18:33 2011
From: python-checkins at python.org (ned.deily)
Date: Thu, 15 Dec 2011 00:18:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=234625=3A_Add_NEWS_entry=2E?=
Message-ID: <E1Ray5p-0001ky-17@dinsdale.python.org>

http://hg.python.org/cpython/rev/f6510cdf4ada
changeset:   73972:f6510cdf4ada
parent:      73969:ba832d42cd9e
parent:      73971:f1fe411bfd6b
user:        Ned Deily <nad at acm.org>
date:        Wed Dec 14 15:14:52 2011 -0800
summary:
  Issue #4625: Add NEWS entry.

files:
  Misc/NEWS |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,6 +409,10 @@
 Library
 -------
 
+- Issue #4625: If IDLE cannot write to its recent file or breakpoint
+  files, display a message popup and continue rather than crash.
+ (original patch by Roger Serwy)
+
 - Issue #13449 sched.scheduler.run() method has a new "blocking" parameter which
   when set to False makes run() execute the scheduled events due to expire
   soonest (if any) and then return.  Patch by Giampaolo Rodol?.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 05:28:04 2011
From: python-checkins at python.org (meador.inge)
Date: Thu, 15 Dec 2011 05:28:04 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTkx?=
 =?utf8?q?=3A_import=5Fmodule_potentially_imports_a_module_twice=2E?=
Message-ID: <E1Rb2vM-0006di-BY@dinsdale.python.org>

http://hg.python.org/cpython/rev/d2504d30f259
changeset:   73973:d2504d30f259
branch:      3.2
parent:      73971:f1fe411bfd6b
user:        Meador Inge <meadori at gmail.com>
date:        Wed Dec 14 22:23:46 2011 -0600
summary:
  Issue #13591: import_module potentially imports a module twice.

files:
  Lib/importlib/_bootstrap.py    |   4 +++-
  Lib/importlib/test/test_api.py |  17 +++++++++++++++++
  Lib/importlib/test/util.py     |   7 ++++++-
  Misc/NEWS                      |   3 +++
  4 files changed, 29 insertions(+), 2 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -816,7 +816,9 @@
         for finder in meta_path:
             loader = finder.find_module(name, path)
             if loader is not None:
-                loader.load_module(name)
+                # The parent import may have already imported this module.
+                if name not in sys.modules:
+                    loader.load_module(name)
                 break
         else:
             raise ImportError(_ERR_MSG.format(name))
diff --git a/Lib/importlib/test/test_api.py b/Lib/importlib/test/test_api.py
--- a/Lib/importlib/test/test_api.py
+++ b/Lib/importlib/test/test_api.py
@@ -67,6 +67,23 @@
             importlib.import_module('.support')
 
 
+    def test_loaded_once(self):
+        # Issue #13591: Modules should only be loaded once when
+        # initializing the parent package attempts to import the
+        # module currently being imported.
+        b_load_count = 0
+        def load_a():
+            importlib.import_module('a.b')
+        def load_b():
+            nonlocal b_load_count
+            b_load_count += 1
+        code = {'a': load_a, 'a.b': load_b}
+        modules = ['a.__init__', 'a.b']
+        with util.mock_modules(*modules, module_code=code) as mock:
+            with util.import_state(meta_path=[mock]):
+                importlib.import_module('a.b')
+        self.assertEqual(b_load_count, 1)
+
 def test_main():
     from test.support import run_unittest
     run_unittest(ImportModuleTests)
diff --git a/Lib/importlib/test/util.py b/Lib/importlib/test/util.py
--- a/Lib/importlib/test/util.py
+++ b/Lib/importlib/test/util.py
@@ -84,8 +84,9 @@
 
     """A mock importer/loader."""
 
-    def __init__(self, *names):
+    def __init__(self, *names, module_code={}):
         self.modules = {}
+        self.module_code = {}
         for name in names:
             if not name.endswith('.__init__'):
                 import_name = name
@@ -105,6 +106,8 @@
             if import_name != name:
                 module.__path__ = ['<mock __path__>']
             self.modules[import_name] = module
+            if import_name in module_code:
+                self.module_code[import_name] = module_code[import_name]
 
     def __getitem__(self, name):
         return self.modules[name]
@@ -120,6 +123,8 @@
             raise ImportError
         else:
             sys.modules[fullname] = self.modules[fullname]
+            if fullname in self.module_code:
+                self.module_code[fullname]()
             return self.modules[fullname]
 
     def __enter__(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1868,6 +1868,9 @@
 Library
 -------
 
+- Issue #13591: A bug in importlib has been fixed that caused import_module
+  to load a module twice.
+
 - logging: added "handler of last resort". See http://bit.ly/last-resort-handler
 
 - test.support: Added TestHandler and Matcher classes for better support of

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 05:28:05 2011
From: python-checkins at python.org (meador.inge)
Date: Thu, 15 Dec 2011 05:28:05 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313591=3A_import=5Fmodule_potentially_imports_a_modu?=
 =?utf8?q?le_twice=2E?=
Message-ID: <E1Rb2vN-0006dr-4i@dinsdale.python.org>

http://hg.python.org/cpython/rev/e8fb61a0a2d7
changeset:   73974:e8fb61a0a2d7
parent:      73972:f6510cdf4ada
parent:      73973:d2504d30f259
user:        Meador Inge <meadori at gmail.com>
date:        Wed Dec 14 22:27:28 2011 -0600
summary:
  Issue #13591: import_module potentially imports a module twice.

files:
  Lib/importlib/_bootstrap.py    |   4 +++-
  Lib/importlib/test/test_api.py |  17 +++++++++++++++++
  Lib/importlib/test/util.py     |   7 ++++++-
  Misc/NEWS                      |   3 +++
  4 files changed, 29 insertions(+), 2 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -837,7 +837,9 @@
         for finder in meta_path:
             loader = finder.find_module(name, path)
             if loader is not None:
-                loader.load_module(name)
+                # The parent import may have already imported this module.
+                if name not in sys.modules:
+                    loader.load_module(name)
                 break
         else:
             raise ImportError(_ERR_MSG.format(name))
diff --git a/Lib/importlib/test/test_api.py b/Lib/importlib/test/test_api.py
--- a/Lib/importlib/test/test_api.py
+++ b/Lib/importlib/test/test_api.py
@@ -67,6 +67,23 @@
             importlib.import_module('.support')
 
 
+    def test_loaded_once(self):
+        # Issue #13591: Modules should only be loaded once when
+        # initializing the parent package attempts to import the
+        # module currently being imported.
+        b_load_count = 0
+        def load_a():
+            importlib.import_module('a.b')
+        def load_b():
+            nonlocal b_load_count
+            b_load_count += 1
+        code = {'a': load_a, 'a.b': load_b}
+        modules = ['a.__init__', 'a.b']
+        with util.mock_modules(*modules, module_code=code) as mock:
+            with util.import_state(meta_path=[mock]):
+                importlib.import_module('a.b')
+        self.assertEqual(b_load_count, 1)
+
 def test_main():
     from test.support import run_unittest
     run_unittest(ImportModuleTests)
diff --git a/Lib/importlib/test/util.py b/Lib/importlib/test/util.py
--- a/Lib/importlib/test/util.py
+++ b/Lib/importlib/test/util.py
@@ -84,8 +84,9 @@
 
     """A mock importer/loader."""
 
-    def __init__(self, *names):
+    def __init__(self, *names, module_code={}):
         self.modules = {}
+        self.module_code = {}
         for name in names:
             if not name.endswith('.__init__'):
                 import_name = name
@@ -105,6 +106,8 @@
             if import_name != name:
                 module.__path__ = ['<mock __path__>']
             self.modules[import_name] = module
+            if import_name in module_code:
+                self.module_code[import_name] = module_code[import_name]
 
     def __getitem__(self, name):
         return self.modules[name]
@@ -120,6 +123,8 @@
             raise ImportError
         else:
             sys.modules[fullname] = self.modules[fullname]
+            if fullname in self.module_code:
+                self.module_code[fullname]()
             return self.modules[fullname]
 
     def __enter__(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -2548,6 +2548,9 @@
 Library
 -------
 
+- Issue #13591: A bug in importlib has been fixed that caused import_module
+  to load a module twice.
+
 - logging: added "handler of last resort". See http://bit.ly/last-resort-handler
 
 - test.support: Added TestHandler and Matcher classes for better support of

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Dec 15 05:31:37 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 15 Dec 2011 05:31:37 +0100
Subject: [Python-checkins] Daily reference leaks (f6510cdf4ada): sum=0
Message-ID: <E1Rb2yn-0004kM-He@ap.vmr.nerim.net>

results for f6510cdf4ada on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogGobnbL', '-x']

From python-checkins at python.org  Thu Dec 15 05:41:11 2011
From: python-checkins at python.org (meador.inge)
Date: Thu, 15 Dec 2011 05:41:11 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTkx?=
 =?utf8?q?=3A_Moving_the_NEWS_line_to_the_right_release=2E?=
Message-ID: <E1Rb383-0008Bc-O3@dinsdale.python.org>

http://hg.python.org/cpython/rev/541f215a31f7
changeset:   73975:541f215a31f7
branch:      3.2
parent:      73973:d2504d30f259
user:        Meador Inge <meadori at gmail.com>
date:        Wed Dec 14 22:37:48 2011 -0600
summary:
  Issue #13591: Moving the NEWS line to the right release.

files:
  Misc/NEWS |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -93,6 +93,9 @@
 Library
 -------
 
+- Issue #13591: A bug in importlib has been fixed that caused import_module
+  to load a module twice.
+
 - Issue #4625: If IDLE cannot write to its recent file or breakpoint
   files, display a message popup and continue rather than crash.
   (original patch by Roger Serwy)
@@ -1868,9 +1871,6 @@
 Library
 -------
 
-- Issue #13591: A bug in importlib has been fixed that caused import_module
-  to load a module twice.
-
 - logging: added "handler of last resort". See http://bit.ly/last-resort-handler
 
 - test.support: Added TestHandler and Matcher classes for better support of

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 05:41:12 2011
From: python-checkins at python.org (meador.inge)
Date: Thu, 15 Dec 2011 05:41:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313591=3A_Moving_the_NEWS_line_to_the_right_release?=
 =?utf8?q?=2E?=
Message-ID: <E1Rb384-0008Bi-G4@dinsdale.python.org>

http://hg.python.org/cpython/rev/92e94fd303d4
changeset:   73976:92e94fd303d4
parent:      73974:e8fb61a0a2d7
parent:      73975:541f215a31f7
user:        Meador Inge <meadori at gmail.com>
date:        Wed Dec 14 22:39:28 2011 -0600
summary:
  Issue #13591: Moving the NEWS line to the right release.

files:
  Misc/NEWS |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,6 +409,9 @@
 Library
 -------
 
+- Issue #13591: A bug in importlib has been fixed that caused import_module
+  to load a module twice.
+
 - Issue #4625: If IDLE cannot write to its recent file or breakpoint
   files, display a message popup and continue rather than crash.
  (original patch by Roger Serwy)
@@ -2548,9 +2551,6 @@
 Library
 -------
 
-- Issue #13591: A bug in importlib has been fixed that caused import_module
-  to load a module twice.
-
 - logging: added "handler of last resort". See http://bit.ly/last-resort-handler
 
 - test.support: Added TestHandler and Matcher classes for better support of

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 05:55:07 2011
From: python-checkins at python.org (meador.inge)
Date: Thu, 15 Dec 2011 05:55:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313593=3A_updating_?=
 =?utf8?q?the_importlib_utility_decorators_for_=5F=5Fqualname=5F=5F=2E?=
Message-ID: <E1Rb3LX-0001PB-7g@dinsdale.python.org>

http://hg.python.org/cpython/rev/54a77c556d9a
changeset:   73977:54a77c556d9a
user:        Meador Inge <meadori at gmail.com>
date:        Wed Dec 14 22:53:13 2011 -0600
summary:
  Issue #13593: updating the importlib utility decorators for __qualname__.

files:
  Lib/importlib/_bootstrap.py     |   2 +-
  Lib/importlib/test/test_util.py |  10 ++++++++++
  2 files changed, 11 insertions(+), 1 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -111,7 +111,7 @@
 
 def _wrap(new, old):
     """Simple substitute for functools.wraps."""
-    for replace in ['__module__', '__name__', '__doc__']:
+    for replace in ['__module__', '__name__', '__qualname__', '__doc__']:
         setattr(new, replace, getattr(old, replace))
     new.__dict__.update(old.__dict__)
 
diff --git a/Lib/importlib/test/test_util.py b/Lib/importlib/test/test_util.py
--- a/Lib/importlib/test/test_util.py
+++ b/Lib/importlib/test/test_util.py
@@ -59,6 +59,11 @@
             self.raise_exception(name)
             self.assertIs(module, sys.modules[name])
 
+    def test_decorator_attrs(self):
+        def fxn(self, module): pass
+        wrapped = util.module_for_loader(fxn)
+        self.assertEqual(wrapped.__name__, fxn.__name__)
+        self.assertEqual(wrapped.__qualname__, fxn.__qualname__)
 
 class SetPackageTests(unittest.TestCase):
 
@@ -108,6 +113,11 @@
             module.__package__ = value
             self.verify(module, value)
 
+    def test_decorator_attrs(self):
+        def fxn(module): pass
+        wrapped = util.set_package(fxn)
+        self.assertEqual(wrapped.__name__, fxn.__name__)
+        self.assertEqual(wrapped.__qualname__, fxn.__qualname__)
 
 def test_main():
     from test import support

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 13:17:35 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 13:17:35 +0100
Subject: [Python-checkins] =?utf8?q?benchmarks=3A_Add_a_logging_benchmark?=
Message-ID: <E1RbAFj-00059K-Tx@dinsdale.python.org>

http://hg.python.org/benchmarks/rev/135d44d0c02a
changeset:   152:135d44d0c02a
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 13:17:14 2011 +0100
summary:
  Add a logging benchmark

files:
  perf.py                   |   49 ++++++++++-
  performance/bm_logging.py |  119 ++++++++++++++++++++++++++
  2 files changed, 167 insertions(+), 1 deletions(-)


diff --git a/perf.py b/perf.py
--- a/perf.py
+++ b/perf.py
@@ -1597,6 +1597,52 @@
 def BM_NQueens(*args, **kwargs):
     return SimpleBenchmark(MeasureNQueens, *args, **kwargs)
 
+def MeasureLogging(python, options, extra_args):
+    """Test the performance of Python's logging module.
+
+    Args:
+        python: prefix of a command line for the Python binary.
+        options: optparse.Values instance.
+        extra_args: list of arguments to append to the command line.
+
+    Returns:
+        RawData instance.
+    """
+    bm_path = Relative("performance/bm_logging.py")
+    return MeasureGeneric(python, options, bm_path, extra_args=extra_args)
+
+
+def _LoggingBenchmark(base_python, changed_python, options, extra_args):
+    """Test the performance of Python's logging module.
+
+    Args:
+        base_python: prefix of a command line for the reference
+                Python binary.
+        changed_python: prefix of a command line for the
+                experimental Python binary.
+        options: optparse.Values instance.
+        extra_args: list of arguments to append to the command line.
+
+    Returns:
+        Summary of whether the experiemental Python is better/worse than the
+        baseline.
+    """
+    return SimpleBenchmark(MeasureLogging,
+                           base_python, changed_python, options, extra_args)
+
+
+def BM_Silent_Logging(base_python, changed_python, options):
+    args = ["no_output"]
+    return _LoggingBenchmark(base_python, changed_python, options, args)
+
+def BM_Simple_Logging(base_python, changed_python, options):
+    args = ["simple_output"]
+    return _LoggingBenchmark(base_python, changed_python, options, args)
+
+def BM_Formatted_Logging(base_python, changed_python, options):
+    args = ["formatted_output"]
+    return _LoggingBenchmark(base_python, changed_python, options, args)
+
 
 def _StartupPython(command, mem_usage, track_memory, inherit_env):
     startup_env = BuildEnv(inherit_env=inherit_env)
@@ -1935,11 +1981,12 @@
                           "call_method_unknown"],
                 "math": ["nbody", "float", "pidigits"],
                 "template" : ["slowspitfire", "django", "mako"],
+                "logging": ["silent_logging", "simple_logging", "formatted_logging"],
                 # Benchmarks natively 2.x- and 3.x-compatible
                 "2n3": ["calls", "math", "fastpickle", "fastunpickle",
                         "json_dump", "json_load", "regex", "threading",
                         "nqueens", "unpack_sequence", "richards",
-                        "normal_startup", "startup_nosite"],
+                        "logging", "normal_startup", "startup_nosite"],
                 # After 2to3-conversion
                 "py3k": ["2to3", "2n3", "mako"]
                }
diff --git a/performance/bm_logging.py b/performance/bm_logging.py
new file mode 100644
--- /dev/null
+++ b/performance/bm_logging.py
@@ -0,0 +1,119 @@
+#!/usr/bin/env python
+
+"""Script for testing the performance of logging simple messages.
+"""
+
+# Python imports
+import io
+import logging
+import optparse
+import sys
+import time
+
+# Local imports
+import util
+from compat import xrange
+
+# A simple format for parametered logging
+FORMAT = 'important: %s'
+MESSAGE = 'some important information to be logged'
+
+
+def test_no_output(iterations, logger):
+    times = []
+    m = MESSAGE
+    for _ in xrange(iterations):
+        t0 = time.time()
+        for _ in xrange(10000):
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+            logger.debug(m)
+        t1 = time.time()
+        times.append(t1 - t0)
+    return times
+
+
+def test_simple_output(iterations, logger):
+    times = []
+    m = MESSAGE
+    for _ in xrange(iterations):
+        t0 = time.time()
+        for _ in xrange(1000):
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+            logger.warn(m)
+        t1 = time.time()
+        times.append(t1 - t0)
+    return times
+
+
+def test_formatted_output(iterations, logger):
+    times = []
+    f = FORMAT
+    m = MESSAGE
+    for _ in xrange(iterations):
+        t0 = time.time()
+        for _ in xrange(1000):
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+            logger.warn(f, m)
+        t1 = time.time()
+        times.append(t1 - t0)
+    return times
+
+
+if __name__ == "__main__":
+    parser = optparse.OptionParser(
+        usage="%prog [no_output|simple_output|formatted_output] [options]",
+        description=("Test the performance of logging."))
+    util.add_standard_options_to(parser)
+    options, args = parser.parse_args()
+
+    benchmarks = ["no_output", "simple_output", "formatted_output"]
+    for bench_name in benchmarks:
+        if bench_name in args:
+            benchmark = globals()["test_" + bench_name]
+            break
+    else:
+        raise RuntimeError("Need to specify one of %s" % benchmarks)
+
+    logger = logging.getLogger("benchlogger")
+    logger.propagate = False
+    # NOTE: StringIO performance will impact the results...
+    if sys.version_info >= (3,):
+        sio = io.StringIO()
+    else:
+        sio = io.BytesIO()
+    handler = logging.StreamHandler(stream=sio)
+    logger = logging.getLogger("benchlogger")
+    logger.propagate = False
+    logger.addHandler(handler)
+    logger.setLevel(logging.WARNING)
+
+    util.run_benchmark(options, options.num_runs, benchmark, logger)
+
+    if benchmark is not test_no_output:
+        assert len(sio.getvalue()) > 0
+    else:
+        assert len(sio.getvalue()) == 0

-- 
Repository URL: http://hg.python.org/benchmarks

From python-checkins at python.org  Thu Dec 15 13:57:48 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 13:57:48 +0100
Subject: [Python-checkins] =?utf8?q?benchmarks=3A_Duplicate_lines?=
Message-ID: <E1RbAse-0001gb-HB@dinsdale.python.org>

http://hg.python.org/benchmarks/rev/eec31e939b64
changeset:   153:eec31e939b64
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 13:42:10 2011 +0100
summary:
  Duplicate lines

files:
  performance/bm_logging.py |  2 --
  1 files changed, 0 insertions(+), 2 deletions(-)


diff --git a/performance/bm_logging.py b/performance/bm_logging.py
--- a/performance/bm_logging.py
+++ b/performance/bm_logging.py
@@ -98,8 +98,6 @@
     else:
         raise RuntimeError("Need to specify one of %s" % benchmarks)
 
-    logger = logging.getLogger("benchlogger")
-    logger.propagate = False
     # NOTE: StringIO performance will impact the results...
     if sys.version_info >= (3,):
         sio = io.StringIO()

-- 
Repository URL: http://hg.python.org/benchmarks

From python-checkins at python.org  Thu Dec 15 14:21:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 14:21:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_the_fix_for?=
 =?utf8?q?_issue_=2312149=3A_it_was_incorrect=2C_although_it_had_the_side?=
Message-ID: <E1RbBFj-0004nC-Bt@dinsdale.python.org>

http://hg.python.org/cpython/rev/b196bcd7c34f
changeset:   73978:b196bcd7c34f
branch:      3.2
parent:      73975:541f215a31f7
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 14:15:31 2011 +0100
summary:
  Fix the fix for issue #12149: it was incorrect, although it had the side
effect of appearing to resolve the issue.  Thanks to Mark Shannon for
noticing.

files:
  Misc/NEWS            |   4 ++++
  Objects/typeobject.c |  18 ++++++++++--------
  2 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Fix the fix for issue #12149: it was incorrect, although it had the side
+  effect of appearing to resolve the issue.  Thanks to Mark Shannon for
+  noticing.
+
 - Issue #13505: Pickle bytes objects in a way that is compatible with
   Python 2 when using protocols <= 2.
 
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -967,8 +967,6 @@
     assert(basedealloc);
     basedealloc(self);
 
-    PyType_Modified(type);
-
     /* Can't reference self beyond this point */
     Py_DECREF(type);
 
@@ -2645,15 +2643,16 @@
        for heaptypes. */
     assert(type->tp_flags & Py_TPFLAGS_HEAPTYPE);
 
-    /* The only field we need to clear is tp_mro, which is part of a
-       hard cycle (its first element is the class itself) that won't
-       be broken otherwise (it's a tuple and tuples don't have a
+    /* We need to invalidate the method cache carefully before clearing
+       the dict, so that other objects caught in a reference cycle
+       don't start calling destroyed methods.
+
+       Otherwise, the only field we need to clear is tp_mro, which is
+       part of a hard cycle (its first element is the class itself) that
+       won't be broken otherwise (it's a tuple and tuples don't have a
        tp_clear handler).  None of the other fields need to be
        cleared, and here's why:
 
-       tp_dict:
-           It is a dict, so the collector will call its tp_clear.
-
        tp_cache:
            Not used; if it were, it would be a dict.
 
@@ -2670,6 +2669,9 @@
            A tuple of strings can't be part of a cycle.
     */
 
+    PyType_Modified(type);
+    if (type->tp_dict)
+        PyDict_Clear(type->tp_dict);
     Py_CLEAR(type->tp_mro);
 
     return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 14:21:40 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 14:21:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Fix_the_fix_for_issue_=2312149=3A_it_was_incorrect=2C_althou?=
 =?utf8?q?gh_it_had_the_side?=
Message-ID: <E1RbBFk-0004nG-5n@dinsdale.python.org>

http://hg.python.org/cpython/rev/195bfd4c3ea1
changeset:   73979:195bfd4c3ea1
parent:      73977:54a77c556d9a
parent:      73978:b196bcd7c34f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 14:17:36 2011 +0100
summary:
  Fix the fix for issue #12149: it was incorrect, although it had the side
effect of appearing to resolve the issue.  Thanks to Mark Shannon for
noticing.

files:
  Misc/NEWS            |   4 ++++
  Objects/typeobject.c |  18 ++++++++++--------
  2 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Fix the fix for issue #12149: it was incorrect, although it had the side
+  effect of appearing to resolve the issue.  Thanks to Mark Shannon for
+  noticing.
+
 - Issue #13505: Pickle bytes objects in a way that is compatible with
   Python 2 when using protocols <= 2.
 
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -999,8 +999,6 @@
     assert(basedealloc);
     basedealloc(self);
 
-    PyType_Modified(type);
-
     /* Can't reference self beyond this point */
     Py_DECREF(type);
 
@@ -2776,15 +2774,16 @@
        for heaptypes. */
     assert(type->tp_flags & Py_TPFLAGS_HEAPTYPE);
 
-    /* The only field we need to clear is tp_mro, which is part of a
-       hard cycle (its first element is the class itself) that won't
-       be broken otherwise (it's a tuple and tuples don't have a
+    /* We need to invalidate the method cache carefully before clearing
+       the dict, so that other objects caught in a reference cycle
+       don't start calling destroyed methods.
+
+       Otherwise, the only field we need to clear is tp_mro, which is
+       part of a hard cycle (its first element is the class itself) that
+       won't be broken otherwise (it's a tuple and tuples don't have a
        tp_clear handler).  None of the other fields need to be
        cleared, and here's why:
 
-       tp_dict:
-           It is a dict, so the collector will call its tp_clear.
-
        tp_cache:
            Not used; if it were, it would be a dict.
 
@@ -2801,6 +2800,9 @@
            A tuple of strings can't be part of a cycle.
     */
 
+    PyType_Modified(type);
+    if (type->tp_dict)
+        PyDict_Clear(type->tp_dict);
     Py_CLEAR(type->tp_mro);
 
     return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 14:21:42 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 14:21:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Fix_the_fix_for?=
 =?utf8?q?_issue_=2312149=3A_it_was_incorrect=2C_although_it_had_the_side?=
Message-ID: <E1RbBFm-0004nS-6V@dinsdale.python.org>

http://hg.python.org/cpython/rev/da1d7f8cd487
changeset:   73980:da1d7f8cd487
branch:      2.7
parent:      73970:f6ca22bbf138
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 14:15:31 2011 +0100
summary:
  Fix the fix for issue #12149: it was incorrect, although it had the side
effect of appearing to resolve the issue.  Thanks to Mark Shannon for
noticing.

files:
  Misc/NEWS            |   4 ++++
  Objects/typeobject.c |  18 ++++++++++--------
  2 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,10 @@
 Core and Builtins
 -----------------
 
+- Fix the fix for issue #12149: it was incorrect, although it had the side
+  effect of appearing to resolve the issue.  Thanks to Mark Shannon for
+  noticing.
+
 - Issue #13546: Fixed an overflow issue that could crash the intepreter when
   calling sys.setrecursionlimit((1<<31)-1).
 
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1013,8 +1013,6 @@
     assert(basedealloc);
     basedealloc(self);
 
-    PyType_Modified(type);
-
     /* Can't reference self beyond this point */
     Py_DECREF(type);
 
@@ -2707,15 +2705,16 @@
        for heaptypes. */
     assert(type->tp_flags & Py_TPFLAGS_HEAPTYPE);
 
-    /* The only field we need to clear is tp_mro, which is part of a
-       hard cycle (its first element is the class itself) that won't
-       be broken otherwise (it's a tuple and tuples don't have a
+    /* We need to invalidate the method cache carefully before clearing
+       the dict, so that other objects caught in a reference cycle
+       don't start calling destroyed methods.
+
+       Otherwise, the only field we need to clear is tp_mro, which is
+       part of a hard cycle (its first element is the class itself) that
+       won't be broken otherwise (it's a tuple and tuples don't have a
        tp_clear handler).  None of the other fields need to be
        cleared, and here's why:
 
-       tp_dict:
-           It is a dict, so the collector will call its tp_clear.
-
        tp_cache:
            Not used; if it were, it would be a dict.
 
@@ -2732,6 +2731,9 @@
            A tuple of strings can't be part of a cycle.
     */
 
+    PyType_Modified(type);
+    if (type->tp_dict)
+        PyDict_Clear(type->tp_dict);
     Py_CLEAR(type->tp_mro);
 
     return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 14:33:09 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 14:33:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_OSError=2E=5F=5Finit=5F?=
 =?utf8?q?=5F_and_OSError=2E=5F=5Fnew=5F=5F_so_that_each_of_them_can_be?=
Message-ID: <E1RbBQr-00068Q-3N@dinsdale.python.org>

http://hg.python.org/cpython/rev/d22c99e77768
changeset:   73981:d22c99e77768
parent:      73979:195bfd4c3ea1
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 14:31:28 2011 +0100
summary:
  Fix OSError.__init__ and OSError.__new__ so that each of them can be
overriden and take additional arguments (followup to issue #12555).

files:
  Lib/test/test_pep3151.py |   60 ++++++-
  Misc/NEWS                |    3 +
  Objects/exceptions.c     |  216 +++++++++++++++++++-------
  3 files changed, 217 insertions(+), 62 deletions(-)


diff --git a/Lib/test/test_pep3151.py b/Lib/test/test_pep3151.py
--- a/Lib/test/test_pep3151.py
+++ b/Lib/test/test_pep3151.py
@@ -12,6 +12,23 @@
 class SubOSError(OSError):
     pass
 
+class SubOSErrorWithInit(OSError):
+    def __init__(self, message, bar):
+        self.bar = bar
+        super().__init__(message)
+
+class SubOSErrorWithNew(OSError):
+    def __new__(cls, message, baz):
+        self = super().__new__(cls, message)
+        self.baz = baz
+        return self
+
+class SubOSErrorCombinedInitFirst(SubOSErrorWithInit, SubOSErrorWithNew):
+    pass
+
+class SubOSErrorCombinedNewFirst(SubOSErrorWithNew, SubOSErrorWithInit):
+    pass
+
 
 class HierarchyTest(unittest.TestCase):
 
@@ -74,11 +91,6 @@
             e = OSError(errcode, "Some message")
             self.assertIs(type(e), OSError)
 
-    def test_OSError_subclass_mapping(self):
-        # When constructing an OSError subclass, errno mapping isn't done
-        e = SubOSError(EEXIST, "Bad file descriptor")
-        self.assertIs(type(e), SubOSError)
-
     def test_try_except(self):
         filename = "some_hopefully_non_existing_file"
 
@@ -144,6 +156,44 @@
     # XXX VMSError not tested
 
 
+class ExplicitSubclassingTest(unittest.TestCase):
+
+    def test_errno_mapping(self):
+        # When constructing an OSError subclass, errno mapping isn't done
+        e = SubOSError(EEXIST, "Bad file descriptor")
+        self.assertIs(type(e), SubOSError)
+
+    def test_init_overriden(self):
+        e = SubOSErrorWithInit("some message", "baz")
+        self.assertEqual(e.bar, "baz")
+        self.assertEqual(e.args, ("some message",))
+
+    def test_init_kwdargs(self):
+        e = SubOSErrorWithInit("some message", bar="baz")
+        self.assertEqual(e.bar, "baz")
+        self.assertEqual(e.args, ("some message",))
+
+    def test_new_overriden(self):
+        e = SubOSErrorWithNew("some message", "baz")
+        self.assertEqual(e.baz, "baz")
+        self.assertEqual(e.args, ("some message",))
+
+    def test_new_kwdargs(self):
+        e = SubOSErrorWithNew("some message", baz="baz")
+        self.assertEqual(e.baz, "baz")
+        self.assertEqual(e.args, ("some message",))
+
+    def test_init_new_overriden(self):
+        e = SubOSErrorCombinedInitFirst("some message", "baz")
+        self.assertEqual(e.bar, "baz")
+        self.assertEqual(e.baz, "baz")
+        self.assertEqual(e.args, ("some message",))
+        e = SubOSErrorCombinedNewFirst("some message", "baz")
+        self.assertEqual(e.bar, "baz")
+        self.assertEqual(e.baz, "baz")
+        self.assertEqual(e.args, ("some message",))
+
+
 def test_main():
     support.run_unittest(__name__)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Fix OSError.__init__ and OSError.__new__ so that each of them can be
+  overriden and take additional arguments (followup to issue #12555).
+
 - Fix the fix for issue #12149: it was incorrect, although it had the side
   effect of appearing to resolve the issue.  Thanks to Mark Shannon for
   noticing.
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -58,7 +58,7 @@
     if (!_PyArg_NoKeywords(Py_TYPE(self)->tp_name, kwds))
         return -1;
 
-    Py_DECREF(self->args);
+    Py_XDECREF(self->args);
     self->args = args;
     Py_INCREF(self->args);
 
@@ -587,37 +587,34 @@
  * when it was supplied.
  */
 
-static PyObject *
-OSError_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
+/* This function doesn't cleanup on error, the caller should */
+static int
+oserror_parse_args(PyObject **p_args,
+                   PyObject **myerrno, PyObject **strerror,
+                   PyObject **filename
+#ifdef MS_WINDOWS
+                   , PyObject **winerror
+#endif
+                  )
 {
-    PyOSErrorObject *self = NULL;
     Py_ssize_t nargs;
+    PyObject *args = *p_args;
 
-    PyObject *myerrno = NULL, *strerror = NULL, *filename = NULL;
-    PyObject *subslice = NULL;
-#ifdef MS_WINDOWS
-    PyObject *winerror = NULL;
-    long winerrcode = 0;
-#endif
-
-    if (!_PyArg_NoKeywords(type->tp_name, kwds))
-        return NULL;
-    Py_INCREF(args);
     nargs = PyTuple_GET_SIZE(args);
 
 #ifdef MS_WINDOWS
     if (nargs >= 2 && nargs <= 4) {
         if (!PyArg_UnpackTuple(args, "OSError", 2, 4,
-                               &myerrno, &strerror, &filename, &winerror))
-            goto error;
-        if (winerror && PyLong_Check(winerror)) {
-            long errcode;
+                               myerrno, strerror, filename, winerror))
+            return -1;
+        if (*winerror && PyLong_Check(*winerror)) {
+            long errcode, winerrcode;
             PyObject *newargs;
             Py_ssize_t i;
 
-            winerrcode = PyLong_AsLong(winerror);
+            winerrcode = PyLong_AsLong(*winerror);
             if (winerrcode == -1 && PyErr_Occurred())
-                goto error;
+                return -1;
             /* Set errno to the corresponding POSIX errno (overriding
                first argument).  Windows Socket error codes (>= 10000)
                have the same value as their POSIX counterparts.
@@ -626,59 +623,55 @@
                 errcode = winerror_to_errno(winerrcode);
             else
                 errcode = winerrcode;
-            myerrno = PyLong_FromLong(errcode);
-            if (!myerrno)
-                goto error;
+            *myerrno = PyLong_FromLong(errcode);
+            if (!*myerrno)
+                return -1;
             newargs = PyTuple_New(nargs);
             if (!newargs)
-                goto error;
-            PyTuple_SET_ITEM(newargs, 0, myerrno);
+                return -1;
+            PyTuple_SET_ITEM(newargs, 0, *myerrno);
             for (i = 1; i < nargs; i++) {
                 PyObject *val = PyTuple_GET_ITEM(args, i);
                 Py_INCREF(val);
                 PyTuple_SET_ITEM(newargs, i, val);
             }
             Py_DECREF(args);
-            args = newargs;
+            args = *p_args = newargs;
         }
     }
 #else
     if (nargs >= 2 && nargs <= 3) {
         if (!PyArg_UnpackTuple(args, "OSError", 2, 3,
-                               &myerrno, &strerror, &filename))
-            goto error;
+                               myerrno, strerror, filename))
+            return -1;
     }
 #endif
-    if (myerrno && PyLong_Check(myerrno) &&
-        errnomap && (PyObject *) type == PyExc_OSError) {
-        PyObject *newtype;
-        newtype = PyDict_GetItem(errnomap, myerrno);
-        if (newtype) {
-            assert(PyType_Check(newtype));
-            type = (PyTypeObject *) newtype;
-        }
-        else if (PyErr_Occurred())
-            goto error;
-    }
 
-    self = (PyOSErrorObject *) type->tp_alloc(type, 0);
-    if (!self)
-        goto error;
+    return 0;
+}
 
-    self->dict = NULL;
-    self->traceback = self->cause = self->context = NULL;
-    self->written = -1;
+static int
+oserror_init(PyOSErrorObject *self, PyObject **p_args,
+             PyObject *myerrno, PyObject *strerror,
+             PyObject *filename
+#ifdef MS_WINDOWS
+             , PyObject *winerror
+#endif
+             )
+{
+    PyObject *args = *p_args;
+    Py_ssize_t nargs = PyTuple_GET_SIZE(args);
 
     /* self->filename will remain Py_None otherwise */
     if (filename && filename != Py_None) {
-        if ((PyObject *) type == PyExc_BlockingIOError &&
+        if (Py_TYPE(self) == (PyTypeObject *) PyExc_BlockingIOError &&
             PyNumber_Check(filename)) {
             /* BlockingIOError's 3rd argument can be the number of
              * characters written.
              */
             self->written = PyNumber_AsSsize_t(filename, PyExc_ValueError);
             if (self->written == -1 && PyErr_Occurred())
-                goto error;
+                return -1;
         }
         else {
             Py_INCREF(filename);
@@ -687,20 +680,15 @@
             if (nargs >= 2 && nargs <= 3) {
                 /* filename is removed from the args tuple (for compatibility
                    purposes, see test_exceptions.py) */
-                subslice = PyTuple_GetSlice(args, 0, 2);
+                PyObject *subslice = PyTuple_GetSlice(args, 0, 2);
                 if (!subslice)
-                    goto error;
+                    return -1;
 
                 Py_DECREF(args);  /* replacing args */
-                args = subslice;
+                *p_args = args = subslice;
             }
         }
     }
-
-    /* Steals the reference to args */
-    self->args = args;
-    args = NULL;
-
     Py_XINCREF(myerrno);
     self->myerrno = myerrno;
 
@@ -712,6 +700,90 @@
     self->winerror = winerror;
 #endif
 
+    /* Steals the reference to args */
+    self->args = args;
+    args = NULL;
+
+    return 0;
+}
+
+static PyObject *
+OSError_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
+static int
+OSError_init(PyOSErrorObject *self, PyObject *args, PyObject *kwds);
+
+static int
+oserror_use_init(PyTypeObject *type)
+{
+    /* When __init__ is defined in a OSError subclass, we want any
+       extraneous argument to __new__ to be ignored.  The only reasonable
+       solution, given __new__ takes a variable number of arguments,
+       is to defer arg parsing and initialization to __init__.
+
+       But when __new__ is overriden as well, it should call our __new__
+       with the right arguments.
+
+       (see http://bugs.python.org/issue12555#msg148829 )
+    */
+    if (type->tp_init != (initproc) OSError_init &&
+        type->tp_new == (newfunc) OSError_new) {
+        assert((PyObject *) type != PyExc_OSError);
+        return 1;
+    }
+    return 0;
+}
+
+static PyObject *
+OSError_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
+{
+    PyOSErrorObject *self = NULL;
+    PyObject *myerrno = NULL, *strerror = NULL, *filename = NULL;
+#ifdef MS_WINDOWS
+    PyObject *winerror = NULL;
+#endif
+
+    if (!oserror_use_init(type)) {
+        if (!_PyArg_NoKeywords(type->tp_name, kwds))
+            return NULL;
+
+        Py_INCREF(args);
+        if (oserror_parse_args(&args, &myerrno, &strerror, &filename
+#ifdef MS_WINDOWS
+                               , &winerror
+#endif
+            ))
+            goto error;
+
+        if (myerrno && PyLong_Check(myerrno) &&
+            errnomap && (PyObject *) type == PyExc_OSError) {
+            PyObject *newtype;
+            newtype = PyDict_GetItem(errnomap, myerrno);
+            if (newtype) {
+                assert(PyType_Check(newtype));
+                type = (PyTypeObject *) newtype;
+            }
+            else if (PyErr_Occurred())
+                goto error;
+        }
+    }
+
+    self = (PyOSErrorObject *) type->tp_alloc(type, 0);
+    if (!self)
+        goto error;
+
+    self->dict = NULL;
+    self->traceback = self->cause = self->context = NULL;
+    self->written = -1;
+
+    if (!oserror_use_init(type)) {
+        if (oserror_init(self, &args, myerrno, strerror, filename
+#ifdef MS_WINDOWS
+                         , winerror
+#endif
+            ))
+            goto error;
+    }
+
     return (PyObject *) self;
 
 error:
@@ -721,10 +793,40 @@
 }
 
 static int
-OSError_init(PySyntaxErrorObject *self, PyObject *args, PyObject *kwds)
+OSError_init(PyOSErrorObject *self, PyObject *args, PyObject *kwds)
 {
-    /* Everything already done in OSError_new */
+    PyObject *myerrno = NULL, *strerror = NULL, *filename = NULL;
+#ifdef MS_WINDOWS
+    PyObject *winerror = NULL;
+#endif
+
+    if (!oserror_use_init(Py_TYPE(self)))
+        /* Everything already done in OSError_new */
+        return 0;
+
+    if (!_PyArg_NoKeywords(Py_TYPE(self)->tp_name, kwds))
+        return -1;
+
+    Py_INCREF(args);
+    if (oserror_parse_args(&args, &myerrno, &strerror, &filename
+#ifdef MS_WINDOWS
+                           , &winerror
+#endif
+        ))
+        goto error;
+
+    if (oserror_init(self, &args, myerrno, strerror, filename
+#ifdef MS_WINDOWS
+                     , winerror
+#endif
+        ))
+        goto error;
+
     return 0;
+
+error:
+    Py_XDECREF(args);
+    return -1;
 }
 
 static int

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 16:26:31 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 16:26:31 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTk3?=
 =?utf8?q?=3A_Improve_documentation_of_standard_streams=2E?=
Message-ID: <E1RbDCZ-00015T-2U@dinsdale.python.org>

http://hg.python.org/cpython/rev/313fa7ea6c79
changeset:   73982:313fa7ea6c79
branch:      3.2
parent:      73978:b196bcd7c34f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 16:25:34 2011 +0100
summary:
  Issue #13597: Improve documentation of standard streams.

files:
  Doc/library/sys.rst |  47 ++++++++++++++++++++------------
  1 files changed, 29 insertions(+), 18 deletions(-)


diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -934,31 +934,42 @@
           stdout
           stderr
 
-   :term:`File objects <file object>` corresponding to the interpreter's standard
-   input, output and error streams.  ``stdin`` is used for all interpreter input
-   except for scripts but including calls to :func:`input`.  ``stdout`` is used
-   for the output of :func:`print` and :term:`expression` statements and for the
-   prompts of :func:`input`. The interpreter's own prompts
-   and (almost all of) its error messages go to ``stderr``.  ``stdout`` and
-   ``stderr`` needn't be built-in file objects: any object is acceptable as long
-   as it has a :meth:`write` method that takes a string argument.  (Changing these
-   objects doesn't affect the standard I/O streams of processes executed by
-   :func:`os.popen`, :func:`os.system` or the :func:`exec\*` family of functions in
-   the :mod:`os` module.)
+   :term:`File objects <file object>` used by the interpreter for standard
+   input, output and errors:
 
-   The standard streams are in text mode by default.  To write or read binary
-   data to these, use the underlying binary buffer.  For example, to write bytes
-   to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``.  Using
-   :meth:`io.TextIOBase.detach` streams can be made binary by default.  This
+   * ``stdin`` is used for all interactive input (including calls to
+     :func:`input`);
+   * ``stdout`` is used for the output of :func:`print` and :term:`expression`
+     statements and for the prompts of :func:`input`;
+   * The interpreter's own prompts and its error messages go to ``stderr``.
+
+   By default, these streams are regular text streams as returned by the
+   :func:`open` function.  Their parameters are chosen as follows:
+
+   * The character encoding is platform-dependent.  Under Windows, if the stream
+     is interactive (that is, if its :meth:`isatty` method returns True), the
+     console codepage is used, otherwise the ANSI code page.  Under other
+     platforms, the locale encoding is used (see :meth:`locale.getpreferredencoding`).
+
+     Under all platforms though, you can override this value by setting the
+     :envvar:`PYTHONIOENCODING` environment variable.
+
+   * When interactive, standard streams are line-buffered.  Otherwise, they
+     are block-buffered like regular text files.  You can override this
+     value with the :option:`-u` command-line option.
+
+   To write or read binary data from/to the standard streams, use the
+   underlying binary :data:`~io.TextIOBase.buffer`.  For example, to write
+   bytes to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``.  Using
+   :meth:`io.TextIOBase.detach`, streams can be made binary by default.  This
    function sets :data:`stdin` and :data:`stdout` to binary::
 
       def make_streams_binary():
           sys.stdin = sys.stdin.detach()
           sys.stdout = sys.stdout.detach()
 
-   Note that the streams can be replaced with objects (like
-   :class:`io.StringIO`) that do not support the
-   :attr:`~io.BufferedIOBase.buffer` attribute or the
+   Note that the streams may be replaced with objects (like :class:`io.StringIO`)
+   that do not support the :attr:`~io.BufferedIOBase.buffer` attribute or the
    :meth:`~io.BufferedIOBase.detach` method and can raise :exc:`AttributeError`
    or :exc:`io.UnsupportedOperation`.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 16:26:31 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 15 Dec 2011 16:26:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313597=3A_Improve_documentation_of_standard_streams?=
 =?utf8?q?=2E?=
Message-ID: <E1RbDCZ-00015z-QG@dinsdale.python.org>

http://hg.python.org/cpython/rev/7343730185a3
changeset:   73983:7343730185a3
parent:      73981:d22c99e77768
parent:      73982:313fa7ea6c79
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 15 16:26:03 2011 +0100
summary:
  Issue #13597: Improve documentation of standard streams.

files:
  Doc/library/sys.rst |  47 ++++++++++++++++++++------------
  1 files changed, 29 insertions(+), 18 deletions(-)


diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -941,31 +941,42 @@
           stdout
           stderr
 
-   :term:`File objects <file object>` corresponding to the interpreter's standard
-   input, output and error streams.  ``stdin`` is used for all interpreter input
-   except for scripts but including calls to :func:`input`.  ``stdout`` is used
-   for the output of :func:`print` and :term:`expression` statements and for the
-   prompts of :func:`input`. The interpreter's own prompts
-   and (almost all of) its error messages go to ``stderr``.  ``stdout`` and
-   ``stderr`` needn't be built-in file objects: any object is acceptable as long
-   as it has a :meth:`write` method that takes a string argument.  (Changing these
-   objects doesn't affect the standard I/O streams of processes executed by
-   :func:`os.popen`, :func:`os.system` or the :func:`exec\*` family of functions in
-   the :mod:`os` module.)
+   :term:`File objects <file object>` used by the interpreter for standard
+   input, output and errors:
 
-   The standard streams are in text mode by default.  To write or read binary
-   data to these, use the underlying binary buffer.  For example, to write bytes
-   to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``.  Using
-   :meth:`io.TextIOBase.detach` streams can be made binary by default.  This
+   * ``stdin`` is used for all interactive input (including calls to
+     :func:`input`);
+   * ``stdout`` is used for the output of :func:`print` and :term:`expression`
+     statements and for the prompts of :func:`input`;
+   * The interpreter's own prompts and its error messages go to ``stderr``.
+
+   By default, these streams are regular text streams as returned by the
+   :func:`open` function.  Their parameters are chosen as follows:
+
+   * The character encoding is platform-dependent.  Under Windows, if the stream
+     is interactive (that is, if its :meth:`isatty` method returns True), the
+     console codepage is used, otherwise the ANSI code page.  Under other
+     platforms, the locale encoding is used (see :meth:`locale.getpreferredencoding`).
+
+     Under all platforms though, you can override this value by setting the
+     :envvar:`PYTHONIOENCODING` environment variable.
+
+   * When interactive, standard streams are line-buffered.  Otherwise, they
+     are block-buffered like regular text files.  You can override this
+     value with the :option:`-u` command-line option.
+
+   To write or read binary data from/to the standard streams, use the
+   underlying binary :data:`~io.TextIOBase.buffer`.  For example, to write
+   bytes to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``.  Using
+   :meth:`io.TextIOBase.detach`, streams can be made binary by default.  This
    function sets :data:`stdin` and :data:`stdout` to binary::
 
       def make_streams_binary():
           sys.stdin = sys.stdin.detach()
           sys.stdout = sys.stdout.detach()
 
-   Note that the streams can be replaced with objects (like
-   :class:`io.StringIO`) that do not support the
-   :attr:`~io.BufferedIOBase.buffer` attribute or the
+   Note that the streams may be replaced with objects (like :class:`io.StringIO`)
+   that do not support the :attr:`~io.BufferedIOBase.buffer` attribute or the
    :meth:`~io.BufferedIOBase.detach` method and can raise :exc:`AttributeError`
    or :exc:`io.UnsupportedOperation`.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:34:10 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 15 Dec 2011 21:34:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_improve_abstract_property_s?=
 =?utf8?q?upport_=28closes_=2311610=29?=
Message-ID: <E1RbI0I-0005Iw-A0@dinsdale.python.org>

http://hg.python.org/cpython/rev/e979b26a9172
changeset:   73984:e979b26a9172
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 15:34:02 2011 -0500
summary:
  improve abstract property support (closes #11610)

Thanks to Darren Dale for patch.

files:
  Doc/library/abc.rst       |   75 +++++++++--
  Doc/whatsnew/3.3.rst      |   17 ++
  Include/object.h          |    1 +
  Lib/abc.py                |   19 ++-
  Lib/numbers.py            |   14 +-
  Lib/test/test_abc.py      |  172 ++++++++++++++++++++++++-
  Lib/test/test_property.py |   23 +++
  Misc/ACKS                 |    1 +
  Misc/NEWS                 |    2 +
  Objects/descrobject.c     |   39 +++++-
  Objects/funcobject.c      |   46 ++++++-
  Objects/object.c          |   23 +++
  12 files changed, 396 insertions(+), 36 deletions(-)


diff --git a/Doc/library/abc.rst b/Doc/library/abc.rst
--- a/Doc/library/abc.rst
+++ b/Doc/library/abc.rst
@@ -127,19 +127,18 @@
    available as a method of ``Foo``, so it is provided separately.
 
 
-It also provides the following decorators:
+The :mod:`abc` module also provides the following decorators:
 
 .. decorator:: abstractmethod(function)
 
    A decorator indicating abstract methods.
 
-   Using this decorator requires that the class's metaclass is :class:`ABCMeta` or
-   is derived from it.
-   A class that has a metaclass derived from :class:`ABCMeta`
-   cannot be instantiated unless all of its abstract methods and
-   properties are overridden.
-   The abstract methods can be called using any of the normal 'super' call
-   mechanisms.
+   Using this decorator requires that the class's metaclass is :class:`ABCMeta`
+   or is derived from it.  A class that has a metaclass derived from
+   :class:`ABCMeta` cannot be instantiated unless all of its abstract methods
+   and properties are overridden.  The abstract methods can be called using any
+   of the normal 'super' call mechanisms.  :func:`abstractmethod` may be used
+   to declare abstract methods for properties and descriptors.
 
    Dynamically adding abstract methods to a class, or attempting to modify the
    abstraction status of a method or class once it is created, are not
@@ -147,12 +146,52 @@
    regular inheritance; "virtual subclasses" registered with the ABC's
    :meth:`register` method are not affected.
 
-   Usage::
+   When :func:`abstractmethod` is applied in combination with other method
+   descriptors, it should be applied as the innermost decorator, as shown in
+   the following usage examples::
 
       class C(metaclass=ABCMeta):
           @abstractmethod
           def my_abstract_method(self, ...):
               ...
+          @classmethod
+          @abstractmethod
+          def my_abstract_classmethod(cls, ...):
+              ...
+          @staticmethod
+          @abstractmethod
+          def my_abstract_staticmethod(...):
+              ...
+
+          @property
+          @abstractmethod
+          def my_abstract_property(self):
+              ...
+          @my_abstract_property.setter
+          @abstractmethod
+          def my_abstract_property(self, val):
+              ...
+
+          @abstractmethod
+          def _get_x(self):
+              ...
+          @abstractmethod
+          def _set_x(self, val):
+              ...
+          x = property(_get_x, _set_x)
+
+   In order to correctly interoperate with the abstract base class machinery,
+   the descriptor must identify itself as abstract using
+   :attr:`__isabstractmethod__`. In general, this attribute should be ``True``
+   if any of the methods used to compose the descriptor are abstract. For
+   example, Python's built-in property does the equivalent of::
+
+      class Descriptor:
+          ...
+          @property
+          def __isabstractmethod__(self):
+              return any(getattr(f, '__isabstractmethod__', False) for
+                         f in (self._fget, self._fset, self._fdel))
 
    .. note::
 
@@ -177,6 +216,8 @@
               ...
 
    .. versionadded:: 3.2
+   .. deprecated:: 3.3
+       Use :class:`classmethod` with :func:`abstractmethod` instead
 
 
 .. decorator:: abstractstaticmethod(function)
@@ -192,18 +233,19 @@
               ...
 
    .. versionadded:: 3.2
+   .. deprecated:: 3.3
+       Use :class:`staticmethod` with :func:`abstractmethod` instead
 
 
 .. decorator:: abstractproperty(fget=None, fset=None, fdel=None, doc=None)
 
    A subclass of the built-in :func:`property`, indicating an abstract property.
 
-   Using this function requires that the class's metaclass is :class:`ABCMeta` or
-   is derived from it.
-   A class that has a metaclass derived from :class:`ABCMeta` cannot be
-   instantiated unless all of its abstract methods and properties are overridden.
-   The abstract properties can be called using any of the normal
-   'super' call mechanisms.
+   Using this function requires that the class's metaclass is :class:`ABCMeta`
+   or is derived from it. A class that has a metaclass derived from
+   :class:`ABCMeta` cannot be instantiated unless all of its abstract methods
+   and properties are overridden. The abstract properties can be called using
+   any of the normal 'super' call mechanisms.
 
    Usage::
 
@@ -220,6 +262,9 @@
           def setx(self, value): ...
           x = abstractproperty(getx, setx)
 
+   .. deprecated:: 3.3
+       Use :class:`property` with :func:`abstractmethod` instead
+
 
 .. rubric:: Footnotes
 
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -352,6 +352,23 @@
 
 (Contributed by I?igo Serna in :issue:`6755`)
 
+abc
+---
+
+Improved support for abstract base classes containing descriptors composed with
+abstract methods. The recommended approach to declaring abstract descriptors is
+now to provide :attr:`__isabstractmethod__` as a dynamically updated
+property. The built-in descriptors have been updated accordingly.
+
+  * :class:`abc.abstractproperty` has been deprecated, use :class:`property`
+    with :func:`abc.abstractmethod` instead.
+  * :class:`abc.abstractclassmethod` has been deprecated, use
+    :class:`classmethod` with :func:`abc.abstractmethod` instead.
+  * :class:`abc.abstractstaticmethod` has been deprecated, use
+    :class:`property` with :func:`abc.abstractmethod` instead.
+
+(Contributed by Darren Dale in :issue:`11610`)
+
 faulthandler
 ------------
 
diff --git a/Include/object.h b/Include/object.h
--- a/Include/object.h
+++ b/Include/object.h
@@ -473,6 +473,7 @@
 PyAPI_FUNC(PyObject *) PyObject_GetAttr(PyObject *, PyObject *);
 PyAPI_FUNC(int) PyObject_SetAttr(PyObject *, PyObject *, PyObject *);
 PyAPI_FUNC(int) PyObject_HasAttr(PyObject *, PyObject *);
+PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *);
 PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *);
 PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *);
 PyAPI_FUNC(int) _PyObject_HasAttrId(PyObject *, struct _Py_Identifier *);
diff --git a/Lib/abc.py b/Lib/abc.py
--- a/Lib/abc.py
+++ b/Lib/abc.py
@@ -26,7 +26,8 @@
 
 
 class abstractclassmethod(classmethod):
-    """A decorator indicating abstract classmethods.
+    """
+    A decorator indicating abstract classmethods.
 
     Similar to abstractmethod.
 
@@ -36,6 +37,9 @@
             @abstractclassmethod
             def my_abstract_classmethod(cls, ...):
                 ...
+
+    'abstractclassmethod' is deprecated. Use 'classmethod' with
+    'abstractmethod' instead.
     """
 
     __isabstractmethod__ = True
@@ -46,7 +50,8 @@
 
 
 class abstractstaticmethod(staticmethod):
-    """A decorator indicating abstract staticmethods.
+    """
+    A decorator indicating abstract staticmethods.
 
     Similar to abstractmethod.
 
@@ -56,6 +61,9 @@
             @abstractstaticmethod
             def my_abstract_staticmethod(...):
                 ...
+
+    'abstractstaticmethod' is deprecated. Use 'staticmethod' with
+    'abstractmethod' instead.
     """
 
     __isabstractmethod__ = True
@@ -66,7 +74,8 @@
 
 
 class abstractproperty(property):
-    """A decorator indicating abstract properties.
+    """
+    A decorator indicating abstract properties.
 
     Requires that the metaclass is ABCMeta or derived from it.  A
     class that has a metaclass derived from ABCMeta cannot be
@@ -88,7 +97,11 @@
             def getx(self): ...
             def setx(self, value): ...
             x = abstractproperty(getx, setx)
+
+    'abstractproperty' is deprecated. Use 'property' with 'abstractmethod'
+    instead.
     """
+
     __isabstractmethod__ = True
 
 
diff --git a/Lib/numbers.py b/Lib/numbers.py
--- a/Lib/numbers.py
+++ b/Lib/numbers.py
@@ -5,7 +5,7 @@
 
 TODO: Fill out more detailed documentation on the operators."""
 
-from abc import ABCMeta, abstractmethod, abstractproperty
+from abc import ABCMeta, abstractmethod
 
 __all__ = ["Number", "Complex", "Real", "Rational", "Integral"]
 
@@ -50,7 +50,8 @@
         """True if self != 0. Called for bool(self)."""
         return self != 0
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def real(self):
         """Retrieve the real component of this number.
 
@@ -58,7 +59,8 @@
         """
         raise NotImplementedError
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def imag(self):
         """Retrieve the imaginary component of this number.
 
@@ -272,11 +274,13 @@
 
     __slots__ = ()
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def numerator(self):
         raise NotImplementedError
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def denominator(self):
         raise NotImplementedError
 
diff --git a/Lib/test/test_abc.py b/Lib/test/test_abc.py
--- a/Lib/test/test_abc.py
+++ b/Lib/test/test_abc.py
@@ -10,14 +10,7 @@
 from inspect import isabstract
 
 
-class TestABC(unittest.TestCase):
-
-    def test_abstractmethod_basics(self):
-        @abc.abstractmethod
-        def foo(self): pass
-        self.assertTrue(foo.__isabstractmethod__)
-        def bar(self): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+class TestLegacyAPI(unittest.TestCase):
 
     def test_abstractproperty_basics(self):
         @abc.abstractproperty
@@ -29,10 +22,12 @@
         class C(metaclass=abc.ABCMeta):
             @abc.abstractproperty
             def foo(self): return 3
+        self.assertRaises(TypeError, C)
         class D(C):
             @property
             def foo(self): return super().foo
         self.assertEqual(D().foo, 3)
+        self.assertFalse(getattr(D.foo, "__isabstractmethod__", False))
 
     def test_abstractclassmethod_basics(self):
         @abc.abstractclassmethod
@@ -40,7 +35,7 @@
         self.assertTrue(foo.__isabstractmethod__)
         @classmethod
         def bar(cls): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
 
         class C(metaclass=abc.ABCMeta):
             @abc.abstractclassmethod
@@ -58,7 +53,7 @@
         self.assertTrue(foo.__isabstractmethod__)
         @staticmethod
         def bar(): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
 
         class C(metaclass=abc.ABCMeta):
             @abc.abstractstaticmethod
@@ -98,6 +93,163 @@
             self.assertRaises(TypeError, F)  # because bar is abstract now
             self.assertTrue(isabstract(F))
 
+
+class TestABC(unittest.TestCase):
+
+    def test_abstractmethod_basics(self):
+        @abc.abstractmethod
+        def foo(self): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        def bar(self): pass
+        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+
+    def test_abstractproperty_basics(self):
+        @property
+        @abc.abstractmethod
+        def foo(self): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        def bar(self): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @property
+            @abc.abstractmethod
+            def foo(self): return 3
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertEqual(D().foo, 3)
+
+    def test_abstractclassmethod_basics(self):
+        @classmethod
+        @abc.abstractmethod
+        def foo(cls): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        @classmethod
+        def bar(cls): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @classmethod
+            @abc.abstractmethod
+            def foo(cls): return cls.__name__
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @classmethod
+            def foo(cls): return super().foo()
+        self.assertEqual(D.foo(), 'D')
+        self.assertEqual(D().foo(), 'D')
+
+    def test_abstractstaticmethod_basics(self):
+        @staticmethod
+        @abc.abstractmethod
+        def foo(): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        @staticmethod
+        def bar(): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @staticmethod
+            @abc.abstractmethod
+            def foo(): return 3
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @staticmethod
+            def foo(): return 4
+        self.assertEqual(D.foo(), 4)
+        self.assertEqual(D().foo(), 4)
+
+    def test_abstractmethod_integration(self):
+        for abstractthing in [abc.abstractmethod, abc.abstractproperty,
+                              abc.abstractclassmethod,
+                              abc.abstractstaticmethod]:
+            class C(metaclass=abc.ABCMeta):
+                @abstractthing
+                def foo(self): pass  # abstract
+                def bar(self): pass  # concrete
+            self.assertEqual(C.__abstractmethods__, {"foo"})
+            self.assertRaises(TypeError, C)  # because foo is abstract
+            self.assertTrue(isabstract(C))
+            class D(C):
+                def bar(self): pass  # concrete override of concrete
+            self.assertEqual(D.__abstractmethods__, {"foo"})
+            self.assertRaises(TypeError, D)  # because foo is still abstract
+            self.assertTrue(isabstract(D))
+            class E(D):
+                def foo(self): pass
+            self.assertEqual(E.__abstractmethods__, set())
+            E()  # now foo is concrete, too
+            self.assertFalse(isabstract(E))
+            class F(E):
+                @abstractthing
+                def bar(self): pass  # abstract override of concrete
+            self.assertEqual(F.__abstractmethods__, {"bar"})
+            self.assertRaises(TypeError, F)  # because bar is abstract now
+            self.assertTrue(isabstract(F))
+
+    def test_descriptors_with_abstractmethod(self):
+        class C(metaclass=abc.ABCMeta):
+            @property
+            @abc.abstractmethod
+            def foo(self): return 3
+            @foo.setter
+            @abc.abstractmethod
+            def foo(self, val): pass
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertRaises(TypeError, D)
+        class E(D):
+            @D.foo.setter
+            def foo(self, val): pass
+        self.assertEqual(E().foo, 3)
+        # check that the property's __isabstractmethod__ descriptor does the
+        # right thing when presented with a value that fails truth testing:
+        class NotBool(object):
+            def __nonzero__(self):
+                raise ValueError()
+            __len__ = __nonzero__
+        with self.assertRaises(ValueError):
+            class F(C):
+                def bar(self):
+                    pass
+                bar.__isabstractmethod__ = NotBool()
+                foo = property(bar)
+
+
+    def test_customdescriptors_with_abstractmethod(self):
+        class Descriptor:
+            def __init__(self, fget, fset=None):
+                self._fget = fget
+                self._fset = fset
+            def getter(self, callable):
+                return Descriptor(callable, self._fget)
+            def setter(self, callable):
+                return Descriptor(self._fget, callable)
+            @property
+            def __isabstractmethod__(self):
+                return (getattr(self._fget, '__isabstractmethod__', False)
+                        or getattr(self._fset, '__isabstractmethod__', False))
+        class C(metaclass=abc.ABCMeta):
+            @Descriptor
+            @abc.abstractmethod
+            def foo(self): return 3
+            @foo.setter
+            @abc.abstractmethod
+            def foo(self, val): pass
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertRaises(TypeError, D)
+        class E(D):
+            @D.foo.setter
+            def foo(self, val): pass
+        self.assertFalse(E.foo.__isabstractmethod__)
+
     def test_metaclass_abc(self):
         # Metaclasses can be ABCs, too.
         class A(metaclass=abc.ABCMeta):
diff --git a/Lib/test/test_property.py b/Lib/test/test_property.py
--- a/Lib/test/test_property.py
+++ b/Lib/test/test_property.py
@@ -128,6 +128,29 @@
         self.assertEqual(newgetter.spam, 8)
         self.assertEqual(newgetter.__class__.spam.__doc__, "new docstring")
 
+    def test_property___isabstractmethod__descriptor(self):
+        for val in (True, False, [], [1], '', '1'):
+            class C(object):
+                def foo(self):
+                    pass
+                foo.__isabstractmethod__ = val
+                foo = property(foo)
+            self.assertIs(C.foo.__isabstractmethod__, bool(val))
+
+        # check that the property's __isabstractmethod__ descriptor does the
+        # right thing when presented with a value that fails truth testing:
+        class NotBool(object):
+            def __nonzero__(self):
+                raise ValueError()
+            __len__ = __nonzero__
+        with self.assertRaises(ValueError):
+            class C(object):
+                def foo(self):
+                    pass
+                foo.__isabstractmethod__ = NotBool()
+                foo = property(foo)
+            C.foo.__isabstractmethod__
+
 
 # Issue 5890: subclasses of property do not preserve method __doc__ strings
 class PropertySub(property):
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -220,6 +220,7 @@
 Antonio Cuni
 Brian Curtin
 Lisandro Dalcin
+Darren Dale
 Andrew Dalke
 Lars Damerow
 Evan Dandrea
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -416,6 +416,8 @@
 Library
 -------
 
+- Issue #11610: Introduce a more general way to declare abstract properties.
+
 - Issue #13591: A bug in importlib has been fixed that caused import_module
   to load a module twice.
 
diff --git a/Objects/descrobject.c b/Objects/descrobject.c
--- a/Objects/descrobject.c
+++ b/Objects/descrobject.c
@@ -1380,6 +1380,43 @@
     return 0;
 }
 
+static PyObject *
+property_get___isabstractmethod__(propertyobject *prop, void *closure)
+{
+    int res = _PyObject_IsAbstract(prop->prop_get);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+
+    res = _PyObject_IsAbstract(prop->prop_set);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+
+    res = _PyObject_IsAbstract(prop->prop_del);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef property_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)property_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(property_doc,
 "property(fget=None, fset=None, fdel=None, doc=None) -> property attribute\n"
 "\n"
@@ -1445,7 +1482,7 @@
     0,                                          /* tp_iternext */
     property_methods,                           /* tp_methods */
     property_members,                           /* tp_members */
-    0,                                          /* tp_getset */
+    property_getsetlist,                        /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     property_descr_get,                         /* tp_descr_get */
diff --git a/Objects/funcobject.c b/Objects/funcobject.c
--- a/Objects/funcobject.c
+++ b/Objects/funcobject.c
@@ -814,6 +814,27 @@
     {NULL}  /* Sentinel */
 };
 
+static PyObject *
+cm_get___isabstractmethod__(classmethod *cm, void *closure)
+{
+    int res = _PyObject_IsAbstract(cm->cm_callable);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef cm_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)cm_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(classmethod_doc,
 "classmethod(function) -> method\n\
 \n\
@@ -865,7 +886,7 @@
     0,                                          /* tp_iternext */
     0,                                          /* tp_methods */
     cm_memberlist,              /* tp_members */
-    0,                                          /* tp_getset */
+    cm_getsetlist,                              /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     cm_descr_get,                               /* tp_descr_get */
@@ -969,6 +990,27 @@
     {NULL}  /* Sentinel */
 };
 
+static PyObject *
+sm_get___isabstractmethod__(staticmethod *sm, void *closure)
+{
+    int res = _PyObject_IsAbstract(sm->sm_callable);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef sm_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)sm_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(staticmethod_doc,
 "staticmethod(function) -> method\n\
 \n\
@@ -1017,7 +1059,7 @@
     0,                                          /* tp_iternext */
     0,                                          /* tp_methods */
     sm_memberlist,              /* tp_members */
-    0,                                          /* tp_getset */
+    sm_getsetlist,                              /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     sm_descr_get,                               /* tp_descr_get */
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -840,6 +840,29 @@
     return res;
 }
 
+int
+_PyObject_IsAbstract(PyObject *obj)
+{
+    int res;
+    PyObject* isabstract;
+    _Py_IDENTIFIER(__isabstractmethod__);
+
+    if (obj == NULL)
+        return 0;
+
+    isabstract = _PyObject_GetAttrId(obj, &PyId___isabstractmethod__);
+    if (isabstract == NULL) {
+        if (PyErr_ExceptionMatches(PyExc_AttributeError)) {
+            PyErr_Clear();
+            return 0;
+        }
+        return -1;
+    }
+    res = PyObject_IsTrue(isabstract);
+    Py_DECREF(isabstract);
+    return res;
+}
+
 PyObject *
 _PyObject_GetAttrId(PyObject *v, _Py_Identifier *name)
 {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:40:39 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 15 Dec 2011 21:40:39 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTQ1?=
 =?utf8?q?=3A_Fix_platform=2Elibc=5Fversion=28=29_is_the_SO_version_is_mis?=
 =?utf8?q?sing?=
Message-ID: <E1RbI6Z-000685-F6@dinsdale.python.org>

http://hg.python.org/cpython/rev/5ec7ecf62c1d
changeset:   73985:5ec7ecf62c1d
branch:      3.2
parent:      73982:313fa7ea6c79
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 15 21:42:03 2011 +0100
summary:
  Issue #13545: Fix platform.libc_version() is the SO version is missing

files:
  Lib/platform.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/platform.py b/Lib/platform.py
--- a/Lib/platform.py
+++ b/Lib/platform.py
@@ -181,7 +181,7 @@
         elif so:
             if lib != 'glibc':
                 lib = 'libc'
-                if soversion > version:
+                if soversion and soversion > version:
                     version = soversion
                 if threads and version[-len(threads):] != threads:
                     version = version + threads

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:40:40 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 15 Dec 2011 21:40:40 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=28Merge_3=2E2=29_Issue_=2313545=3A_Fix_platform=2Elibc=5Fve?=
 =?utf8?q?rsion=28=29_is_the_SO_version_is?=
Message-ID: <E1RbI6a-000689-Ab@dinsdale.python.org>

http://hg.python.org/cpython/rev/a9ee21ac0879
changeset:   73986:a9ee21ac0879
parent:      73984:e979b26a9172
parent:      73985:5ec7ecf62c1d
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 15 21:42:28 2011 +0100
summary:
  (Merge 3.2) Issue #13545: Fix platform.libc_version() is the SO version is missing

files:
  Lib/platform.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/platform.py b/Lib/platform.py
--- a/Lib/platform.py
+++ b/Lib/platform.py
@@ -186,7 +186,7 @@
         elif so:
             if lib != 'glibc':
                 lib = 'libc'
-                if soversion > version:
+                if soversion and soversion > version:
                     version = soversion
                 if threads and version[-len(threads):] != threads:
                     version = version + threads

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:44:00 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 15 Dec 2011 21:44:00 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTQ1?=
 =?utf8?q?=3A_Fix_platform=2Elibc=5Fversion=28=29_is_the_SO_version_is_mis?=
 =?utf8?q?sing?=
Message-ID: <E1RbI9o-0006NY-CK@dinsdale.python.org>

http://hg.python.org/cpython/rev/f2a5dcced66d
changeset:   73987:f2a5dcced66d
branch:      2.7
parent:      73980:da1d7f8cd487
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 15 21:42:03 2011 +0100
summary:
  Issue #13545: Fix platform.libc_version() is the SO version is missing

files:
  Lib/platform.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/platform.py b/Lib/platform.py
--- a/Lib/platform.py
+++ b/Lib/platform.py
@@ -183,7 +183,7 @@
         elif so:
             if lib != 'glibc':
                 lib = 'libc'
-                if soversion > version:
+                if soversion and soversion > version:
                     version = soversion
                 if threads and version[-len(threads):] != threads:
                     version = version + threads

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:44:21 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 15 Dec 2011 21:44:21 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_input=28=29_in_this_sense_i?=
	=?utf8?q?s_gone?=
Message-ID: <E1RbIA9-0006Oo-Ql@dinsdale.python.org>

http://hg.python.org/cpython/rev/4de895bb25f6
changeset:   73988:4de895bb25f6
parent:      73984:e979b26a9172
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 15:43:56 2011 -0500
summary:
  input() in this sense is gone

files:
  Grammar/Grammar |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Grammar/Grammar b/Grammar/Grammar
--- a/Grammar/Grammar
+++ b/Grammar/Grammar
@@ -13,7 +13,7 @@
 # Start symbols for the grammar:
 #       single_input is a single interactive statement;
 #       file_input is a module or sequence of commands read from an input file;
-#       eval_input is the input for the eval() and input() functions.
+#       eval_input is the input for the eval() functions.
 # NB: compound_stmt in single_input is followed by extra NEWLINE!
 single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
 file_input: (NEWLINE | stmt)* ENDMARKER

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:44:22 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 15 Dec 2011 21:44:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RbIAA-0006Os-LE@dinsdale.python.org>

http://hg.python.org/cpython/rev/df5c28c01deb
changeset:   73989:df5c28c01deb
parent:      73988:4de895bb25f6
parent:      73986:a9ee21ac0879
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 15:44:16 2011 -0500
summary:
  merge heads

files:
  Lib/platform.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/platform.py b/Lib/platform.py
--- a/Lib/platform.py
+++ b/Lib/platform.py
@@ -186,7 +186,7 @@
         elif so:
             if lib != 'glibc':
                 lib = 'libc'
-                if soversion > version:
+                if soversion and soversion > version:
                     version = soversion
                 if threads and version[-len(threads):] != threads:
                     version = version + threads

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:47:09 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 15 Dec 2011 21:47:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Close_=2313596=3A_Only_reco?=
 =?utf8?q?mpile_Lib/=5Fsysconfigdata=2Epy_when_needed?=
Message-ID: <E1RbICr-0006gp-Dp@dinsdale.python.org>

http://hg.python.org/cpython/rev/06d83098d9a9
changeset:   73990:06d83098d9a9
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 15 21:48:39 2011 +0100
summary:
  Close #13596: Only recompile Lib/_sysconfigdata.py when needed

files:
  Makefile.pre.in |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -379,6 +379,7 @@
 		Objects/unicodectype.o \
 		Objects/weakrefobject.o
 
+SYSCONFIGDATA=$(srcdir)/Lib/_sysconfigdata.py
 
 ##########################################################################
 # objects that get linked into the Python library
@@ -396,7 +397,7 @@
 
 # Default target
 all:		build_all
-build_all:	$(BUILDPYTHON) sysconfig oldsharedmods sharedmods gdbhooks Modules/_testembed
+build_all:	$(BUILDPYTHON) $(SYSCONFIGDATA) oldsharedmods sharedmods gdbhooks Modules/_testembed
 
 # Compile a binary with gcc profile guided optimization.
 profile-opt:
@@ -429,15 +430,15 @@
 $(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
 	$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
 
-platform: $(BUILDPYTHON) sysconfig
+platform: $(BUILDPYTHON) $(SYSCONFIGDATA)
 	$(RUNSHARED) ./$(BUILDPYTHON) -E -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
 
 # Generate the sysconfig build-time data
-sysconfig: $(BUILDPYTHON)
+$(SYSCONFIGDATA): $(BUILDPYTHON)
 	$(RUNSHARED) ./$(BUILDPYTHON) -SE -m sysconfig --generate-posix-vars
 
 # Build the shared modules
-sharedmods: $(BUILDPYTHON) sysconfig
+sharedmods: $(BUILDPYTHON) $(SYSCONFIGDATA)
 	@case $$MAKEFLAGS in \
 	*s*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py -q build;; \
 	*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py build;; \
@@ -1315,7 +1316,7 @@
 	find build -name 'fficonfig.h' -exec rm -f {} ';' || true
 	find build -name 'fficonfig.py' -exec rm -f {} ';' || true
 	-rm -f Lib/lib2to3/*Grammar*.pickle
-	-rm -f Lib/_sysconfigdata.py
+	-rm -f $(SYSCONFIGDATA)
 	-rm -f Modules/_testembed
 
 profile-removal:
@@ -1393,7 +1394,7 @@
 Python/thread.o: @THREADHEADERS@
 
 # Declare targets that aren't real files
-.PHONY: all build_all sysconfig sharedmods oldsharedmods test quicktest
+.PHONY: all build_all sharedmods oldsharedmods test quicktest
 .PHONY: install altinstall oldsharedinstall bininstall altbininstall
 .PHONY: maninstall libinstall inclinstall libainstall sharedinstall
 .PHONY: frameworkinstall frameworkinstallframework frameworkinstallstructure

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:57:51 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 15 Dec 2011 21:57:51 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_this_test_to_actually_t?=
 =?utf8?q?est_something_=28closes_=2313606=29?=
Message-ID: <E1RbIND-0007UN-Tz@dinsdale.python.org>

http://hg.python.org/cpython/rev/c9ae0eb66959
changeset:   73991:c9ae0eb66959
parent:      73989:df5c28c01deb
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 15:57:15 2011 -0500
summary:
  fix this test to actually test something (closes #13606)

Thanks Mark Shannon.

files:
  Lib/test/test_module.py |  6 ++++--
  1 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_module.py b/Lib/test/test_module.py
--- a/Lib/test/test_module.py
+++ b/Lib/test/test_module.py
@@ -70,9 +70,11 @@
         m = ModuleType("foo")
         m.destroyed = destroyed
         s = """class A:
+    def __init__(self, l):
+        self.l = l
     def __del__(self):
-        destroyed.append(1)
-a = A()"""
+        self.l.append(1)
+a = A(destroyed)"""
         exec(s, m.__dict__)
         del m
         gc_collect()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 15 21:57:52 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 15 Dec 2011 21:57:52 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RbINE-0007UR-Me@dinsdale.python.org>

http://hg.python.org/cpython/rev/df57bea858f3
changeset:   73992:df57bea858f3
parent:      73991:c9ae0eb66959
parent:      73990:06d83098d9a9
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 15:57:47 2011 -0500
summary:
  merge heads

files:
  Makefile.pre.in |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -379,6 +379,7 @@
 		Objects/unicodectype.o \
 		Objects/weakrefobject.o
 
+SYSCONFIGDATA=$(srcdir)/Lib/_sysconfigdata.py
 
 ##########################################################################
 # objects that get linked into the Python library
@@ -396,7 +397,7 @@
 
 # Default target
 all:		build_all
-build_all:	$(BUILDPYTHON) sysconfig oldsharedmods sharedmods gdbhooks Modules/_testembed
+build_all:	$(BUILDPYTHON) $(SYSCONFIGDATA) oldsharedmods sharedmods gdbhooks Modules/_testembed
 
 # Compile a binary with gcc profile guided optimization.
 profile-opt:
@@ -429,15 +430,15 @@
 $(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
 	$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
 
-platform: $(BUILDPYTHON) sysconfig
+platform: $(BUILDPYTHON) $(SYSCONFIGDATA)
 	$(RUNSHARED) ./$(BUILDPYTHON) -E -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
 
 # Generate the sysconfig build-time data
-sysconfig: $(BUILDPYTHON)
+$(SYSCONFIGDATA): $(BUILDPYTHON)
 	$(RUNSHARED) ./$(BUILDPYTHON) -SE -m sysconfig --generate-posix-vars
 
 # Build the shared modules
-sharedmods: $(BUILDPYTHON) sysconfig
+sharedmods: $(BUILDPYTHON) $(SYSCONFIGDATA)
 	@case $$MAKEFLAGS in \
 	*s*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py -q build;; \
 	*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py build;; \
@@ -1315,7 +1316,7 @@
 	find build -name 'fficonfig.h' -exec rm -f {} ';' || true
 	find build -name 'fficonfig.py' -exec rm -f {} ';' || true
 	-rm -f Lib/lib2to3/*Grammar*.pickle
-	-rm -f Lib/_sysconfigdata.py
+	-rm -f $(SYSCONFIGDATA)
 	-rm -f Modules/_testembed
 
 profile-removal:
@@ -1393,7 +1394,7 @@
 Python/thread.o: @THREADHEADERS@
 
 # Declare targets that aren't real files
-.PHONY: all build_all sysconfig sharedmods oldsharedmods test quicktest
+.PHONY: all build_all sharedmods oldsharedmods test quicktest
 .PHONY: install altinstall oldsharedinstall bininstall altbininstall
 .PHONY: maninstall libinstall inclinstall libainstall sharedinstall
 .PHONY: frameworkinstall frameworkinstallframework frameworkinstallstructure

-- 
Repository URL: http://hg.python.org/cpython

From ncoghlan at gmail.com  Fri Dec 16 00:18:16 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Dec 2011 09:18:16 +1000
Subject: [Python-checkins] cpython: improve abstract property support
 (closes #11610)
In-Reply-To: <E1RbI0I-0005Iw-A0@dinsdale.python.org>
References: <E1RbI0I-0005Iw-A0@dinsdale.python.org>
Message-ID: <CADiSq7dWDtS1BacWSDmHos89Y5Gr0_rOvpNajOa_HkwmtjdV2g@mail.gmail.com>

On Fri, Dec 16, 2011 at 6:34 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> +abc
> +---
> +
> +Improved support for abstract base classes containing descriptors composed with
> +abstract methods. The recommended approach to declaring abstract descriptors is
> +now to provide :attr:`__isabstractmethod__` as a dynamically updated
> +property. The built-in descriptors have been updated accordingly.
> +
> + ?* :class:`abc.abstractproperty` has been deprecated, use :class:`property`
> + ? ?with :func:`abc.abstractmethod` instead.
> + ?* :class:`abc.abstractclassmethod` has been deprecated, use
> + ? ?:class:`classmethod` with :func:`abc.abstractmethod` instead.
> + ?* :class:`abc.abstractstaticmethod` has been deprecated, use
> + ? ?:class:`property` with :func:`abc.abstractmethod` instead.
> +
> +(Contributed by Darren Dale in :issue:`11610`)

s/property/staticmethod/ in the final bullet point here.

Cheers,
Nick.
-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From python-checkins at python.org  Fri Dec 16 00:21:47 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 00:21:47 +0100
Subject: [Python-checkins] =?utf8?q?benchmarks=3A_Fix_gcbench_for_Python_3?=
Message-ID: <E1RbKcV-0002ih-O9@dinsdale.python.org>

http://hg.python.org/benchmarks/rev/3f1cef15cf22
changeset:   154:3f1cef15cf22
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 00:21:23 2011 +0100
summary:
  Fix gcbench for Python 3

files:
  performance/gcbench.py |  31 ++++++++++++++++-------------
  1 files changed, 17 insertions(+), 14 deletions(-)


diff --git a/performance/gcbench.py b/performance/gcbench.py
--- a/performance/gcbench.py
+++ b/performance/gcbench.py
@@ -43,6 +43,9 @@
 #               - No attempt to measure variation with object size
 #               - Results are sensitive to locking cost, but we dont
 #                 check for proper locking
+
+from __future__ import print_function
+
 import os, time
 
 USAGE = """gcbench [num_repetitions] [--depths=N,N,N..] [--threads=N]"""
@@ -67,7 +70,7 @@
 
 def num_iters(i):
     "Number of iterations to use for a given tree depth"
-    return 2 * tree_size(kStretchTreeDepth) / tree_size(i);
+    return 2 * tree_size(kStretchTreeDepth) // tree_size(i);
 
 def populate(depth, node):
     "Build tree top down, assigning to older objects."
@@ -93,20 +96,20 @@
 
 def time_construction(depth):
     niters = num_iters(depth)
-    print "Creating %d trees of depth %d" % (niters, depth)
+    print("Creating %d trees of depth %d" % (niters, depth))
     t_start = time.time()
     for i in range(niters):
         temp_tree = Node()
         populate(depth, temp_tree)
         temp_tree = None
     t_finish = time.time()
-    print "\tTop down constrution took %f ms" % ((t_finish-t_start)*1000.)
+    print("\tTop down constrution took %f ms" % ((t_finish-t_start)*1000.))
     t_start = time.time()
     for i in range(niters):
         temp_tree = make_tree(depth)
         temp_tree = None
     t_finish = time.time()
-    print "\tBottom up constrution took %f ms" % ((t_finish-t_start)*1000.)
+    print("\tBottom up constrution took %f ms" % ((t_finish-t_start)*1000.))
 
 DEFAULT_DEPTHS = range(kMinTreeDepth, kMaxTreeDepth+1, 2)
 
@@ -117,30 +120,30 @@
 def time_parallel_constructions(depths, nthreads):
     import threading
     threadlist = []
-    print "Starting %d parallel threads..." % (nthreads,)
+    print("Starting %d parallel threads..." % (nthreads,))
     for n in range(nthreads):
         t = threading.Thread(target=time_constructions, args=(depths,))
         t.start()
         threadlist.append(t)
     for t in threadlist:
         t.join()
-    print "All %d threads finished" % (nthreads,)
+    print("All %d threads finished" % (nthreads,))
 
 def main(depths=DEFAULT_DEPTHS, threads=0):
-    print "Garbage Collector Test"
-    print " Stretching memory with a binary tree of depth %d" % kStretchTreeDepth
+    print("Garbage Collector Test")
+    print(" Stretching memory with a binary tree of depth %d" % kStretchTreeDepth)
     print_diagnostics()
     t_start = time.time()
     temp_tree = make_tree(kStretchTreeDepth)
     temp_tree = None
 
     # Create a long lived object
-    print " Creating a long-lived binary tree of depth %d" % kLongLivedTreeDepth
+    print(" Creating a long-lived binary tree of depth %d" % kLongLivedTreeDepth)
     long_lived_tree = Node()
     populate(kLongLivedTreeDepth, long_lived_tree)
 
     # Create long-lived array, filling half of it
-    print " Creating a long-lived array of %d doubles" % kArraySize
+    print(" Creating a long-lived array of %d doubles" % kArraySize)
     array = [0.0] * kArraySize
     i = 1
     while i < kArraySize/2:
@@ -158,15 +161,15 @@
 
     t_finish = time.time()
     print_diagnostics()
-    print "Completed in %f ms." % ((t_finish-t_start)*1000.)
+    print("Completed in %f ms." % ((t_finish-t_start)*1000.))
 
 class Failed(Exception):
     pass
 
 
 def argerror():
-    print "Usage:"
-    print "   ", USAGE
+    print("Usage:")
+    print("   ", USAGE)
     return 2
 
 def entry_point(argv):
@@ -177,7 +180,7 @@
         if arg.startswith('--threads='):
             arg = arg[len('--threads='):]
             if not ENABLE_THREADS:
-                print "threads disabled (they cannot be translated)"
+                print("threads disabled (they cannot be translated)")
                 return 1
             try:
                 threads = int(arg)

-- 
Repository URL: http://hg.python.org/benchmarks

From python-checkins at python.org  Fri Dec 16 01:24:53 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Fri, 16 Dec 2011 01:24:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_property_-=3E_staticmethod?=
Message-ID: <E1RbLbZ-00073I-TG@dinsdale.python.org>

http://hg.python.org/cpython/rev/8d6d068e9966
changeset:   73993:8d6d068e9966
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 15 19:24:49 2011 -0500
summary:
  property -> staticmethod

files:
  Doc/whatsnew/3.3.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -365,7 +365,7 @@
   * :class:`abc.abstractclassmethod` has been deprecated, use
     :class:`classmethod` with :func:`abc.abstractmethod` instead.
   * :class:`abc.abstractstaticmethod` has been deprecated, use
-    :class:`property` with :func:`abc.abstractmethod` instead.
+    :class:`staticmethod` with :func:`abc.abstractmethod` instead.
 
 (Contributed by Darren Dale in :issue:`11610`)
 

-- 
Repository URL: http://hg.python.org/cpython

From jimjjewett at gmail.com  Fri Dec 16 02:06:02 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 15 Dec 2011 20:06:02 -0500
Subject: [Python-checkins] benchmarks: Fix gcbench for Python 3
In-Reply-To: <E1RbKcV-0002ih-O9@dinsdale.python.org>
References: <E1RbKcV-0002ih-O9@dinsdale.python.org>
Message-ID: <CA+OGgf5N45xg5z8PW6ORw2GhdxTzviSzK7g_ZJS6e8ciWsu6iw@mail.gmail.com>

If this is the first time it will work with Python 3, it is probably a
good time to fix the spelling of "constrution" -> "construction"

On Thu, Dec 15, 2011 at 6:21 PM, antoine.pitrou
<python-checkins at python.org> wrote:
> http://hg.python.org/benchmarks/rev/3f1cef15cf22
> changeset: ? 154:3f1cef15cf22
> user: ? ? ? ?Antoine Pitrou <solipsis at pitrou.net>
> date: ? ? ? ?Fri Dec 16 00:21:23 2011 +0100
> summary:
> ?Fix gcbench for Python 3
>
> files:
> ?performance/gcbench.py | ?31 ++++++++++++++++-------------
> ?1 files changed, 17 insertions(+), 14 deletions(-)
>
>
> diff --git a/performance/gcbench.py b/performance/gcbench.py
> --- a/performance/gcbench.py
> +++ b/performance/gcbench.py
> @@ -43,6 +43,9 @@
> ?# ? ? ? ? ? ? ? - No attempt to measure variation with object size
> ?# ? ? ? ? ? ? ? - Results are sensitive to locking cost, but we dont
> ?# ? ? ? ? ? ? ? ? check for proper locking
> +
> +from __future__ import print_function
> +
> ?import os, time
>
> ?USAGE = """gcbench [num_repetitions] [--depths=N,N,N..] [--threads=N]"""
> @@ -67,7 +70,7 @@
>
> ?def num_iters(i):
> ? ? "Number of iterations to use for a given tree depth"
> - ? ?return 2 * tree_size(kStretchTreeDepth) / tree_size(i);
> + ? ?return 2 * tree_size(kStretchTreeDepth) // tree_size(i);
>
> ?def populate(depth, node):
> ? ? "Build tree top down, assigning to older objects."
> @@ -93,20 +96,20 @@
>
> ?def time_construction(depth):
> ? ? niters = num_iters(depth)
> - ? ?print "Creating %d trees of depth %d" % (niters, depth)
> + ? ?print("Creating %d trees of depth %d" % (niters, depth))
> ? ? t_start = time.time()
> ? ? for i in range(niters):
> ? ? ? ? temp_tree = Node()
> ? ? ? ? populate(depth, temp_tree)
> ? ? ? ? temp_tree = None
> ? ? t_finish = time.time()
> - ? ?print "\tTop down constrution took %f ms" % ((t_finish-t_start)*1000.)
> + ? ?print("\tTop down constrution took %f ms" % ((t_finish-t_start)*1000.))
> ? ? t_start = time.time()
> ? ? for i in range(niters):
> ? ? ? ? temp_tree = make_tree(depth)
> ? ? ? ? temp_tree = None
> ? ? t_finish = time.time()
> - ? ?print "\tBottom up constrution took %f ms" % ((t_finish-t_start)*1000.)
> + ? ?print("\tBottom up constrution took %f ms" % ((t_finish-t_start)*1000.))
>
> ?DEFAULT_DEPTHS = range(kMinTreeDepth, kMaxTreeDepth+1, 2)
>
> @@ -117,30 +120,30 @@
> ?def time_parallel_constructions(depths, nthreads):
> ? ? import threading
> ? ? threadlist = []
> - ? ?print "Starting %d parallel threads..." % (nthreads,)
> + ? ?print("Starting %d parallel threads..." % (nthreads,))
> ? ? for n in range(nthreads):
> ? ? ? ? t = threading.Thread(target=time_constructions, args=(depths,))
> ? ? ? ? t.start()
> ? ? ? ? threadlist.append(t)
> ? ? for t in threadlist:
> ? ? ? ? t.join()
> - ? ?print "All %d threads finished" % (nthreads,)
> + ? ?print("All %d threads finished" % (nthreads,))
>
> ?def main(depths=DEFAULT_DEPTHS, threads=0):
> - ? ?print "Garbage Collector Test"
> - ? ?print " Stretching memory with a binary tree of depth %d" % kStretchTreeDepth
> + ? ?print("Garbage Collector Test")
> + ? ?print(" Stretching memory with a binary tree of depth %d" % kStretchTreeDepth)
> ? ? print_diagnostics()
> ? ? t_start = time.time()
> ? ? temp_tree = make_tree(kStretchTreeDepth)
> ? ? temp_tree = None
>
> ? ? # Create a long lived object
> - ? ?print " Creating a long-lived binary tree of depth %d" % kLongLivedTreeDepth
> + ? ?print(" Creating a long-lived binary tree of depth %d" % kLongLivedTreeDepth)
> ? ? long_lived_tree = Node()
> ? ? populate(kLongLivedTreeDepth, long_lived_tree)
>
> ? ? # Create long-lived array, filling half of it
> - ? ?print " Creating a long-lived array of %d doubles" % kArraySize
> + ? ?print(" Creating a long-lived array of %d doubles" % kArraySize)
> ? ? array = [0.0] * kArraySize
> ? ? i = 1
> ? ? while i < kArraySize/2:
> @@ -158,15 +161,15 @@
>
> ? ? t_finish = time.time()
> ? ? print_diagnostics()
> - ? ?print "Completed in %f ms." % ((t_finish-t_start)*1000.)
> + ? ?print("Completed in %f ms." % ((t_finish-t_start)*1000.))
>
> ?class Failed(Exception):
> ? ? pass
>
>
> ?def argerror():
> - ? ?print "Usage:"
> - ? ?print " ? ", USAGE
> + ? ?print("Usage:")
> + ? ?print(" ? ", USAGE)
> ? ? return 2
>
> ?def entry_point(argv):
> @@ -177,7 +180,7 @@
> ? ? ? ? if arg.startswith('--threads='):
> ? ? ? ? ? ? arg = arg[len('--threads='):]
> ? ? ? ? ? ? if not ENABLE_THREADS:
> - ? ? ? ? ? ? ? ?print "threads disabled (they cannot be translated)"
> + ? ? ? ? ? ? ? ?print("threads disabled (they cannot be translated)")
> ? ? ? ? ? ? ? ? return 1
> ? ? ? ? ? ? try:
> ? ? ? ? ? ? ? ? threads = int(arg)
>
> --
> Repository URL: http://hg.python.org/benchmarks
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From solipsis at pitrou.net  Fri Dec 16 05:34:06 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 16 Dec 2011 05:34:06 +0100
Subject: [Python-checkins] Daily reference leaks (8d6d068e9966): sum=0
Message-ID: <E1RbPUk-0004gd-Ug@ap.vmr.nerim.net>

results for 8d6d068e9966 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog09suNZ', '-x']

From python-checkins at python.org  Fri Dec 16 11:25:42 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 11:25:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=236695=3A_Full_garba?=
 =?utf8?q?ge_collection_runs_now_clear_the_freelist_of_set_objects=2E?=
Message-ID: <E1RbUz0-0003Mb-58@dinsdale.python.org>

http://hg.python.org/cpython/rev/57f0af61da53
changeset:   73994:57f0af61da53
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 11:24:27 2011 +0100
summary:
  Issue #6695: Full garbage collection runs now clear the freelist of set objects.
Initial patch by Matthias Troffaes.

files:
  Doc/c-api/set.rst   |   7 +++++++
  Include/setobject.h |   2 ++
  Misc/NEWS           |   3 +++
  Modules/gcmodule.c  |   1 +
  Objects/setobject.c |  12 ++++++++++--
  5 files changed, 23 insertions(+), 2 deletions(-)


diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst
--- a/Doc/c-api/set.rst
+++ b/Doc/c-api/set.rst
@@ -157,3 +157,10 @@
 .. c:function:: int PySet_Clear(PyObject *set)
 
    Empty an existing set of all elements.
+
+
+.. c:function:: int PySet_ClearFreeList()
+
+   Clear the free list. Return the total number of freed items.
+
+   .. versionadded:: 3.3
diff --git a/Include/setobject.h b/Include/setobject.h
--- a/Include/setobject.h
+++ b/Include/setobject.h
@@ -99,6 +99,8 @@
 PyAPI_FUNC(PyObject *) PySet_Pop(PyObject *set);
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(int) _PySet_Update(PyObject *set, PyObject *iterable);
+
+PyAPI_FUNC(int) PySet_ClearFreeList(void);
 #endif
 
 #ifdef __cplusplus
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #6695: Full garbage collection runs now clear the freelist of set
+  objects.  Initial patch by Matthias Troffaes.
+
 - Fix OSError.__init__ and OSError.__new__ so that each of them can be
   overriden and take additional arguments (followup to issue #12555).
 
diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c
--- a/Modules/gcmodule.c
+++ b/Modules/gcmodule.c
@@ -764,6 +764,7 @@
     (void)PyFloat_ClearFreeList();
     (void)PyList_ClearFreeList();
     (void)PyDict_ClearFreeList();
+    (void)PySet_ClearFreeList();
 }
 
 static double
diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -1068,9 +1068,10 @@
     return emptyfrozenset;
 }
 
-void
-PySet_Fini(void)
+int
+PySet_ClearFreeList(void)
 {
+    int freelist_size = numfree;
     PySetObject *so;
 
     while (numfree) {
@@ -1078,6 +1079,13 @@
         so = free_list[numfree];
         PyObject_GC_Del(so);
     }
+    return freelist_size;
+}
+
+void
+PySet_Fini(void)
+{
+    PySet_ClearFreeList();
     Py_CLEAR(dummy);
     Py_CLEAR(emptyfrozenset);
 }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 11:25:42 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 11:25:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Add_ACKS_entry_for_57f0af61?=
	=?utf8?q?da53=2E?=
Message-ID: <E1RbUz0-0003Mf-UY@dinsdale.python.org>

http://hg.python.org/cpython/rev/ffe981aeb06f
changeset:   73995:ffe981aeb06f
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 11:25:15 2011 +0100
summary:
  Add ACKS entry for 57f0af61da53.

files:
  Misc/ACKS |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -988,6 +988,7 @@
 Sandro Tosi
 Richard Townsend
 Laurence Tratt
+Matthias Troffaes
 John Tromp
 Jason Trowbridge
 Anthony Tuininga

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 12:32:38 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 12:32:38 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEwMzUw?=
 =?utf8?q?=3A_Read_and_save_errno_before_calling_a_function_which_might?=
Message-ID: <E1RbW1m-0008VP-TG@dinsdale.python.org>

http://hg.python.org/cpython/rev/6a966179c73a
changeset:   73996:6a966179c73a
branch:      3.2
parent:      73985:5ec7ecf62c1d
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 12:28:32 2011 +0100
summary:
  Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.

files:
  Misc/NEWS                            |   3 +++
  Modules/_io/fileio.c                 |  14 +++++++++++---
  Modules/_multiprocessing/semaphore.c |   4 +++-
  Modules/main.c                       |   3 ++-
  Modules/readline.c                   |  13 ++++++++-----
  Modules/timemodule.c                 |   4 +++-
  Parser/myreadline.c                  |   4 +++-
  7 files changed, 33 insertions(+), 12 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #10350: Read and save errno before calling a function which might
+  overwrite it.  Original patch by Hallvard B Furuseth.
+
 - Issue #13591: A bug in importlib has been fixed that caused import_module
   to load a module twice.
 
diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -506,6 +506,7 @@
 {
     Py_buffer pbuf;
     Py_ssize_t n, len;
+    int err;
 
     if (self->fd < 0)
         return err_closed();
@@ -529,10 +530,12 @@
         Py_END_ALLOW_THREADS
     } else
         n = -1;
+    err = errno;
     PyBuffer_Release(&pbuf);
     if (n < 0) {
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
@@ -675,9 +678,11 @@
         n = -1;
 
     if (n < 0) {
+        int err = errno;
         Py_DECREF(bytes);
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
@@ -697,6 +702,7 @@
 {
     Py_buffer pbuf;
     Py_ssize_t n, len;
+    int err;
 
     if (self->fd < 0)
         return err_closed();
@@ -727,12 +733,14 @@
         Py_END_ALLOW_THREADS
     } else
         n = -1;
+    err = errno;
 
     PyBuffer_Release(&pbuf);
 
     if (n < 0) {
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
diff --git a/Modules/_multiprocessing/semaphore.c b/Modules/_multiprocessing/semaphore.c
--- a/Modules/_multiprocessing/semaphore.c
+++ b/Modules/_multiprocessing/semaphore.c
@@ -267,7 +267,7 @@
 static PyObject *
 semlock_acquire(SemLockObject *self, PyObject *args, PyObject *kwds)
 {
-    int blocking = 1, res;
+    int blocking = 1, res, err = 0;
     double timeout;
     PyObject *timeout_obj = Py_None;
     struct timespec deadline = {0};
@@ -313,11 +313,13 @@
         else
             res = sem_timedwait(self->handle, &deadline);
         Py_END_ALLOW_THREADS
+        err = errno;
         if (res == MP_EXCEPTION_HAS_BEEN_SET)
             break;
     } while (res < 0 && errno == EINTR && !PyErr_CheckSignals());
 
     if (res < 0) {
+        errno = err;
         if (errno == EAGAIN || errno == ETIMEDOUT)
             Py_RETURN_FALSE;
         else if (errno == EINTR)
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -654,13 +654,14 @@
             if (fp == NULL) {
                 char *cfilename_buffer;
                 const char *cfilename;
+                int err = errno;
                 cfilename_buffer = _Py_wchar2char(filename, NULL);
                 if (cfilename_buffer != NULL)
                     cfilename = cfilename_buffer;
                 else
                     cfilename = "<unprintable file name>";
                 fprintf(stderr, "%ls: can't open file '%s': [Errno %d] %s\n",
-                    argv[0], cfilename, errno, strerror(errno));
+                    argv[0], cfilename, err, strerror(err));
                 if (cfilename_buffer)
                     PyMem_Free(cfilename_buffer);
                 return 2;
diff --git a/Modules/readline.c b/Modules/readline.c
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -154,6 +154,7 @@
 {
     PyObject *filename_obj = Py_None, *filename_bytes;
     char *filename;
+    int err;
     if (!PyArg_ParseTuple(args, "|O:write_history_file", &filename_obj))
         return NULL;
     if (filename_obj != Py_None) {
@@ -164,10 +165,11 @@
         filename_bytes = NULL;
         filename = NULL;
     }
-    errno = write_history(filename);
-    if (!errno && _history_length >= 0)
+    errno = err = write_history(filename);
+    if (!err && _history_length >= 0)
         history_truncate_file(filename, _history_length);
     Py_XDECREF(filename_bytes);
+    errno = err;
     if (errno)
         return PyErr_SetFromErrno(PyExc_IOError);
     Py_RETURN_NONE;
@@ -970,7 +972,7 @@
     completed_input_string = not_done_reading;
 
     while (completed_input_string == not_done_reading) {
-        int has_input = 0;
+        int has_input = 0, err = 0;
 
         while (!has_input)
         {               struct timeval timeout = {0, 100000}; /* 0.1 seconds */
@@ -984,13 +986,14 @@
             /* select resets selectset if no input was available */
             has_input = select(fileno(rl_instream) + 1, &selectset,
                                NULL, NULL, timeoutp);
+            err = errno;
             if(PyOS_InputHook) PyOS_InputHook();
         }
 
-        if(has_input > 0) {
+        if (has_input > 0) {
             rl_callback_read_char();
         }
-        else if (errno == EINTR) {
+        else if (err == EINTR) {
             int s;
 #ifdef WITH_THREAD
             PyEval_RestoreThread(_PyOS_ReadlineTState);
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -527,12 +527,14 @@
      * will be ahead of time...
      */
     for (i = 1024; ; i += i) {
+        int err;
         outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char));
         if (outbuf == NULL) {
             PyErr_NoMemory();
             break;
         }
         buflen = format_time(outbuf, i, fmt, &buf);
+        err = errno;
         if (buflen > 0 || i >= 256 * fmtlen) {
             /* If the buffer is 256 times as long as the format,
                it's probably not failing for lack of room!
@@ -550,7 +552,7 @@
         PyMem_Free(outbuf);
 #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         /* VisualStudio .NET 2005 does this properly */
-        if (buflen == 0 && errno == EINVAL) {
+        if (buflen == 0 && err == EINVAL) {
             PyErr_SetString(PyExc_ValueError, "Invalid format string");
             break;
         }
diff --git a/Parser/myreadline.c b/Parser/myreadline.c
--- a/Parser/myreadline.c
+++ b/Parser/myreadline.c
@@ -36,6 +36,7 @@
 my_fgets(char *buf, int len, FILE *fp)
 {
     char *p;
+    int err;
     while (1) {
         if (PyOS_InputHook != NULL)
             (void)(PyOS_InputHook)();
@@ -44,6 +45,7 @@
         p = fgets(buf, len, fp);
         if (p != NULL)
             return 0; /* No error */
+        err = errno;
 #ifdef MS_WINDOWS
         /* In the case of a Ctrl+C or some other external event
            interrupting the operation:
@@ -78,7 +80,7 @@
             return -1; /* EOF */
         }
 #ifdef EINTR
-        if (errno == EINTR) {
+        if (err == EINTR) {
             int s;
 #ifdef WITH_THREAD
             PyEval_RestoreThread(_PyOS_ReadlineTState);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 12:32:41 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 12:32:41 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2310350=3A_Read_and_save_errno_before_calling_a_funct?=
 =?utf8?q?ion_which_might?=
Message-ID: <E1RbW1p-0008Vv-Un@dinsdale.python.org>

http://hg.python.org/cpython/rev/8e0b2e75ca7a
changeset:   73997:8e0b2e75ca7a
parent:      73995:ffe981aeb06f
parent:      73996:6a966179c73a
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 12:29:37 2011 +0100
summary:
  Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.

files:
  Misc/NEWS                            |   3 +++
  Modules/_io/fileio.c                 |  14 +++++++++++---
  Modules/_multiprocessing/semaphore.c |   4 +++-
  Modules/main.c                       |   3 ++-
  Modules/readline.c                   |  13 ++++++++-----
  Modules/timemodule.c                 |   4 +++-
  Parser/myreadline.c                  |   4 +++-
  7 files changed, 33 insertions(+), 12 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #10350: Read and save errno before calling a function which might
+  overwrite it.  Original patch by Hallvard B Furuseth.
+
 - Issue #11610: Introduce a more general way to declare abstract properties.
 
 - Issue #13591: A bug in importlib has been fixed that caused import_module
diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -530,6 +530,7 @@
 {
     Py_buffer pbuf;
     Py_ssize_t n, len;
+    int err;
 
     if (self->fd < 0)
         return err_closed();
@@ -553,10 +554,12 @@
         Py_END_ALLOW_THREADS
     } else
         n = -1;
+    err = errno;
     PyBuffer_Release(&pbuf);
     if (n < 0) {
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
@@ -726,9 +729,11 @@
         n = -1;
 
     if (n < 0) {
+        int err = errno;
         Py_DECREF(bytes);
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
@@ -748,6 +753,7 @@
 {
     Py_buffer pbuf;
     Py_ssize_t n, len;
+    int err;
 
     if (self->fd < 0)
         return err_closed();
@@ -778,12 +784,14 @@
         Py_END_ALLOW_THREADS
     } else
         n = -1;
+    err = errno;
 
     PyBuffer_Release(&pbuf);
 
     if (n < 0) {
-        if (errno == EAGAIN)
+        if (err == EAGAIN)
             Py_RETURN_NONE;
+        errno = err;
         PyErr_SetFromErrno(PyExc_IOError);
         return NULL;
     }
diff --git a/Modules/_multiprocessing/semaphore.c b/Modules/_multiprocessing/semaphore.c
--- a/Modules/_multiprocessing/semaphore.c
+++ b/Modules/_multiprocessing/semaphore.c
@@ -255,7 +255,7 @@
 static PyObject *
 semlock_acquire(SemLockObject *self, PyObject *args, PyObject *kwds)
 {
-    int blocking = 1, res;
+    int blocking = 1, res, err = 0;
     double timeout;
     PyObject *timeout_obj = Py_None;
     struct timespec deadline = {0};
@@ -301,11 +301,13 @@
         else
             res = sem_timedwait(self->handle, &deadline);
         Py_END_ALLOW_THREADS
+        err = errno;
         if (res == MP_EXCEPTION_HAS_BEEN_SET)
             break;
     } while (res < 0 && errno == EINTR && !PyErr_CheckSignals());
 
     if (res < 0) {
+        errno = err;
         if (errno == EAGAIN || errno == ETIMEDOUT)
             Py_RETURN_FALSE;
         else if (errno == EINTR)
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -655,13 +655,14 @@
             if (fp == NULL) {
                 char *cfilename_buffer;
                 const char *cfilename;
+                int err = errno;
                 cfilename_buffer = _Py_wchar2char(filename, NULL);
                 if (cfilename_buffer != NULL)
                     cfilename = cfilename_buffer;
                 else
                     cfilename = "<unprintable file name>";
                 fprintf(stderr, "%ls: can't open file '%s': [Errno %d] %s\n",
-                    argv[0], cfilename, errno, strerror(errno));
+                    argv[0], cfilename, err, strerror(err));
                 if (cfilename_buffer)
                     PyMem_Free(cfilename_buffer);
                 return 2;
diff --git a/Modules/readline.c b/Modules/readline.c
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -154,6 +154,7 @@
 {
     PyObject *filename_obj = Py_None, *filename_bytes;
     char *filename;
+    int err;
     if (!PyArg_ParseTuple(args, "|O:write_history_file", &filename_obj))
         return NULL;
     if (filename_obj != Py_None) {
@@ -164,10 +165,11 @@
         filename_bytes = NULL;
         filename = NULL;
     }
-    errno = write_history(filename);
-    if (!errno && _history_length >= 0)
+    errno = err = write_history(filename);
+    if (!err && _history_length >= 0)
         history_truncate_file(filename, _history_length);
     Py_XDECREF(filename_bytes);
+    errno = err;
     if (errno)
         return PyErr_SetFromErrno(PyExc_IOError);
     Py_RETURN_NONE;
@@ -969,7 +971,7 @@
     completed_input_string = not_done_reading;
 
     while (completed_input_string == not_done_reading) {
-        int has_input = 0;
+        int has_input = 0, err = 0;
 
         while (!has_input)
         {               struct timeval timeout = {0, 100000}; /* 0.1 seconds */
@@ -983,13 +985,14 @@
             /* select resets selectset if no input was available */
             has_input = select(fileno(rl_instream) + 1, &selectset,
                                NULL, NULL, timeoutp);
+            err = errno;
             if(PyOS_InputHook) PyOS_InputHook();
         }
 
-        if(has_input > 0) {
+        if (has_input > 0) {
             rl_callback_read_char();
         }
-        else if (errno == EINTR) {
+        else if (err == EINTR) {
             int s;
 #ifdef WITH_THREAD
             PyEval_RestoreThread(_PyOS_ReadlineTState);
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -515,12 +515,14 @@
      * will be ahead of time...
      */
     for (i = 1024; ; i += i) {
+        int err;
         outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char));
         if (outbuf == NULL) {
             PyErr_NoMemory();
             break;
         }
         buflen = format_time(outbuf, i, fmt, &buf);
+        err = errno;
         if (buflen > 0 || i >= 256 * fmtlen) {
             /* If the buffer is 256 times as long as the format,
                it's probably not failing for lack of room!
@@ -538,7 +540,7 @@
         PyMem_Free(outbuf);
 #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         /* VisualStudio .NET 2005 does this properly */
-        if (buflen == 0 && errno == EINVAL) {
+        if (buflen == 0 && err == EINVAL) {
             PyErr_SetString(PyExc_ValueError, "Invalid format string");
             break;
         }
diff --git a/Parser/myreadline.c b/Parser/myreadline.c
--- a/Parser/myreadline.c
+++ b/Parser/myreadline.c
@@ -36,6 +36,7 @@
 my_fgets(char *buf, int len, FILE *fp)
 {
     char *p;
+    int err;
     while (1) {
         if (PyOS_InputHook != NULL)
             (void)(PyOS_InputHook)();
@@ -44,6 +45,7 @@
         p = fgets(buf, len, fp);
         if (p != NULL)
             return 0; /* No error */
+        err = errno;
 #ifdef MS_WINDOWS
         /* In the case of a Ctrl+C or some other external event
            interrupting the operation:
@@ -78,7 +80,7 @@
             return -1; /* EOF */
         }
 #ifdef EINTR
-        if (errno == EINTR) {
+        if (err == EINTR) {
             int s;
 #ifdef WITH_THREAD
             PyEval_RestoreThread(_PyOS_ReadlineTState);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 14:47:06 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 14:47:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=238373=3A_The_filesy?=
 =?utf8?q?stem_path_of_AF=5FUNIX_sockets_now_uses_the_filesystem?=
Message-ID: <E1RbY7u-00042Y-Ja@dinsdale.python.org>

http://hg.python.org/cpython/rev/1f23bb74f4bc
changeset:   73998:1f23bb74f4bc
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 14:46:36 2011 +0100
summary:
  Issue #8373: The filesystem path of AF_UNIX sockets now uses the filesystem
encoding and the surrogateescape error handler, rather than UTF-8.  Patch
by David Watson.

files:
  Doc/library/socket.rst  |  18 +++++++-
  Lib/test/test_socket.py |  63 ++++++++++++++++++++++++++++-
  Misc/NEWS               |   4 +
  Modules/socketmodule.c  |  25 ++++++++--
  4 files changed, 101 insertions(+), 9 deletions(-)


diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst
--- a/Doc/library/socket.rst
+++ b/Doc/library/socket.rst
@@ -40,9 +40,23 @@
 Depending on the system and the build options, various socket families
 are supported by this module.
 
-Socket addresses are represented as follows:
+The address format required by a particular socket object is automatically
+selected based on the address family specified when the socket object was
+created.  Socket addresses are represented as follows:
 
-- A single string is used for the :const:`AF_UNIX` address family.
+- The address of an :const:`AF_UNIX` socket bound to a file system node
+  is represented as a string, using the file system encoding and the
+  ``'surrogateescape'`` error handler (see :pep:`383`).  An address in
+  Linux's abstract namespace is returned as a :class:`bytes` object with
+  an initial null byte; note that sockets in this namespace can
+  communicate with normal file system sockets, so programs intended to
+  run on Linux may need to deal with both types of address.  A string or
+  :class:`bytes` object can be used for either type of address when
+  passing it as an argument.
+
+   .. versionchanged:: 3.3
+      Previously, :const:`AF_UNIX` socket paths were assumed to use UTF-8
+      encoding.
 
 - A pair ``(host, port)`` is used for the :const:`AF_INET` address family,
   where *host* is a string representing either a hostname in Internet domain
diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1538,7 +1538,6 @@
     def _testRecvFromNegative(self):
         self.cli.sendto(MSG, 0, (HOST, self.port))
 
-
 # Tests for the sendmsg()/recvmsg() interface.  Where possible, the
 # same test code is used with different families and types of socket
 # (e.g. stream, datagram), and tests using recvmsg() are repeated
@@ -4241,6 +4240,66 @@
         with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
             self.assertRaises(socket.error, s.bind, address)
 
+    def testStrName(self):
+        # Check that an abstract name can be passed as a string.
+        s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        try:
+            s.bind("\x00python\x00test\x00")
+            self.assertEqual(s.getsockname(), b"\x00python\x00test\x00")
+        finally:
+            s.close()
+
+class TestUnixDomain(unittest.TestCase):
+
+    def setUp(self):
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+
+    def tearDown(self):
+        self.sock.close()
+
+    def encoded(self, path):
+        # Return the given path encoded in the file system encoding,
+        # or skip the test if this is not possible.
+        try:
+            return os.fsencode(path)
+        except UnicodeEncodeError:
+            self.skipTest(
+                "Pathname {0!a} cannot be represented in file "
+                "system encoding {1!r}".format(
+                    path, sys.getfilesystemencoding()))
+
+    def testStrAddr(self):
+        # Test binding to and retrieving a normal string pathname.
+        path = os.path.abspath(support.TESTFN)
+        self.sock.bind(path)
+        self.addCleanup(support.unlink, path)
+        self.assertEqual(self.sock.getsockname(), path)
+
+    def testBytesAddr(self):
+        # Test binding to a bytes pathname.
+        path = os.path.abspath(support.TESTFN)
+        self.sock.bind(self.encoded(path))
+        self.addCleanup(support.unlink, path)
+        self.assertEqual(self.sock.getsockname(), path)
+
+    def testSurrogateescapeBind(self):
+        # Test binding to a valid non-ASCII pathname, with the
+        # non-ASCII bytes supplied using surrogateescape encoding.
+        path = os.path.abspath(support.TESTFN_UNICODE)
+        b = self.encoded(path)
+        self.sock.bind(b.decode("ascii", "surrogateescape"))
+        self.addCleanup(support.unlink, path)
+        self.assertEqual(self.sock.getsockname(), path)
+
+    def testUnencodableAddr(self):
+        # Test binding to a pathname that cannot be encoded in the
+        # file system encoding.
+        if support.TESTFN_UNENCODABLE is None:
+            self.skipTest("No unencodable filename available")
+        path = os.path.abspath(support.TESTFN_UNENCODABLE)
+        self.sock.bind(path)
+        self.addCleanup(support.unlink, path)
+        self.assertEqual(self.sock.getsockname(), path)
 
 @unittest.skipUnless(thread, 'Threading required for this test.')
 class BufferIOTest(SocketConnectedTest):
@@ -4517,6 +4576,8 @@
     ])
     if hasattr(socket, "socketpair"):
         tests.append(BasicSocketPairTest)
+    if hasattr(socket, "AF_UNIX"):
+        tests.append(TestUnixDomain)
     if sys.platform == 'linux':
         tests.append(TestLinuxAbstractNamespace)
     if isTipcAvailable():
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,10 @@
 Library
 -------
 
+- Issue #8373: The filesystem path of AF_UNIX sockets now uses the filesystem
+  encoding and the surrogateescape error handler, rather than UTF-8.  Patch
+  by David Watson.
+
 - Issue #10350: Read and save errno before calling a function which might
   overwrite it.  Original patch by Hallvard B Furuseth.
 
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -1073,7 +1073,7 @@
 #endif /* linux */
         {
             /* regular NULL-terminated string */
-            return PyUnicode_FromString(a->sun_path);
+            return PyUnicode_DecodeFSDefault(a->sun_path);
         }
     }
 #endif /* AF_UNIX */
@@ -1269,8 +1269,18 @@
         struct sockaddr_un* addr;
         char *path;
         int len;
-        if (!PyArg_Parse(args, "s#", &path, &len))
-            return 0;
+        int retval = 0;
+
+        /* PEP 383.  Not using PyUnicode_FSConverter since we need to
+           allow embedded nulls on Linux. */
+        if (PyUnicode_Check(args)) {
+            if ((args = PyUnicode_EncodeFSDefault(args)) == NULL)
+                return 0;
+        }
+        else
+            Py_INCREF(args);
+        if (!PyArg_Parse(args, "y#", &path, &len))
+            goto unix_out;
 
         addr = (struct sockaddr_un*)addr_ret;
 #ifdef linux
@@ -1279,7 +1289,7 @@
             if (len > sizeof addr->sun_path) {
                 PyErr_SetString(PyExc_OSError,
                                 "AF_UNIX path too long");
-                return 0;
+                goto unix_out;
             }
         }
         else
@@ -1289,7 +1299,7 @@
             if (len >= sizeof addr->sun_path) {
                 PyErr_SetString(PyExc_OSError,
                                 "AF_UNIX path too long");
-                return 0;
+                goto unix_out;
             }
             addr->sun_path[len] = 0;
         }
@@ -1300,7 +1310,10 @@
 #else
         *len_ret = len + offsetof(struct sockaddr_un, sun_path);
 #endif
-        return 1;
+        retval = 1;
+    unix_out:
+        Py_DECREF(args);
+        return retval;
     }
 #endif /* AF_UNIX */
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 15:04:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 16 Dec 2011 15:04:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Skip_test_if_the_path_is_to?=
 =?utf8?q?o_long_for_a_AF=5FUNIX_socket?=
Message-ID: <E1RbYOt-0005dD-I3@dinsdale.python.org>

http://hg.python.org/cpython/rev/fb887a9ce2c9
changeset:   73999:fb887a9ce2c9
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 16 15:04:12 2011 +0100
summary:
  Skip test if the path is too long for a AF_UNIX socket

files:
  Lib/test/test_socket.py |  20 ++++++++++++++++----
  1 files changed, 16 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -4268,17 +4268,29 @@
                 "system encoding {1!r}".format(
                     path, sys.getfilesystemencoding()))
 
+    def bind(self, sock, path):
+        # Bind the socket
+        try:
+            sock.bind(path)
+        except OSError as e:
+            if str(e) == "AF_UNIX path too long":
+                self.skipTest(
+                    "Pathname {0!a} is too long to serve as a AF_UNIX path"
+                    .format(path))
+            else:
+                raise
+
     def testStrAddr(self):
         # Test binding to and retrieving a normal string pathname.
         path = os.path.abspath(support.TESTFN)
-        self.sock.bind(path)
+        self.bind(self.sock, path)
         self.addCleanup(support.unlink, path)
         self.assertEqual(self.sock.getsockname(), path)
 
     def testBytesAddr(self):
         # Test binding to a bytes pathname.
         path = os.path.abspath(support.TESTFN)
-        self.sock.bind(self.encoded(path))
+        self.bind(self.sock, self.encoded(path))
         self.addCleanup(support.unlink, path)
         self.assertEqual(self.sock.getsockname(), path)
 
@@ -4287,7 +4299,7 @@
         # non-ASCII bytes supplied using surrogateescape encoding.
         path = os.path.abspath(support.TESTFN_UNICODE)
         b = self.encoded(path)
-        self.sock.bind(b.decode("ascii", "surrogateescape"))
+        self.bind(self.sock, b.decode("ascii", "surrogateescape"))
         self.addCleanup(support.unlink, path)
         self.assertEqual(self.sock.getsockname(), path)
 
@@ -4297,7 +4309,7 @@
         if support.TESTFN_UNENCODABLE is None:
             self.skipTest("No unencodable filename available")
         path = os.path.abspath(support.TESTFN_UNENCODABLE)
-        self.sock.bind(path)
+        self.bind(self.sock, path)
         self.addCleanup(support.unlink, path)
         self.assertEqual(self.sock.getsockname(), path)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 23:47:05 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 16 Dec 2011 23:47:05 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogbWFpbigpIG5vdyBk?=
 =?utf8?q?isplays_an_error_message_before_exiting_if_a_command_line_argume?=
 =?utf8?q?nt?=
Message-ID: <E1RbgYT-0001Jq-2T@dinsdale.python.org>

http://hg.python.org/cpython/rev/3b17210f34d9
changeset:   74000:3b17210f34d9
branch:      3.2
parent:      73996:6a966179c73a
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 16 23:48:31 2011 +0100
summary:
  main() now displays an error message before exiting if a command line argument
cannot be decoded

files:
  Modules/python.c |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Modules/python.c b/Modules/python.c
--- a/Modules/python.c
+++ b/Modules/python.c
@@ -50,8 +50,12 @@
 #else
         argv_copy[i] = _Py_char2wchar(argv[i], NULL);
 #endif
-        if (!argv_copy[i])
+        if (!argv_copy[i]) {
+            fprintf(stderr, "Fatal Python error: "
+                            "unable to decode the command line argument #%i\n",
+                            i + 1);
             return 1;
+        }
         argv_copy2[i] = argv_copy[i];
     }
     setlocale(LC_ALL, oldloc);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 23:47:05 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 16 Dec 2011 23:47:05 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=28Merge_3=2E2=29_main=28=29_now_displays_an_error_message_b?=
 =?utf8?q?efore_exiting_if_a_command?=
Message-ID: <E1RbgYT-0001K0-T2@dinsdale.python.org>

http://hg.python.org/cpython/rev/5d305270e58f
changeset:   74001:5d305270e58f
parent:      73999:fb887a9ce2c9
parent:      74000:3b17210f34d9
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 16 23:48:55 2011 +0100
summary:
  (Merge 3.2) main() now displays an error message before exiting if a command
line argument cannot be decoded

files:
  Modules/python.c |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Modules/python.c b/Modules/python.c
--- a/Modules/python.c
+++ b/Modules/python.c
@@ -50,8 +50,12 @@
 #else
         argv_copy[i] = _Py_char2wchar(argv[i], NULL);
 #endif
-        if (!argv_copy[i])
+        if (!argv_copy[i]) {
+            fprintf(stderr, "Fatal Python error: "
+                            "unable to decode the command line argument #%i\n",
+                            i + 1);
             return 1;
+        }
         argv_copy2[i] = argv_copy[i];
     }
     setlocale(LC_ALL, oldloc);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 16 23:57:02 2011
From: python-checkins at python.org (victor.stinner)
Date: Fri, 16 Dec 2011 23:57:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Add_PyUnicode=5FDecodeLocal?=
 =?utf8?q?eAndSize=28=29_and_PyUnicode=5FDecodeLocale=28=29?=
Message-ID: <E1Rbgi6-0001xw-4S@dinsdale.python.org>

http://hg.python.org/cpython/rev/279b0aee0cfb
changeset:   74002:279b0aee0cfb
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Fri Dec 16 23:56:01 2011 +0100
summary:
  Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()

 * PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale() decode a string
   from the current locale encoding
 * _Py_char2wchar() writes an "error code" in the size argument to indicate
   if the function failed because of memory allocation failure or because of a
   decoding error. The function doesn't write the error message directly to
   stderr.
 * Fix time.strftime() (if wcsftime() is missing): decode strftime() result
   from the current locale encoding, not from the filesystem encoding.

files:
  Doc/c-api/unicode.rst   |  40 ++++++++++++
  Include/unicodeobject.h |  22 ++++++
  Modules/_localemodule.c |  57 +++--------------
  Modules/main.c          |  15 +--
  Modules/timemodule.c    |   6 +-
  Objects/unicodeobject.c |  95 +++++++++++++++++++++++-----
  Python/fileutils.c      |  25 ++++--
  7 files changed, 175 insertions(+), 85 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -699,6 +699,39 @@
    throughout the interpreter whenever coercion to Unicode is needed.
 
 
+Locale Encoding
+"""""""""""""""
+
+The current locale encoding can be used to decode text from the operating
+system.
+
+.. c:function:: PyObject* PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t len, int surrogateescape)
+
+   Decode a string from the current locale encoding. The decoder is strict if
+   *surrogateescape* is equal to zero, otherwise it uses the
+   ``'surrogateescape'`` error handler (:pep:`383`) to escape undecodable
+   bytes. If a byte sequence can be decoded as a surrogate character and
+   *surrogateescape* is not equal to zero, the byte sequence is escaped using
+   the ``'surrogateescape'`` error handler instead of being decoded.  *str*
+   must end with a null character but cannot contain embedded null character.
+
+   .. seealso::
+
+      Use :c:func:`PyUnicode_DecodeFSDefaultAndSize` to decode a string from
+      :c:data:`Py_FileSystemDefaultEncoding` (the locale encoding read at
+      Python startup).
+
+   .. versionadded:: 3.3
+
+
+.. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, int surrogateescape)
+
+   Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string
+   length using :c:func:`strlen`.
+
+   .. versionadded:: 3.3
+
+
 File System Encoding
 """"""""""""""""""""
 
@@ -739,6 +772,13 @@
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
 
+   .. seealso::
+
+      :c:data:`Py_FileSystemDefaultEncoding` is initialized at startup from the
+      locale encoding and cannot be modified later. If you need to decode a
+      string from the current locale encoding, use
+      :c:func:`PyUnicode_DecodeLocaleAndSize`.
+
    .. versionchanged:: 3.2
       Use ``'strict'`` error handler on Windows.
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -1595,6 +1595,28 @@
     );
 #endif
 
+/* --- Locale encoding --------------------------------------------------- */
+
+/* Decode a string from the current locale encoding. The decoder is strict if
+   *surrogateescape* is equal to zero, otherwise it uses the 'surrogateescape'
+   error handler (PEP 383) to escape undecodable bytes. If a byte sequence can
+   be decoded as a surrogate character and *surrogateescape* is not equal to
+   zero, the byte sequence is escaped using the 'surrogateescape' error handler
+   instead of being decoded. *str* must end with a null character but cannot
+   contain embedded null character. */
+
+PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocaleAndSize(
+    const char *str,
+    Py_ssize_t len,
+    int surrogateescape);
+
+/* Similar to PyUnicode_DecodeLocaleAndSize(), but compute the string
+   length using strlen(). */
+
+PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocale(
+    const char *str,
+    int surrogateescape);
+
 /* --- File system encoding ---------------------------------------------- */
 
 /* ParseTuple converter: encode str objects to bytes using
diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c
--- a/Modules/_localemodule.c
+++ b/Modules/_localemodule.c
@@ -42,43 +42,6 @@
 
 static PyObject *Error;
 
-/* Convert a char* to a Unicode object according to the current locale */
-static PyObject*
-str2uni(const char* s)
-{
-#ifdef HAVE_BROKEN_MBSTOWCS
-    size_t needed = strlen(s);
-#else
-    size_t needed = mbstowcs(NULL, s, 0);
-#endif
-    size_t res1;
-    wchar_t smallbuf[30];
-    wchar_t *dest;
-    PyObject *res2;
-    if (needed == (size_t)-1) {
-        PyErr_SetString(PyExc_ValueError, "Cannot convert byte to string");
-        return NULL;
-    }
-    if (needed*sizeof(wchar_t) < sizeof(smallbuf))
-        dest = smallbuf;
-    else {
-        dest = PyMem_Malloc((needed+1)*sizeof(wchar_t));
-        if (!dest)
-            return PyErr_NoMemory();
-    }
-    /* This shouldn't fail now */
-    res1 = mbstowcs(dest, s, needed+1);
-#ifdef HAVE_BROKEN_MBSTOWCS
-    assert(res1 != (size_t)-1);
-#else
-    assert(res1 == needed);
-#endif
-    res2 = PyUnicode_FromWideChar(dest, res1);
-    if (dest != smallbuf)
-        PyMem_Free(dest);
-    return res2;
-}
-
 /* support functions for formatting floating point numbers */
 
 PyDoc_STRVAR(setlocale__doc__,
@@ -149,7 +112,7 @@
             PyErr_SetString(Error, "unsupported locale setting");
             return NULL;
         }
-        result_object = str2uni(result);
+        result_object = PyUnicode_DecodeLocale(result, 0);
         if (!result_object)
             return NULL;
     } else {
@@ -159,7 +122,7 @@
             PyErr_SetString(Error, "locale query failed");
             return NULL;
         }
-        result_object = str2uni(result);
+        result_object = PyUnicode_DecodeLocale(result, 0);
     }
     return result_object;
 }
@@ -185,7 +148,7 @@
        involved herein */
 
 #define RESULT_STRING(s)\
-    x = str2uni(l->s);   \
+    x = PyUnicode_DecodeLocale(l->s, 0);   \
     if (!x) goto failed;\
     PyDict_SetItemString(result, #s, x);\
     Py_XDECREF(x)
@@ -476,7 +439,7 @@
                instead of an empty string for nl_langinfo(ERA).  */
             const char *result = nl_langinfo(item);
             result = result != NULL ? result : "";
-            return str2uni(result);
+            return PyUnicode_DecodeLocale(result, 0);
         }
     PyErr_SetString(PyExc_ValueError, "unsupported langinfo constant");
     return NULL;
@@ -495,7 +458,7 @@
     char *in;
     if (!PyArg_ParseTuple(args, "s", &in))
         return 0;
-    return str2uni(gettext(in));
+    return PyUnicode_DecodeLocale(gettext(in), 0);
 }
 
 PyDoc_STRVAR(dgettext__doc__,
@@ -508,7 +471,7 @@
     char *domain, *in;
     if (!PyArg_ParseTuple(args, "zs", &domain, &in))
         return 0;
-    return str2uni(dgettext(domain, in));
+    return PyUnicode_DecodeLocale(dgettext(domain, in), 0);
 }
 
 PyDoc_STRVAR(dcgettext__doc__,
@@ -522,7 +485,7 @@
     int category;
     if (!PyArg_ParseTuple(args, "zsi", &domain, &msgid, &category))
         return 0;
-    return str2uni(dcgettext(domain,msgid,category));
+    return PyUnicode_DecodeLocale(dcgettext(domain,msgid,category), 0);
 }
 
 PyDoc_STRVAR(textdomain__doc__,
@@ -540,7 +503,7 @@
         PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
-    return str2uni(domain);
+    return PyUnicode_DecodeLocale(domain, 0);
 }
 
 PyDoc_STRVAR(bindtextdomain__doc__,
@@ -572,7 +535,7 @@
         PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
-    result = str2uni(current_dirname);
+    result = PyUnicode_DecodeLocale(current_dirname, 0);
     Py_XDECREF(dirname_bytes);
     return result;
 }
@@ -590,7 +553,7 @@
         return NULL;
     codeset = bind_textdomain_codeset(domain, codeset);
     if (codeset)
-        return str2uni(codeset);
+        return PyUnicode_DecodeLocale(codeset, 0);
     Py_RETURN_NONE;
 }
 #endif
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -495,16 +495,13 @@
             /* Use utf-8 on Mac OS X */
             unicode = PyUnicode_FromString(p);
 #else
-            wchar_t *wchar;
-            size_t len;
-            wchar = _Py_char2wchar(p, &len);
-            if (wchar == NULL)
+            unicode = PyUnicode_DecodeLocale(p, 1);
+#endif
+            if (unicode == NULL) {
+                /* ignore errors */
+                PyErr_Clear();
                 continue;
-            unicode = PyUnicode_FromWideChar(wchar, len);
-            PyMem_Free(wchar);
-#endif
-            if (unicode == NULL)
-                continue;
+            }
             PySys_AddWarnOptionUnicode(unicode);
             Py_DECREF(unicode);
         }
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -532,7 +532,7 @@
 #ifdef HAVE_WCSFTIME
             ret = PyUnicode_FromWideChar(outbuf, buflen);
 #else
-            ret = PyUnicode_DecodeFSDefaultAndSize(outbuf, buflen);
+            ret = PyUnicode_DecodeLocaleAndSize(outbuf, buflen, 1);
 #endif
             PyMem_Free(outbuf);
             break;
@@ -764,8 +764,8 @@
 #endif /* PYOS_OS2 */
 #endif
     PyModule_AddIntConstant(m, "daylight", daylight);
-    otz0 = PyUnicode_DecodeFSDefaultAndSize(tzname[0], strlen(tzname[0]));
-    otz1 = PyUnicode_DecodeFSDefaultAndSize(tzname[1], strlen(tzname[1]));
+    otz0 = PyUnicode_DecodeLocale(tzname[0], 1);
+    otz1 = PyUnicode_DecodeLocale(tzname[1], 1);
     PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1));
 #else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/
 #ifdef HAVE_STRUCT_TM_TM_ZONE
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3235,6 +3235,83 @@
 }
 
 PyObject*
+PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t len,
+                              int surrogateescape)
+{
+    wchar_t smallbuf[256];
+    size_t smallbuf_len = Py_ARRAY_LENGTH(smallbuf);
+    wchar_t *wstr;
+    size_t wlen, wlen2;
+    PyObject *unicode;
+
+    if (str[len] != '\0' || len != strlen(str)) {
+        PyErr_SetString(PyExc_TypeError, "embedded null character");
+        return NULL;
+    }
+
+    if (surrogateescape)
+    {
+        wstr = _Py_char2wchar(str, &wlen);
+        if (wstr == NULL) {
+            if (wlen == (size_t)-1)
+                PyErr_NoMemory();
+            else
+                PyErr_SetFromErrno(PyExc_OSError);
+            return NULL;
+        }
+
+        unicode = PyUnicode_FromWideChar(wstr, wlen);
+        PyMem_Free(wstr);
+    }
+    else {
+#ifndef HAVE_BROKEN_MBSTOWCS
+        wlen = mbstowcs(NULL, str, 0);
+#else
+        wlen = len;
+#endif
+        if (wlen == (size_t)-1) {
+            PyErr_SetFromErrno(PyExc_OSError);
+            return NULL;
+        }
+        if (wlen+1 <= smallbuf_len) {
+            wstr = smallbuf;
+        }
+        else {
+            if (wlen > PY_SSIZE_T_MAX / sizeof(wchar_t) - 1)
+                return PyErr_NoMemory();
+
+            wstr = PyMem_Malloc((wlen+1) * sizeof(wchar_t));
+            if (!wstr)
+                return PyErr_NoMemory();
+        }
+
+        /* This shouldn't fail now */
+        wlen2 = mbstowcs(wstr, str, wlen+1);
+        if (wlen2 == (size_t)-1) {
+            if (wstr != smallbuf)
+                PyMem_Free(wstr);
+            PyErr_SetFromErrno(PyExc_OSError);
+            return NULL;
+        }
+#ifdef HAVE_BROKEN_MBSTOWCS
+        assert(wlen2 == wlen);
+#endif
+        unicode = PyUnicode_FromWideChar(wstr, wlen2);
+        if (wstr != smallbuf)
+            PyMem_Free(wstr);
+    }
+    return unicode;
+}
+
+PyObject*
+PyUnicode_DecodeLocale(const char *str, int surrogateescape)
+{
+    Py_ssize_t size = (Py_ssize_t)strlen(str);
+    return PyUnicode_DecodeLocaleAndSize(str, size, surrogateescape);
+}
+
+
+PyObject*
 PyUnicode_DecodeFSDefault(const char *s) {
     Py_ssize_t size = (Py_ssize_t)strlen(s);
     return PyUnicode_DecodeFSDefaultAndSize(s, size);
@@ -3264,23 +3341,7 @@
                                 "surrogateescape");
     }
     else {
-        /* locale encoding with surrogateescape */
-        wchar_t *wchar;
-        PyObject *unicode;
-        size_t len;
-
-        if (s[size] != '\0' || size != strlen(s)) {
-            PyErr_SetString(PyExc_TypeError, "embedded NUL character");
-            return NULL;
-        }
-
-        wchar = _Py_char2wchar(s, &len);
-        if (wchar == NULL)
-            return PyErr_NoMemory();
-
-        unicode = PyUnicode_FromWideChar(wchar, len);
-        PyMem_Free(wchar);
-        return unicode;
+        return PyUnicode_DecodeLocaleAndSize(s, size, 1);
     }
 #endif
 }
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -16,7 +16,9 @@
    Return a pointer to a newly allocated wide character string (use
    PyMem_Free() to free the memory) and write the number of written wide
    characters excluding the null character into *size if size is not NULL, or
-   NULL on error (conversion or memory allocation error).
+   NULL on error (decoding or memory allocation error). If size is not NULL,
+   *size is set to (size_t)-1 on memory error and (size_t)-2 on decoding
+   error.
 
    Conversion errors should never happen, unless there is a bug in the C
    library. */
@@ -82,8 +84,9 @@
                since we provide everything that we have -
                unless there is a bug in the C library, or I
                misunderstood how mbrtowc works. */
-            fprintf(stderr, "unexpected mbrtowc result -2\n");
             PyMem_Free(res);
+            if (size != NULL)
+                *size = (size_t)-2;
             return NULL;
         }
         if (converted == (size_t)-1) {
@@ -112,7 +115,8 @@
        is ASCII (i.e. escape all bytes > 128. This will still roundtrip
        correctly in the locale's charset, which must be an ASCII superset. */
     res = PyMem_Malloc((strlen(arg)+1)*sizeof(wchar_t));
-    if (!res) goto oom;
+    if (!res)
+        goto oom;
     in = (unsigned char*)arg;
     out = res;
     while(*in)
@@ -126,7 +130,8 @@
         *size = out - res;
     return res;
 oom:
-    fprintf(stderr, "out of memory\n");
+    if (size != NULL)
+        *size = (size_t)-1;
     return NULL;
 }
 
@@ -137,10 +142,10 @@
    This function is the reverse of _Py_char2wchar().
 
    Return a pointer to a newly allocated byte string (use PyMem_Free() to free
-   the memory), or NULL on conversion or memory allocation error.
+   the memory), or NULL on encoding or memory allocation error.
 
    If error_pos is not NULL: *error_pos is the index of the invalid character
-   on conversion error, or (size_t)-1 otherwise. */
+   on encoding error, or (size_t)-1 otherwise. */
 char*
 _Py_wchar2char(const wchar_t *text, size_t *error_pos)
 {
@@ -328,7 +333,7 @@
 #ifdef HAVE_READLINK
 
 /* Read value of symbolic link. Encode the path to the locale encoding, decode
-   the result from the locale encoding. */
+   the result from the locale encoding. Return -1 on error. */
 
 int
 _Py_wreadlink(const wchar_t *path, wchar_t *buf, size_t bufsiz)
@@ -372,7 +377,8 @@
 #ifdef HAVE_REALPATH
 
 /* Return the canonicalized absolute pathname. Encode path to the locale
-   encoding, decode the result from the locale encoding. */
+   encoding, decode the result from the locale encoding.
+   Return NULL on error. */
 
 wchar_t*
 _Py_wrealpath(const wchar_t *path,
@@ -410,7 +416,8 @@
 #endif
 
 /* Get the current directory. size is the buffer size in wide characters
-   including the null character. Decode the path from the locale encoding. */
+   including the null character. Decode the path from the locale encoding.
+   Return NULL on error. */
 
 wchar_t*
 _Py_wgetcwd(wchar_t *buf, size_t size)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 00:20:21 2011
From: python-checkins at python.org (ezio.melotti)
Date: Sat, 17 Dec 2011 00:20:21 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNjEzOiBmaXgg?=
	=?utf8?q?example_in_re_doc=2E?=
Message-ID: <E1Rbh4f-0003mP-Od@dinsdale.python.org>

http://hg.python.org/cpython/rev/766a21ebf82e
changeset:   74003:766a21ebf82e
branch:      2.7
parent:      73987:f2a5dcced66d
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sat Dec 17 01:17:17 2011 +0200
summary:
  #13613: fix example in re doc.

files:
  Doc/library/re.rst |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/library/re.rst b/Doc/library/re.rst
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -1000,16 +1000,16 @@
 
 Suppose you are writing a poker program where a player's hand is represented as
 a 5-character string with each character representing a card, "a" for ace, "k"
-for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
+for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9"
 representing the card with that value.
 
 To see if a given string is a valid hand, one could do the following:
 
-   >>> valid = re.compile(r"[0-9akqj]{5}$")
-   >>> displaymatch(valid.match("ak05q"))  # Valid.
-   "<Match: 'ak05q', groups=()>"
-   >>> displaymatch(valid.match("ak05e"))  # Invalid.
-   >>> displaymatch(valid.match("ak0"))    # Invalid.
+   >>> valid = re.compile(r"^[a2-9tjqk]{5}$")
+   >>> displaymatch(valid.match("akt5q"))  # Valid.
+   "<Match: 'akt5q', groups=()>"
+   >>> displaymatch(valid.match("akt5e"))  # Invalid.
+   >>> displaymatch(valid.match("akt"))    # Invalid.
    >>> displaymatch(valid.match("727ak"))  # Valid.
    "<Match: '727ak', groups=()>"
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 00:20:22 2011
From: python-checkins at python.org (ezio.melotti)
Date: Sat, 17 Dec 2011 00:20:22 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNjEzOiBmaXgg?=
	=?utf8?q?example_in_re_doc=2E?=
Message-ID: <E1Rbh4g-0003mU-HY@dinsdale.python.org>

http://hg.python.org/cpython/rev/0b86da9d6964
changeset:   74004:0b86da9d6964
branch:      3.2
parent:      74000:3b17210f34d9
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sat Dec 17 01:17:17 2011 +0200
summary:
  #13613: fix example in re doc.

files:
  Doc/library/re.rst |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/library/re.rst b/Doc/library/re.rst
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -1018,16 +1018,16 @@
 
 Suppose you are writing a poker program where a player's hand is represented as
 a 5-character string with each character representing a card, "a" for ace, "k"
-for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
+for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9"
 representing the card with that value.
 
 To see if a given string is a valid hand, one could do the following:
 
-   >>> valid = re.compile(r"[0-9akqj]{5}$")
-   >>> displaymatch(valid.match("ak05q"))  # Valid.
-   "<Match: 'ak05q', groups=()>"
-   >>> displaymatch(valid.match("ak05e"))  # Invalid.
-   >>> displaymatch(valid.match("ak0"))    # Invalid.
+   >>> valid = re.compile(r"^[a2-9tjqk]{5}$")
+   >>> displaymatch(valid.match("akt5q"))  # Valid.
+   "<Match: 'akt5q', groups=()>"
+   >>> displaymatch(valid.match("akt5e"))  # Invalid.
+   >>> displaymatch(valid.match("akt"))    # Invalid.
    >>> displaymatch(valid.match("727ak"))  # Valid.
    "<Match: '727ak', groups=()>"
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 00:20:23 2011
From: python-checkins at python.org (ezio.melotti)
Date: Sat, 17 Dec 2011 00:20:23 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313613=3A_merge_with_3=2E2=2E?=
Message-ID: <E1Rbh4h-0003n0-9D@dinsdale.python.org>

http://hg.python.org/cpython/rev/f2e1867f33b8
changeset:   74005:f2e1867f33b8
parent:      74002:279b0aee0cfb
parent:      74004:0b86da9d6964
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sat Dec 17 01:20:02 2011 +0200
summary:
  #13613: merge with 3.2.

files:
  Doc/library/re.rst |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/library/re.rst b/Doc/library/re.rst
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -1021,16 +1021,16 @@
 
 Suppose you are writing a poker program where a player's hand is represented as
 a 5-character string with each character representing a card, "a" for ace, "k"
-for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
+for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9"
 representing the card with that value.
 
 To see if a given string is a valid hand, one could do the following:
 
-   >>> valid = re.compile(r"[0-9akqj]{5}$")
-   >>> displaymatch(valid.match("ak05q"))  # Valid.
-   "<Match: 'ak05q', groups=()>"
-   >>> displaymatch(valid.match("ak05e"))  # Invalid.
-   >>> displaymatch(valid.match("ak0"))    # Invalid.
+   >>> valid = re.compile(r"^[a2-9tjqk]{5}$")
+   >>> displaymatch(valid.match("akt5q"))  # Valid.
+   "<Match: 'akt5q', groups=()>"
+   >>> displaymatch(valid.match("akt5e"))  # Invalid.
+   >>> displaymatch(valid.match("akt"))    # Invalid.
    >>> displaymatch(valid.match("727ak"))  # Valid.
    "<Match: '727ak', groups=()>"
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 04:46:06 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 04:46:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313560=3A_Add_PyUni?=
	=?utf8?q?code=5FEncodeLocale=28=29?=
Message-ID: <E1RblDq-0002y3-Sy@dinsdale.python.org>

http://hg.python.org/cpython/rev/88198b93ff2f
changeset:   74006:88198b93ff2f
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 04:13:41 2011 +0100
summary:
  Issue #13560: Add PyUnicode_EncodeLocale()

 * Use PyUnicode_EncodeLocale() in time.strftime() if wcsftime() is not
   available
 * Document my last changes in Misc/NEWS

files:
  Doc/c-api/unicode.rst   |   25 ++++-
  Include/unicodeobject.h |   12 +-
  Misc/NEWS               |    8 +-
  Modules/timemodule.c    |    2 +-
  Objects/unicodeobject.c |  167 ++++++++++++++++++++++-----
  5 files changed, 177 insertions(+), 37 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -713,7 +713,7 @@
    bytes. If a byte sequence can be decoded as a surrogate character and
    *surrogateescape* is not equal to zero, the byte sequence is escaped using
    the ``'surrogateescape'`` error handler instead of being decoded.  *str*
-   must end with a null character but cannot contain embedded null character.
+   must end with a null character but cannot contain embedded null characters.
 
    .. seealso::
 
@@ -732,6 +732,22 @@
    .. versionadded:: 3.3
 
 
+.. c:function:: PyObject* PyUnicode_EncodeLocale(PyObject *unicode, int surrogateescape)
+
+   Encode a Unicode object to the current locale encoding. The encoder is
+   strict if *surrogateescape* is equal to zero, otherwise it uses the
+   ``'surrogateescape'`` error handler (:pep:`383`). Return a :class:`bytes`
+   object. *str* cannot contain embedded null characters.
+
+   .. seealso::
+
+      Use :c:func:`PyUnicode_EncodeFSDefault` to encode a string to
+      :c:data:`Py_FileSystemDefaultEncoding` (the locale encoding read at
+      Python startup).
+
+   .. versionadded:: 3.3
+
+
 File System Encoding
 """"""""""""""""""""
 
@@ -806,6 +822,13 @@
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
 
+   .. seealso::
+
+      :c:data:`Py_FileSystemDefaultEncoding` is initialized at startup from the
+      locale encoding and cannot be modified later. If you need to encode a
+      string to the current locale encoding, use
+      :c:func:`PyUnicode_EncodeLocale`.
+
    .. versionadded:: 3.2
 
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -1603,7 +1603,7 @@
    be decoded as a surrogate character and *surrogateescape* is not equal to
    zero, the byte sequence is escaped using the 'surrogateescape' error handler
    instead of being decoded. *str* must end with a null character but cannot
-   contain embedded null character. */
+   contain embedded null characters. */
 
 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocaleAndSize(
     const char *str,
@@ -1617,6 +1617,16 @@
     const char *str,
     int surrogateescape);
 
+/* Encode a Unicode object to the current locale encoding. The encoder is
+   strict is *surrogateescape* is equal to zero, otherwise the
+   "surrogateescape" error handler is used. Return a bytes object. The string
+   cannot contain embedded null characters.. */
+
+PyAPI_FUNC(PyObject*) PyUnicode_EncodeLocale(
+    PyObject *unicode,
+    int surrogateescape
+    );
+
 /* --- File system encoding ---------------------------------------------- */
 
 /* ParseTuple converter: encode str objects to bytes using
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,10 @@
 Library
 -------
 
+- Issue #13560: Add PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
+  and PyUnicode_EncodeLocale() functions to the C API to decode/encode from/to
+  the current locale encoding.
+
 - Issue #8373: The filesystem path of AF_UNIX sockets now uses the filesystem
   encoding and the surrogateescape error handler, rather than UTF-8.  Patch
   by David Watson.
@@ -451,8 +455,8 @@
   'importlib.abc.PyPycLoader', 'nntplib.NNTP.xgtitle', 'nntplib.NNTP.xpath',
   and private attributes of 'smtpd.SMTPChannel'.
 
-- Issue #5905: time.strftime() is now using the locale encoding, instead of
-  UTF-8, if the wcsftime() function is not available.
+- Issue #5905, #13560: time.strftime() is now using the current locale
+  encoding, instead of UTF-8, if the wcsftime() function is not available.
 
 - Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..".
   Patch by Tal Einat.
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -486,7 +486,7 @@
     fmt = format;
 #else
     /* Convert the unicode string to an ascii one */
-    format = PyUnicode_EncodeFSDefault(format_arg);
+    format = PyUnicode_EncodeLocale(format_arg, 1);
     if (format == NULL)
         return NULL;
     fmt = PyBytes_AS_STRING(format);
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3073,6 +3073,140 @@
     return NULL;
 }
 
+static size_t
+wcstombs_errorpos(const wchar_t *wstr)
+{
+    size_t len;
+#if SIZEOF_WCHAR_T == 2
+    wchar_t buf[3];
+#else
+    wchar_t buf[2];
+#endif
+    char outbuf[MB_LEN_MAX];
+    const wchar_t *start, *previous;
+    int save_errno;
+
+    save_errno = errno;
+#if SIZEOF_WCHAR_T == 2
+    buf[2] = 0;
+#else
+    buf[1] = 0;
+#endif
+    start = wstr;
+    while (*wstr != L'\0')
+    {
+        previous = wstr;
+#if SIZEOF_WCHAR_T == 2
+        if (Py_UNICODE_IS_HIGH_SURROGATE(wstr[0])
+            && Py_UNICODE_IS_LOW_SURROGATE(wstr[1]))
+        {
+            buf[0] = wstr[0];
+            buf[1] = wstr[1];
+            wstr += 2;
+        }
+        else {
+            buf[0] = *wstr;
+            buf[1] = 0;
+            wstr++;
+        }
+#else
+        buf[0] = *wstr;
+        wstr++;
+#endif
+        len = wcstombs(outbuf, buf, sizeof(outbuf));
+        if (len == (size_t)-1) {
+            errno = save_errno;
+            return previous - start;
+        }
+    }
+
+    /* failed to find the unencodable character */
+    errno = save_errno;
+    return 0;
+}
+
+PyObject *
+PyUnicode_EncodeLocale(PyObject *unicode, int surrogateescape)
+{
+    Py_ssize_t wlen, wlen2;
+    wchar_t *wstr;
+    PyObject *bytes = NULL;
+    char *errmsg;
+    PyObject *exc;
+    size_t error_pos;
+
+    wstr = PyUnicode_AsWideCharString(unicode, &wlen);
+    if (wstr == NULL)
+        return NULL;
+
+    wlen2 = wcslen(wstr);
+    if (wlen2 != wlen) {
+        PyMem_Free(wstr);
+        PyErr_SetString(PyExc_TypeError, "embedded null character");
+        return NULL;
+    }
+
+    if (surrogateescape) {
+        /* locale encoding with surrogateescape */
+        char *str;
+
+        str = _Py_wchar2char(wstr, &error_pos);
+        if (str == NULL) {
+            if (error_pos == (size_t)-1) {
+                PyErr_NoMemory();
+                PyMem_Free(wstr);
+                return NULL;
+            }
+            else {
+                goto encode_error;
+            }
+        }
+        PyMem_Free(wstr);
+
+        bytes = PyBytes_FromString(str);
+        PyMem_Free(str);
+    }
+    else {
+        size_t len, len2;
+
+        len = wcstombs(NULL, wstr, 0);
+        if (len == (size_t)-1) {
+            error_pos = wcstombs_errorpos(wstr);
+            goto encode_error;
+        }
+
+        bytes = PyBytes_FromStringAndSize(NULL, len);
+        if (bytes == NULL) {
+            PyMem_Free(wstr);
+            return NULL;
+        }
+
+        len2 = wcstombs(PyBytes_AS_STRING(bytes), wstr, len+1);
+        if (len2 == (size_t)-1 || len2 > len) {
+            error_pos = wcstombs_errorpos(wstr);
+            goto encode_error;
+        }
+        PyMem_Free(wstr);
+    }
+    return bytes;
+
+encode_error:
+    errmsg = strerror(errno);
+    assert(errmsg != NULL);
+    if (errmsg == NULL)
+        errmsg = "wcstombs() encountered an unencodable wide character";
+    PyMem_Free(wstr);
+    Py_XDECREF(bytes);
+
+    exc = NULL;
+    raise_encode_exception(&exc,
+        "locale", unicode,
+        error_pos, error_pos+1,
+        errmsg);
+    Py_XDECREF(exc);
+    return NULL;
+}
+
 PyObject *
 PyUnicode_EncodeFSDefault(PyObject *unicode)
 {
@@ -3097,38 +3231,7 @@
                                          "surrogateescape");
     }
     else {
-        /* locale encoding with surrogateescape */
-        wchar_t *wchar;
-        char *bytes;
-        PyObject *bytes_obj;
-        size_t error_pos;
-
-        wchar = PyUnicode_AsWideCharString(unicode, NULL);
-        if (wchar == NULL)
-            return NULL;
-        bytes = _Py_wchar2char(wchar, &error_pos);
-        if (bytes == NULL) {
-            if (error_pos != (size_t)-1) {
-                char *errmsg = strerror(errno);
-                PyObject *exc = NULL;
-                if (errmsg == NULL)
-                    errmsg = "Py_wchar2char() failed";
-                raise_encode_exception(&exc,
-                    "filesystemencoding", unicode,
-                    error_pos, error_pos+1,
-                    errmsg);
-                Py_XDECREF(exc);
-            }
-            else
-                PyErr_NoMemory();
-            PyMem_Free(wchar);
-            return NULL;
-        }
-        PyMem_Free(wchar);
-
-        bytes_obj = PyBytes_FromString(bytes);
-        PyMem_Free(bytes);
-        return bytes_obj;
+        return PyUnicode_EncodeLocale(unicode, 1);
     }
 #endif
 }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 04:46:07 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 04:46:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313560=3A_os=2Estre?=
 =?utf8?q?rror=28=29_now_uses_the_current_locale_encoding_instead_of?=
Message-ID: <E1RblDr-0002y8-Q9@dinsdale.python.org>

http://hg.python.org/cpython/rev/51412b4b81ae
changeset:   74007:51412b4b81ae
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 04:45:09 2011 +0100
summary:
  Issue #13560: os.strerror() now uses the current locale encoding instead of UTF-8

files:
  Misc/NEWS               |   3 +++
  Modules/posixmodule.c   |   2 +-
  Modules/socketmodule.c  |   5 ++---
  Objects/unicodeobject.c |  28 ++++++++++++++++++++--------
  Python/errors.c         |  17 +++++++++--------
  5 files changed, 35 insertions(+), 20 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #13560: os.strerror() now uses the current locale encoding instead of
+  UTF-8.
+
 - Issue #13560: Add PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
   and PyUnicode_EncodeLocale() functions to the C API to decode/encode from/to
   the current locale encoding.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -7891,7 +7891,7 @@
                         "strerror() argument out of range");
         return NULL;
     }
-    return PyUnicode_FromString(message);
+    return PyUnicode_DecodeLocale(message, 1);
 }
 
 
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4032,9 +4032,8 @@
 
     if (h->h_addrtype != af) {
         /* Let's get real error message to return */
-        PyErr_SetString(PyExc_OSError,
-                        (char *)strerror(EAFNOSUPPORT));
-
+        errno = EAFNOSUPPORT;
+        PyErr_SetFromErrno(PyExc_OSError);
         return NULL;
     }
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3132,6 +3132,7 @@
     wchar_t *wstr;
     PyObject *bytes = NULL;
     char *errmsg;
+    PyObject *reason;
     PyObject *exc;
     size_t error_pos;
 
@@ -3193,17 +3194,28 @@
 encode_error:
     errmsg = strerror(errno);
     assert(errmsg != NULL);
-    if (errmsg == NULL)
-        errmsg = "wcstombs() encountered an unencodable wide character";
     PyMem_Free(wstr);
     Py_XDECREF(bytes);
 
-    exc = NULL;
-    raise_encode_exception(&exc,
-        "locale", unicode,
-        error_pos, error_pos+1,
-        errmsg);
-    Py_XDECREF(exc);
+    if (errmsg != NULL)
+        reason = PyUnicode_DecodeLocale(errmsg, 1);
+    else
+        reason = PyUnicode_FromString(
+            "wcstombs() encountered an unencodable "
+            "wide character");
+    if (reason == NULL)
+        return NULL;
+
+    exc = PyObject_CallFunction(PyExc_UnicodeEncodeError, "sOnnO",
+                                "locale", unicode,
+                                (Py_ssize_t)error_pos,
+                                (Py_ssize_t)(error_pos+1),
+                                reason);
+    Py_DECREF(reason);
+    if (exc != NULL) {
+        PyCodec_StrictErrors(exc);
+        Py_XDECREF(exc);
+    }
     return NULL;
 }
 
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -343,9 +343,7 @@
     PyObject *message;
     PyObject *v, *args;
     int i = errno;
-#ifndef MS_WINDOWS
-    char *s;
-#else
+#ifdef MS_WINDOWS
     WCHAR *s_buf = NULL;
 #endif /* Unix/Windows */
 
@@ -355,11 +353,14 @@
 #endif
 
 #ifndef MS_WINDOWS
-    if (i == 0)
-        s = "Error"; /* Sometimes errno didn't get set */
-    else
-        s = strerror(i);
-    message = PyUnicode_DecodeUTF8(s, strlen(s), "ignore");
+    if (i != 0) {
+        char *s = strerror(i);
+        message = PyUnicode_DecodeLocale(s, 1);
+    }
+    else {
+        /* Sometimes errno didn't get set */
+        message = PyUnicode_FromString("Error");
+    }
 #else
     if (i == 0)
         message = PyUnicode_FromString("Error"); /* Sometimes errno didn't get set */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 04:57:14 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 04:57:14 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_What=27s_New_in_Python_3=2E?=
 =?utf8?q?3=3A_complete_the_deprecation_list?=
Message-ID: <E1RblOc-0003kO-Az@dinsdale.python.org>

http://hg.python.org/cpython/rev/880c7cffd7eb
changeset:   74008:880c7cffd7eb
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 04:59:06 2011 +0100
summary:
  What's New in Python 3.3: complete the deprecation list

Add also FIXMEs in unicodeobject.c

files:
  Doc/whatsnew/3.3.rst    |  3 +++
  Objects/unicodeobject.c |  2 ++
  2 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -798,6 +798,8 @@
    strings)
  * :c:func:`PyUnicode_AsUnicodeCopy`: use :c:func:`PyUnicode_AsUCS4Copy` or
    :c:func:`PyUnicode_AsWideCharString`
+ * :c:func:`PyUnicode_GetMax`
+
 
 Functions and macros manipulating Py_UNICODE* strings:
 
@@ -813,6 +815,7 @@
  * :c:macro:`Py_UNICODE_strchr`, :c:macro:`Py_UNICODE_strrchr`: use
    :c:func:`PyUnicode_FindChar`
  * :c:macro:`Py_UNICODE_FILL`
+ * :c:macro:`Py_UNICODE_MATCH`
 
 Encoders:
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -7042,6 +7042,7 @@
 
     if (*v == NULL) {
         /* Create unicode object */
+        /* FIXME: don't use _PyUnicode_New(), but allocate a wchar_t* buffer */
         *v = (PyObject*)_PyUnicode_New(outsize);
         if (*v == NULL)
             return -1;
@@ -7122,6 +7123,7 @@
             PyErr_NoMemory();
             goto error;
         }
+        /* FIXME: don't use _PyUnicode_New(), but allocate a wchar_t* buffer */
         *v = (PyObject*)_PyUnicode_New(size * Py_ARRAY_LENGTH(buffer));
         if (*v == NULL)
             goto error;

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Dec 17 05:38:40 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 17 Dec 2011 05:38:40 +0100
Subject: [Python-checkins] Daily reference leaks (f2e1867f33b8): sum=0
Message-ID: <E1Rbm2i-0004HR-6I@ap.vmr.nerim.net>

results for f2e1867f33b8 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogB9PJEQ', '-x']

From python-checkins at python.org  Sat Dec 17 05:46:20 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 05:46:20 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313560=3A_Locale_co?=
 =?utf8?q?dec_functions_use_the_classic_=22errors=22_parameter=2C?=
Message-ID: <E1RbmA8-0006aT-Mu@dinsdale.python.org>

http://hg.python.org/cpython/rev/07802351ccad
changeset:   74009:07802351ccad
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 05:47:23 2011 +0100
summary:
  Issue #13560: Locale codec functions use the classic "errors" parameter,
instead of surrogateescape

So it would be possible to support more error handlers later.

files:
  Include/unicodeobject.h |   6 +-
  Modules/main.c          |   2 +-
  Modules/posixmodule.c   |   2 +-
  Modules/timemodule.c    |   9 +++--
  Objects/unicodeobject.c |  47 ++++++++++++++++++++++++----
  Python/errors.c         |   2 +-
  6 files changed, 50 insertions(+), 18 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -1608,14 +1608,14 @@
 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocaleAndSize(
     const char *str,
     Py_ssize_t len,
-    int surrogateescape);
+    const char *errors);
 
 /* Similar to PyUnicode_DecodeLocaleAndSize(), but compute the string
    length using strlen(). */
 
 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocale(
     const char *str,
-    int surrogateescape);
+    const char *errors);
 
 /* Encode a Unicode object to the current locale encoding. The encoder is
    strict is *surrogateescape* is equal to zero, otherwise the
@@ -1624,7 +1624,7 @@
 
 PyAPI_FUNC(PyObject*) PyUnicode_EncodeLocale(
     PyObject *unicode,
-    int surrogateescape
+    const char *errors
     );
 
 /* --- File system encoding ---------------------------------------------- */
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -495,7 +495,7 @@
             /* Use utf-8 on Mac OS X */
             unicode = PyUnicode_FromString(p);
 #else
-            unicode = PyUnicode_DecodeLocale(p, 1);
+            unicode = PyUnicode_DecodeLocale(p, "surrogateescape");
 #endif
             if (unicode == NULL) {
                 /* ignore errors */
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -7891,7 +7891,7 @@
                         "strerror() argument out of range");
         return NULL;
     }
-    return PyUnicode_DecodeLocale(message, 1);
+    return PyUnicode_DecodeLocale(message, "surrogateescape");
 }
 
 
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -486,7 +486,7 @@
     fmt = format;
 #else
     /* Convert the unicode string to an ascii one */
-    format = PyUnicode_EncodeLocale(format_arg, 1);
+    format = PyUnicode_EncodeLocale(format_arg, "surrogateescape");
     if (format == NULL)
         return NULL;
     fmt = PyBytes_AS_STRING(format);
@@ -532,7 +532,8 @@
 #ifdef HAVE_WCSFTIME
             ret = PyUnicode_FromWideChar(outbuf, buflen);
 #else
-            ret = PyUnicode_DecodeLocaleAndSize(outbuf, buflen, 1);
+            ret = PyUnicode_DecodeLocaleAndSize(outbuf, buflen,
+                                                "surrogateescape");
 #endif
             PyMem_Free(outbuf);
             break;
@@ -764,8 +765,8 @@
 #endif /* PYOS_OS2 */
 #endif
     PyModule_AddIntConstant(m, "daylight", daylight);
-    otz0 = PyUnicode_DecodeLocale(tzname[0], 1);
-    otz1 = PyUnicode_DecodeLocale(tzname[1], 1);
+    otz0 = PyUnicode_DecodeLocale(tzname[0], "surrogateescape");
+    otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape");
     PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1));
 #else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/
 #ifdef HAVE_STRUCT_TM_TM_ZONE
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3125,8 +3125,31 @@
     return 0;
 }
 
-PyObject *
-PyUnicode_EncodeLocale(PyObject *unicode, int surrogateescape)
+static int
+locale_error_handler(const char *errors, int *surrogateescape)
+{
+    if (errors == NULL) {
+        *surrogateescape = 0;
+        return 0;
+    }
+
+    if (strcmp(errors, "strict") == 0) {
+        *surrogateescape = 0;
+        return 0;
+    }
+    if (strcmp(errors, "surrogateescape") == 0) {
+        *surrogateescape = 1;
+        return 0;
+    }
+    PyErr_Format(PyExc_ValueError,
+                 "only 'strict' and 'surrogateescape' error handlers "
+                 "are supported, not '%s'",
+                 errors);
+    return -1;
+}
+
+PyObject *
+PyUnicode_EncodeLocale(PyObject *unicode, const char *errors)
 {
     Py_ssize_t wlen, wlen2;
     wchar_t *wstr;
@@ -3135,6 +3158,10 @@
     PyObject *reason;
     PyObject *exc;
     size_t error_pos;
+    int surrogateescape;
+
+    if (locale_error_handler(errors, &surrogateescape) < 0)
+        return NULL;
 
     wstr = PyUnicode_AsWideCharString(unicode, &wlen);
     if (wstr == NULL)
@@ -3198,7 +3225,7 @@
     Py_XDECREF(bytes);
 
     if (errmsg != NULL)
-        reason = PyUnicode_DecodeLocale(errmsg, 1);
+        reason = PyUnicode_DecodeLocale(errmsg, "surrogateescape");
     else
         reason = PyUnicode_FromString(
             "wcstombs() encountered an unencodable "
@@ -3243,7 +3270,7 @@
                                          "surrogateescape");
     }
     else {
-        return PyUnicode_EncodeLocale(unicode, 1);
+        return PyUnicode_EncodeLocale(unicode, "surrogateescape");
     }
 #endif
 }
@@ -3351,13 +3378,17 @@
 
 PyObject*
 PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t len,
-                              int surrogateescape)
+                              const char *errors)
 {
     wchar_t smallbuf[256];
     size_t smallbuf_len = Py_ARRAY_LENGTH(smallbuf);
     wchar_t *wstr;
     size_t wlen, wlen2;
     PyObject *unicode;
+    int surrogateescape;
+
+    if (locale_error_handler(errors, &surrogateescape) < 0)
+        return NULL;
 
     if (str[len] != '\0' || len != strlen(str)) {
         PyErr_SetString(PyExc_TypeError, "embedded null character");
@@ -3419,10 +3450,10 @@
 }
 
 PyObject*
-PyUnicode_DecodeLocale(const char *str, int surrogateescape)
+PyUnicode_DecodeLocale(const char *str, const char *errors)
 {
     Py_ssize_t size = (Py_ssize_t)strlen(str);
-    return PyUnicode_DecodeLocaleAndSize(str, size, surrogateescape);
+    return PyUnicode_DecodeLocaleAndSize(str, size, errors);
 }
 
 
@@ -3456,7 +3487,7 @@
                                 "surrogateescape");
     }
     else {
-        return PyUnicode_DecodeLocaleAndSize(s, size, 1);
+        return PyUnicode_DecodeLocaleAndSize(s, size, "surrogateescape");
     }
 #endif
 }
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -355,7 +355,7 @@
 #ifndef MS_WINDOWS
     if (i != 0) {
         char *s = strerror(i);
-        message = PyUnicode_DecodeLocale(s, 1);
+        message = PyUnicode_DecodeLocale(s, "surrogateescape");
     }
     else {
         /* Sometimes errno didn't get set */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 07:07:48 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 07:07:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_The_locale_decoder_raises_a?=
 =?utf8?q?_UnicodeDecodeError_instead_of_an_OSError?=
Message-ID: <E1RbnQy-0003TQ-5C@dinsdale.python.org>

http://hg.python.org/cpython/rev/ea421c534305
changeset:   74010:ea421c534305
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 07:08:30 2011 +0100
summary:
  The locale decoder raises a UnicodeDecodeError instead of an OSError

Search the invalid character using mbrtowc().

files:
  Objects/unicodeobject.c |  105 +++++++++++++++++++++++----
  1 files changed, 87 insertions(+), 18 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3084,9 +3084,7 @@
 #endif
     char outbuf[MB_LEN_MAX];
     const wchar_t *start, *previous;
-    int save_errno;
-
-    save_errno = errno;
+
 #if SIZEOF_WCHAR_T == 2
     buf[2] = 0;
 #else
@@ -3114,14 +3112,11 @@
         wstr++;
 #endif
         len = wcstombs(outbuf, buf, sizeof(outbuf));
-        if (len == (size_t)-1) {
-            errno = save_errno;
+        if (len == (size_t)-1)
             return previous - start;
-        }
     }
 
     /* failed to find the unencodable character */
-    errno = save_errno;
     return 0;
 }
 
@@ -3199,7 +3194,7 @@
 
         len = wcstombs(NULL, wstr, 0);
         if (len == (size_t)-1) {
-            error_pos = wcstombs_errorpos(wstr);
+            error_pos = (size_t)-1;
             goto encode_error;
         }
 
@@ -3211,7 +3206,7 @@
 
         len2 = wcstombs(PyBytes_AS_STRING(bytes), wstr, len+1);
         if (len2 == (size_t)-1 || len2 > len) {
-            error_pos = wcstombs_errorpos(wstr);
+            error_pos = (size_t)-1;
             goto encode_error;
         }
         PyMem_Free(wstr);
@@ -3221,12 +3216,23 @@
 encode_error:
     errmsg = strerror(errno);
     assert(errmsg != NULL);
+
+    if (error_pos == (size_t)-1)
+        error_pos = wcstombs_errorpos(wstr);
+
     PyMem_Free(wstr);
     Py_XDECREF(bytes);
 
-    if (errmsg != NULL)
-        reason = PyUnicode_DecodeLocale(errmsg, "surrogateescape");
-    else
+    if (errmsg != NULL) {
+        size_t errlen;
+        wstr = _Py_char2wchar(errmsg, &errlen);
+        if (wstr != NULL) {
+            reason = PyUnicode_FromWideChar(wstr, errlen);
+            PyMem_Free(wstr);
+        } else
+            errmsg = NULL;
+    }
+    if (errmsg == NULL)
         reason = PyUnicode_FromString(
             "wcstombs() encountered an unencodable "
             "wide character");
@@ -3376,6 +3382,37 @@
     return NULL;
 }
 
+static size_t
+mbstowcs_errorpos(const char *str, size_t len)
+{
+#ifdef HAVE_MBRTOWC
+    const char *start = str;
+    mbstate_t mbs;
+    size_t converted;
+    wchar_t ch;
+
+    memset(&mbs, 0, sizeof mbs);
+    while (len)
+    {
+        converted = mbrtowc(&ch, (char*)str, len, &mbs);
+        if (converted == 0)
+            /* Reached end of string */
+            break;
+        if (converted == (size_t)-1 || converted == (size_t)-2) {
+            /* Conversion error or incomplete character */
+            return str - start;
+        }
+        else {
+            str += converted;
+            len -= converted;
+        }
+    }
+    /* failed to find the undecodable byte sequence */
+    return 0;
+#endif
+    return 0;
+}
+
 PyObject*
 PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t len,
                               const char *errors)
@@ -3386,6 +3423,9 @@
     size_t wlen, wlen2;
     PyObject *unicode;
     int surrogateescape;
+    size_t error_pos;
+    char *errmsg;
+    PyObject *reason, *exc;
 
     if (locale_error_handler(errors, &surrogateescape) < 0)
         return NULL;
@@ -3415,10 +3455,8 @@
 #else
         wlen = len;
 #endif
-        if (wlen == (size_t)-1) {
-            PyErr_SetFromErrno(PyExc_OSError);
-            return NULL;
-        }
+        if (wlen == (size_t)-1)
+            goto decode_error;
         if (wlen+1 <= smallbuf_len) {
             wstr = smallbuf;
         }
@@ -3436,8 +3474,7 @@
         if (wlen2 == (size_t)-1) {
             if (wstr != smallbuf)
                 PyMem_Free(wstr);
-            PyErr_SetFromErrno(PyExc_OSError);
-            return NULL;
+            goto decode_error;
         }
 #ifdef HAVE_BROKEN_MBSTOWCS
         assert(wlen2 == wlen);
@@ -3447,6 +3484,38 @@
             PyMem_Free(wstr);
     }
     return unicode;
+
+decode_error:
+    errmsg = strerror(errno);
+    assert(errmsg != NULL);
+
+    error_pos = mbstowcs_errorpos(str, len);
+    if (errmsg != NULL) {
+        size_t errlen;
+        wstr = _Py_char2wchar(errmsg, &errlen);
+        if (wstr != NULL) {
+            reason = PyUnicode_FromWideChar(wstr, errlen);
+            PyMem_Free(wstr);
+        } else
+            errmsg = NULL;
+    }
+    if (errmsg == NULL)
+        reason = PyUnicode_FromString(
+            "mbstowcs() encountered an invalid multibyte sequence");
+    if (reason == NULL)
+        return NULL;
+
+    exc = PyObject_CallFunction(PyExc_UnicodeDecodeError, "sy#nnO",
+                                "locale", str, len,
+                                (Py_ssize_t)error_pos,
+                                (Py_ssize_t)(error_pos+1),
+                                reason);
+    Py_DECREF(reason);
+    if (exc != NULL) {
+        PyCodec_StrictErrors(exc);
+        Py_XDECREF(exc);
+    }
+    return NULL;
 }
 
 PyObject*

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 14:02:23 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 17 Dec 2011 14:02:23 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_possible_NULL_dereferen?=
	=?utf8?q?ce?=
Message-ID: <E1RbtuB-00065r-Od@dinsdale.python.org>

http://hg.python.org/cpython/rev/8c355edc5b1d
changeset:   74011:8c355edc5b1d
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Dec 17 08:02:20 2011 -0500
summary:
  fix possible NULL dereference

files:
  Objects/descrobject.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Objects/descrobject.c b/Objects/descrobject.c
--- a/Objects/descrobject.c
+++ b/Objects/descrobject.c
@@ -628,7 +628,9 @@
             Py_DECREF(descr);
             descr = NULL;
         }
-        descr->d_qualname = NULL;
+        else {
+            descr->d_qualname = NULL;
+        }
     }
     return descr;
 }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 15:01:34 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sat, 17 Dec 2011 15:01:34 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2312809=3A_Expose_IP?=
 =?utf8?q?=5FTRANSPARENT_in_the_socket_module=2E_Patch_by_Michael?=
Message-ID: <E1RbupS-0001MP-Fu@dinsdale.python.org>

http://hg.python.org/cpython/rev/59ea1d1a4137
changeset:   74012:59ea1d1a4137
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sat Dec 17 14:59:56 2011 +0100
summary:
  Issue #12809: Expose IP_TRANSPARENT in the socket module. Patch by Michael
Farrell.

files:
  Misc/ACKS              |  1 +
  Modules/socketmodule.c |  3 +++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -304,6 +304,7 @@
 Clovis Fabricio
 Andreas Faerber
 Bill Fancher
+Michael Farrell
 Troy J. Farrell
 Mark Favas
 Boris Feld
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -6192,6 +6192,9 @@
 #ifdef  IP_MAX_MEMBERSHIPS
     PyModule_AddIntConstant(m, "IP_MAX_MEMBERSHIPS", IP_MAX_MEMBERSHIPS);
 #endif
+#ifdef  IP_TRANSPARENT
+    PyModule_AddIntConstant(m, "IP_TRANSPARENT", IP_TRANSPARENT);
+#endif
 
     /* IPv6 [gs]etsockopt options, defined in RFC2553 */
 #ifdef  IPV6_JOIN_GROUP

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 21:39:55 2011
From: python-checkins at python.org (gregory.p.smith)
Date: Sat, 17 Dec 2011 21:39:55 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Mention_that_level_can_be_a?=
 =?utf8?q?n_int_or_str_in_the_setLevel_docstring=2E?=
Message-ID: <E1Rc12x-0004mt-Oi@dinsdale.python.org>

http://hg.python.org/cpython/rev/652605cdd3d6
changeset:   74013:652605cdd3d6
user:        Gregory P. Smith <greg at krypto.org>
date:        Sat Dec 17 12:36:34 2011 -0800
summary:
  Mention that level can be an int or str in the setLevel docstring.

files:
  Lib/logging/__init__.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -789,7 +789,7 @@
 
     def setLevel(self, level):
         """
-        Set the logging level of this handler.
+        Set the logging level of this handler.  level must be an int or a str.
         """
         self.level = _checkLevel(level)
 
@@ -1194,7 +1194,7 @@
 
     def setLevel(self, level):
         """
-        Set the logging level of this logger.
+        Set the logging level of this logger.  level must be an int or a str.
         """
         self.level = _checkLevel(level)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 22:38:27 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 22:38:27 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Move_PyUnicode=5FWCHAR=5FKI?=
 =?utf8?q?ND_outside_PyUnicode=5FKind_enum?=
Message-ID: <E1Rc1xb-0001Rp-OL@dinsdale.python.org>

http://hg.python.org/cpython/rev/7876cd49300d
changeset:   74014:7876cd49300d
parent:      74012:59ea1d1a4137
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 22:18:27 2011 +0100
summary:
  Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum

files:
  Include/unicodeobject.h |  7 ++++---
  1 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -445,12 +445,13 @@
 #define PyUnicode_IS_COMPACT_ASCII(op)                 \
     (((PyASCIIObject*)op)->state.ascii && PyUnicode_IS_COMPACT(op))
 
-enum PyUnicode_Kind {
 /* String contains only wstr byte characters.  This is only possible
    when the string was created with a legacy API and _PyUnicode_Ready()
    has not been called yet.  */
-    PyUnicode_WCHAR_KIND = 0,
-/* Return values of the PyUnicode_KIND() macro: */
+#define PyUnicode_WCHAR_KIND 0
+
+/* Return values of the PyUnicode_KIND() macro */
+enum PyUnicode_Kind {
     PyUnicode_1BYTE_KIND = 1,
     PyUnicode_2BYTE_KIND = 2,
     PyUnicode_4BYTE_KIND = 4

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 22:38:28 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 22:38:28 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEwOTUx?=
 =?utf8?q?=3A_Fix_a_compiler_warning_in_timemodule=2Ec?=
Message-ID: <E1Rc1xc-0001Ru-Lk@dinsdale.python.org>

http://hg.python.org/cpython/rev/d7862e855274
changeset:   74015:d7862e855274
branch:      3.2
parent:      74004:0b86da9d6964
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 22:37:18 2011 +0100
summary:
  Issue #10951: Fix a compiler warning in timemodule.c

files:
  Modules/timemodule.c |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -527,14 +527,18 @@
      * will be ahead of time...
      */
     for (i = 1024; ; i += i) {
+#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         int err;
+#endif
         outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char));
         if (outbuf == NULL) {
             PyErr_NoMemory();
             break;
         }
         buflen = format_time(outbuf, i, fmt, &buf);
+#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         err = errno;
+#endif
         if (buflen > 0 || i >= 256 * fmtlen) {
             /* If the buffer is 256 times as long as the format,
                it's probably not failing for lack of room!

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 22:38:29 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 22:38:29 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2310951=3A_Fix_compiler_warnings_in_timemodule=2Ec_an?=
 =?utf8?q?d_unicodeobject=2Ec?=
Message-ID: <E1Rc1xd-0001Ry-I4@dinsdale.python.org>

http://hg.python.org/cpython/rev/49b85dba251d
changeset:   74016:49b85dba251d
parent:      74014:7876cd49300d
parent:      74015:d7862e855274
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 22:39:43 2011 +0100
summary:
  Issue #10951: Fix compiler warnings in timemodule.c and unicodeobject.c

Thanks J?r?my Anger for the fix.

files:
  Modules/timemodule.c    |  4 ++++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -515,14 +515,18 @@
      * will be ahead of time...
      */
     for (i = 1024; ; i += i) {
+#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         int err;
+#endif
         outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char));
         if (outbuf == NULL) {
             PyErr_NoMemory();
             break;
         }
         buflen = format_time(outbuf, i, fmt, &buf);
+#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__)
         err = errno;
+#endif
         if (buflen > 0 || i >= 256 * fmtlen) {
             /* If the buffer is 256 times as long as the format,
                it's probably not failing for lack of room!
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -4832,7 +4832,7 @@
     if (maxchar < 128 && size == unicode_size) {
         if (consumed)
             *consumed = size;
-        return unicode_fromascii(s, size);
+        return unicode_fromascii((const unsigned char *)s, size);
     }
 
     unicode = PyUnicode_New(unicode_size, maxchar);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 22:38:30 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 22:38:30 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_Merge?=
Message-ID: <E1Rc1xe-0001S7-9S@dinsdale.python.org>

http://hg.python.org/cpython/rev/cc2af102c73b
changeset:   74017:cc2af102c73b
parent:      74016:49b85dba251d
parent:      74013:652605cdd3d6
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 22:40:19 2011 +0100
summary:
  Merge

files:
  Lib/logging/__init__.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -789,7 +789,7 @@
 
     def setLevel(self, level):
         """
-        Set the logging level of this handler.
+        Set the logging level of this handler.  level must be an int or a str.
         """
         self.level = _checkLevel(level)
 
@@ -1194,7 +1194,7 @@
 
     def setLevel(self, level):
         """
-        Set the logging level of this logger.
+        Set the logging level of this logger.  level must be an int or a str.
         """
         self.level = _checkLevel(level)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 23:13:52 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 23:13:52 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTMw?=
 =?utf8?q?=3A_Document_os=2Elseek=28=29_result?=
Message-ID: <E1Rc2Vs-0003uD-8B@dinsdale.python.org>

http://hg.python.org/cpython/rev/46c418d8a480
changeset:   74018:46c418d8a480
branch:      3.2
parent:      74015:d7862e855274
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 23:15:09 2011 +0100
summary:
  Issue #13530: Document os.lseek() result

Patch written by J?r?my Anger.

files:
  Doc/library/os.rst    |  2 +-
  Misc/ACKS             |  1 +
  Modules/posixmodule.c |  3 ++-
  3 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -706,7 +706,7 @@
    by *how*: :const:`SEEK_SET` or ``0`` to set the position relative to the
    beginning of the file; :const:`SEEK_CUR` or ``1`` to set it relative to the
    current position; :const:`os.SEEK_END` or ``2`` to set it relative to the end of
-   the file.
+   the file. Return the new cursor position in bytes, starting from the beginning.
 
    Availability: Unix, Windows.
 
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -30,6 +30,7 @@
 Erik Anders?n
 Oliver Andrich
 Ross Andrus
+J?r?my Anger
 Jon Anglin
 ?ric Araujo
 Alicia Arlen
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5687,7 +5687,8 @@
 
 PyDoc_STRVAR(posix_lseek__doc__,
 "lseek(fd, pos, how) -> newpos\n\n\
-Set the current position of a file descriptor.");
+Set the current position of a file descriptor.\n\
+Return the new cursor position in bytes, starting from the beginning.");
 
 static PyObject *
 posix_lseek(PyObject *self, PyObject *args)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 23:13:53 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 23:13:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313530=3A_Document_os=2Elseek=28=29_result?=
Message-ID: <E1Rc2Vt-0003uH-3F@dinsdale.python.org>

http://hg.python.org/cpython/rev/b05caa600c40
changeset:   74019:b05caa600c40
parent:      74017:cc2af102c73b
parent:      74018:46c418d8a480
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 23:15:22 2011 +0100
summary:
  Issue #13530: Document os.lseek() result

Patch written by J?r?my Anger.

files:
  Doc/library/os.rst    |  2 +-
  Misc/ACKS             |  1 +
  Modules/posixmodule.c |  3 ++-
  3 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -977,7 +977,7 @@
    by *how*: :const:`SEEK_SET` or ``0`` to set the position relative to the
    beginning of the file; :const:`SEEK_CUR` or ``1`` to set it relative to the
    current position; :const:`os.SEEK_END` or ``2`` to set it relative to the end of
-   the file.
+   the file. Return the new cursor position in bytes, starting from the beginning.
 
    Availability: Unix, Windows.
 
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -32,6 +32,7 @@
 Erik Anders?n
 Oliver Andrich
 Ross Andrus
+J?r?my Anger
 Jon Anglin
 ?ric Araujo
 Alicia Arlen
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6946,7 +6946,8 @@
 
 PyDoc_STRVAR(posix_lseek__doc__,
 "lseek(fd, pos, how) -> newpos\n\n\
-Set the current position of a file descriptor.");
+Set the current position of a file descriptor.\n\
+Return the new cursor position in bytes, starting from the beginning.");
 
 static PyObject *
 posix_lseek(PyObject *self, PyObject *args)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 23:16:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 23:16:57 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExMjMx?=
 =?utf8?q?=3A_Fix_bytes_and_bytearray_docstrings?=
Message-ID: <E1Rc2Yr-0004Ix-4j@dinsdale.python.org>

http://hg.python.org/cpython/rev/beac7d6c7be4
changeset:   74020:beac7d6c7be4
branch:      3.2
parent:      74018:46c418d8a480
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 23:18:07 2011 +0100
summary:
  Issue #11231: Fix bytes and bytearray docstrings

Patch written by Brice Berna.

files:
  Misc/ACKS                 |   1 +
  Objects/bytearrayobject.c |  12 +++++-------
  Objects/bytesobject.c     |   7 ++++---
  3 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -80,6 +80,7 @@
 Andy Bensky
 Michel Van den Bergh
 Julian Berman
+Brice Berna
 Eric Beser
 Steven Bethard
 Stephen Bevan
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -2797,18 +2797,16 @@
 PyDoc_STRVAR(bytearray_doc,
 "bytearray(iterable_of_ints) -> bytearray\n\
 bytearray(string, encoding[, errors]) -> bytearray\n\
-bytearray(bytes_or_bytearray) -> mutable copy of bytes_or_bytearray\n\
-bytearray(memory_view) -> bytearray\n\
+bytearray(bytes_or_buffer) -> mutable copy of bytes_or_buffer\n\
+bytearray(int) -> bytes array of size given by the parameter initialized with null bytes\n\
+bytearray() -> empty bytes array\n\
 \n\
 Construct an mutable bytearray object from:\n\
   - an iterable yielding integers in range(256)\n\
   - a text string encoded using the specified encoding\n\
-  - a bytes or a bytearray object\n\
+  - a bytes or a buffer object\n\
   - any object implementing the buffer API.\n\
-\n\
-bytearray(int) -> bytearray\n\
-\n\
-Construct a zero-initialized bytearray of the given length.");
+  - an integer");
 
 
 static PyObject *bytearray_iter(PyObject *seq);
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -2721,13 +2721,14 @@
 "bytes(iterable_of_ints) -> bytes\n\
 bytes(string, encoding[, errors]) -> bytes\n\
 bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer\n\
-bytes(memory_view) -> bytes\n\
+bytes(int) -> bytes object of size given by the parameter initialized with null bytes\n\
+bytes() -> empty bytes object\n\
 \n\
 Construct an immutable array of bytes from:\n\
   - an iterable yielding integers in range(256)\n\
   - a text string encoded using the specified encoding\n\
-  - a bytes or a buffer object\n\
-  - any object implementing the buffer API.");
+  - any object implementing the buffer API.\n\
+  - an integer");
 
 static PyObject *bytes_iter(PyObject *seq);
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 17 23:16:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Sat, 17 Dec 2011 23:16:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2311231=3A_Fix_bytes_and_bytearray_docstrings?=
Message-ID: <E1Rc2Yr-0004J1-Uz@dinsdale.python.org>

http://hg.python.org/cpython/rev/dc670add1e28
changeset:   74021:dc670add1e28
parent:      74019:b05caa600c40
parent:      74020:beac7d6c7be4
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sat Dec 17 23:18:43 2011 +0100
summary:
  Issue #11231: Fix bytes and bytearray docstrings

Patch written by Brice Berna.

files:
  Misc/ACKS                 |   1 +
  Objects/bytearrayobject.c |  12 +++++-------
  Objects/bytesobject.c     |   7 ++++---
  3 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -84,6 +84,7 @@
 Andy Bensky
 Michel Van den Bergh
 Julian Berman
+Brice Berna
 Eric Beser
 Steven Bethard
 Stephen Bevan
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -2890,18 +2890,16 @@
 PyDoc_STRVAR(bytearray_doc,
 "bytearray(iterable_of_ints) -> bytearray\n\
 bytearray(string, encoding[, errors]) -> bytearray\n\
-bytearray(bytes_or_bytearray) -> mutable copy of bytes_or_bytearray\n\
-bytearray(memory_view) -> bytearray\n\
+bytearray(bytes_or_buffer) -> mutable copy of bytes_or_buffer\n\
+bytearray(int) -> bytes array of size given by the parameter initialized with null bytes\n\
+bytearray() -> empty bytes array\n\
 \n\
 Construct an mutable bytearray object from:\n\
   - an iterable yielding integers in range(256)\n\
   - a text string encoded using the specified encoding\n\
-  - a bytes or a bytearray object\n\
+  - a bytes or a buffer object\n\
   - any object implementing the buffer API.\n\
-\n\
-bytearray(int) -> bytearray\n\
-\n\
-Construct a zero-initialized bytearray of the given length.");
+  - an integer");
 
 
 static PyObject *bytearray_iter(PyObject *seq);
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -2732,13 +2732,14 @@
 "bytes(iterable_of_ints) -> bytes\n\
 bytes(string, encoding[, errors]) -> bytes\n\
 bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer\n\
-bytes(memory_view) -> bytes\n\
+bytes(int) -> bytes object of size given by the parameter initialized with null bytes\n\
+bytes() -> empty bytes object\n\
 \n\
 Construct an immutable array of bytes from:\n\
   - an iterable yielding integers in range(256)\n\
   - a text string encoded using the specified encoding\n\
-  - a bytes or a buffer object\n\
-  - any object implementing the buffer API.");
+  - any object implementing the buffer API.\n\
+  - an integer");
 
 static PyObject *bytes_iter(PyObject *seq);
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 01:17:54 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 01:17:54 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313623=3A_Fix_a_per?=
 =?utf8?q?formance_regression_introduced_by_issue_=2312170_in?=
Message-ID: <E1Rc4Ru-0006by-6z@dinsdale.python.org>

http://hg.python.org/cpython/rev/75648db1b3f3
changeset:   74022:75648db1b3f3
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 01:17:41 2011 +0100
summary:
  Issue #13623: Fix a performance regression introduced by issue #12170 in
bytes.find() and handle correctly OverflowError (raise the same ValueError than
the error for -1).

files:
  Lib/test/test_bytes.py   |   5 ++++
  Objects/stringlib/find.h |  33 ++++++++++++++++-----------
  2 files changed, 25 insertions(+), 13 deletions(-)


diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py
--- a/Lib/test/test_bytes.py
+++ b/Lib/test/test_bytes.py
@@ -362,6 +362,11 @@
         self.assertEqual(b.find(i, 1, 3), 1)
         self.assertEqual(b.find(w, 1, 3), -1)
 
+        for index in (-1, 256, sys.maxsize + 1):
+            self.assertRaisesRegex(
+                ValueError, r'byte must be in range\(0, 256\)',
+                b.find, index)
+
     def test_rfind(self):
         b = self.type2test(b'mississippi')
         i = 105
diff --git a/Objects/stringlib/find.h b/Objects/stringlib/find.h
--- a/Objects/stringlib/find.h
+++ b/Objects/stringlib/find.h
@@ -186,27 +186,34 @@
 {
     PyObject *tmp_subobj;
     Py_ssize_t ival;
+    PyObject *err;
 
     if(!STRINGLIB(parse_args_finds)(function_name, args, &tmp_subobj,
                                     start, end))
         return 0;
 
-    ival = PyNumber_AsSsize_t(tmp_subobj, PyExc_ValueError);
-    if (ival == -1 && PyErr_Occurred()) {
-        PyErr_Clear();
+    if (!PyNumber_Check(tmp_subobj)) {
         *subobj = tmp_subobj;
-    }
-    else {
-        /* The first argument was an integer */
-        if(ival < 0 || ival > 255) {
-            PyErr_SetString(PyExc_ValueError, "byte must be in range(0, 256)");
-            return 0;
-        }
-
-        *subobj = NULL;
-        *byte = (char)ival;
+        return 1;
     }
 
+    ival = PyNumber_AsSsize_t(tmp_subobj, PyExc_OverflowError);
+    if (ival == -1) {
+        err = PyErr_Occurred();
+        if (err && !PyErr_GivenExceptionMatches(err, PyExc_OverflowError)) {
+            PyErr_Clear();
+            *subobj = tmp_subobj;
+            return 1;
+        }
+    }
+
+    if (ival < 0 || ival > 255) {
+        PyErr_SetString(PyExc_ValueError, "byte must be in range(0, 256)");
+        return 0;
+    }
+
+    *subobj = NULL;
+    *byte = (char)ival;
     return 1;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 01:29:00 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 01:29:00 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTIy?=
 =?utf8?q?=3A_document_error_return_values_of_some_float_and_complex_C_API?=
Message-ID: <E1Rc4ce-0007Ko-K8@dinsdale.python.org>

http://hg.python.org/cpython/rev/68cbf6551710
changeset:   74023:68cbf6551710
branch:      3.2
parent:      74020:beac7d6c7be4
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 01:25:27 2011 +0100
summary:
  Issue #13522: document error return values of some float and complex C API functions.

files:
  Doc/c-api/complex.rst |  8 +++++++-
  Doc/c-api/float.rst   |  2 ++
  Misc/ACKS             |  1 +
  3 files changed, 10 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -63,12 +63,18 @@
    Return the quotient of two complex numbers, using the C :c:type:`Py_complex`
    representation.
 
+   If *divisor* is null, this method returns zero and sets
+   :c:data:`errno` to :c:data:`EDOM`.
+
 
 .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp)
 
    Return the exponentiation of *num* by *exp*, using the C :c:type:`Py_complex`
    representation.
 
+   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`.
+
 
 Complex Numbers as Python Objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -123,4 +129,4 @@
 
    If *op* is not a Python complex number object but has a :meth:`__complex__`
    method, this method will first be called to convert *op* to a Python complex
-   number object.
+   number object. Upon failure, this method returns ``-1.0`` as a real value.
diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst
--- a/Doc/c-api/float.rst
+++ b/Doc/c-api/float.rst
@@ -47,6 +47,8 @@
    Return a C :c:type:`double` representation of the contents of *pyfloat*.  If
    *pyfloat* is not a Python floating point object but has a :meth:`__float__`
    method, this method will first be called to convert *pyfloat* into a float.
+   This method returns ``-1.0`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
 
 
 .. c:function:: double PyFloat_AS_DOUBLE(PyObject *pyfloat)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -139,6 +139,7 @@
 Lee Busby
 Ralph Butler
 Jp Calderone
+Arnaud Calmettes
 Daniel Calvelo
 Tony Campbell
 Brett Cannon

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 01:29:09 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 01:29:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313522=3A_document_error_return_values_of_some_float?=
 =?utf8?q?_and_complex_C_API?=
Message-ID: <E1Rc4cn-0007LQ-Ce@dinsdale.python.org>

http://hg.python.org/cpython/rev/1f096611baf4
changeset:   74024:1f096611baf4
parent:      74022:75648db1b3f3
parent:      74023:68cbf6551710
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 01:26:05 2011 +0100
summary:
  Issue #13522: document error return values of some float and complex C API functions.

files:
  Doc/c-api/complex.rst |  8 +++++++-
  Doc/c-api/float.rst   |  2 ++
  Misc/ACKS             |  1 +
  3 files changed, 10 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -63,12 +63,18 @@
    Return the quotient of two complex numbers, using the C :c:type:`Py_complex`
    representation.
 
+   If *divisor* is null, this method returns zero and sets
+   :c:data:`errno` to :c:data:`EDOM`.
+
 
 .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp)
 
    Return the exponentiation of *num* by *exp*, using the C :c:type:`Py_complex`
    representation.
 
+   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`.
+
 
 Complex Numbers as Python Objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -123,4 +129,4 @@
 
    If *op* is not a Python complex number object but has a :meth:`__complex__`
    method, this method will first be called to convert *op* to a Python complex
-   number object.
+   number object. Upon failure, this method returns ``-1.0`` as a real value.
diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst
--- a/Doc/c-api/float.rst
+++ b/Doc/c-api/float.rst
@@ -47,6 +47,8 @@
    Return a C :c:type:`double` representation of the contents of *pyfloat*.  If
    *pyfloat* is not a Python floating point object but has a :meth:`__float__`
    method, this method will first be called to convert *pyfloat* into a float.
+   This method returns ``-1.0`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
 
 
 .. c:function:: double PyFloat_AS_DOUBLE(PyObject *pyfloat)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -149,6 +149,7 @@
 Ralph Butler
 Nicolas Cadou
 Jp Calderone
+Arnaud Calmettes
 Daniel Calvelo
 Tony Campbell
 Brett Cannon

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 01:29:12 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 01:29:12 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTIy?=
 =?utf8?q?=3A_document_error_return_values_of_some_float_and_complex_C_API?=
Message-ID: <E1Rc4cq-0007LU-Jn@dinsdale.python.org>

http://hg.python.org/cpython/rev/059e4d752fbe
changeset:   74025:059e4d752fbe
branch:      2.7
parent:      74003:766a21ebf82e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 01:25:27 2011 +0100
summary:
  Issue #13522: document error return values of some float and complex C API functions.

files:
  Doc/c-api/complex.rst |  7 +++++++
  Doc/c-api/float.rst   |  2 ++
  Misc/ACKS             |  1 +
  3 files changed, 10 insertions(+), 0 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -63,12 +63,18 @@
    Return the quotient of two complex numbers, using the C :ctype:`Py_complex`
    representation.
 
+   If *divisor* is null, this method returns zero and sets
+   :cdata:`errno` to :cdata:`EDOM`.
+
 
 .. cfunction:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp)
 
    Return the exponentiation of *num* by *exp*, using the C :ctype:`Py_complex`
    representation.
 
+   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   this method returns zero and sets :cdata:`errno` to :cdata:`EDOM`.
+
 
 Complex Numbers as Python Objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -125,6 +131,7 @@
 .. cfunction:: Py_complex PyComplex_AsCComplex(PyObject *op)
 
    Return the :ctype:`Py_complex` value of the complex number *op*.
+   Upon failure, this method returns ``-1.0`` as a real value.
 
    .. versionchanged:: 2.6
       If *op* is not a Python complex number object but has a :meth:`__complex__`
diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst
--- a/Doc/c-api/float.rst
+++ b/Doc/c-api/float.rst
@@ -55,6 +55,8 @@
    Return a C :ctype:`double` representation of the contents of *pyfloat*.  If
    *pyfloat* is not a Python floating point object but has a :meth:`__float__`
    method, this method will first be called to convert *pyfloat* into a float.
+   This method returns ``-1.0`` upon failure, so one should call
+   :cfunc:`PyErr_Occurred` to check for errors.
 
 
 .. cfunction:: double PyFloat_AS_DOUBLE(PyObject *pyfloat)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -120,6 +120,7 @@
 Lee Busby
 Ralph Butler
 Jp Calderone
+Arnaud Calmettes
 Daniel Calvelo
 Tony Campbell
 Brett Cannon

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 02:43:37 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 02:43:37 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313621=3A_Optimize_?=
 =?utf8?q?str=2Ereplace=28char1=2C_char2=29?=
Message-ID: <E1Rc5mr-0002uJ-9g@dinsdale.python.org>

http://hg.python.org/cpython/rev/c802bfc8acfc
changeset:   74026:c802bfc8acfc
parent:      74024:1f096611baf4
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 02:43:08 2011 +0100
summary:
  Issue #13621: Optimize str.replace(char1, char2)

Use findchar() which is more optimized than a dummy loop using
PyUnicode_READ().  PyUnicode_READ() is a complex and slow macro.

files:
  Objects/unicodeobject.c |  30 ++++++++++++++++++++--------
  1 files changed, 21 insertions(+), 9 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10285,7 +10285,6 @@
     maxchar = Py_MAX(maxchar, maxchar_str2);
 
     if (len1 == len2) {
-        Py_ssize_t i;
         /* same length */
         if (len1 == 0)
             goto nothing;
@@ -10293,9 +10292,12 @@
             /* replace characters */
             Py_UCS4 u1, u2;
             int rkind;
+            Py_ssize_t index, pos;
+            char *src;
+
             u1 = PyUnicode_READ_CHAR(str1, 0);
-            if (findchar(sbuf, PyUnicode_KIND(self),
-                         slen, u1, 1) < 0)
+            pos = findchar(sbuf, PyUnicode_KIND(self), slen, u1, 1);
+            if (pos < 0)
                 goto nothing;
             u2 = PyUnicode_READ_CHAR(str2, 0);
             u = PyUnicode_New(slen, maxchar);
@@ -10303,16 +10305,26 @@
                 goto error;
             copy_characters(u, 0, self, 0, slen);
             rkind = PyUnicode_KIND(u);
-            for (i = 0; i < PyUnicode_GET_LENGTH(u); i++)
-                if (PyUnicode_READ(rkind, PyUnicode_DATA(u), i) == u1) {
-                    if (--maxcount < 0)
-                        break;
-                    PyUnicode_WRITE(rkind, PyUnicode_DATA(u), i, u2);
-                }
+
+            PyUnicode_WRITE(rkind, PyUnicode_DATA(u), pos, u2);
+            index = 0;
+            src = sbuf;
+            while (--maxcount)
+            {
+                pos++;
+                src += pos * PyUnicode_KIND(self);
+                slen -= pos;
+                index += pos;
+                pos = findchar(src, PyUnicode_KIND(self), slen, u1, 1);
+                if (pos < 0)
+                    break;
+                PyUnicode_WRITE(rkind, PyUnicode_DATA(u), index + pos, u2);
+            }
         }
         else {
             int rkind = skind;
             char *res;
+            Py_ssize_t i;
 
             if (kind1 < rkind) {
                 /* widen substring */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 02:54:27 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 02:54:27 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313522=3A_Fix_=5FPy?=
 =?utf8?q?=5Fco=5Fpow=28=29_documentation?=
Message-ID: <E1Rc5xL-0003WU-GI@dinsdale.python.org>

http://hg.python.org/cpython/rev/67a4e8fe650e
changeset:   74027:67a4e8fe650e
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 02:56:18 2011 +0100
summary:
  Issue #13522: Fix _Py_co_pow() documentation

Patch written by Arnaud Calmettes.

files:
  Doc/c-api/complex.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -72,7 +72,7 @@
    Return the exponentiation of *num* by *exp*, using the C :c:type:`Py_complex`
    representation.
 
-   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   If *num* is null and *exp* is not a positive real number,
    this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`.
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 03:24:52 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 03:24:52 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Optimize_str_*_n_for_len=28?=
 =?utf8?q?str=29=3D=3D1_and_UCS-2_or_UCS-4?=
Message-ID: <E1Rc6Qm-000603-Tw@dinsdale.python.org>

http://hg.python.org/cpython/rev/20f2f5193b04
changeset:   74028:20f2f5193b04
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 03:26:31 2011 +0100
summary:
  Optimize str * n for len(str)==1 and UCS-2 or UCS-4

files:
  Objects/unicodeobject.c |  15 +++++++++++----
  1 files changed, 11 insertions(+), 4 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -12028,12 +12028,19 @@
     if (PyUnicode_GET_LENGTH(str) == 1) {
         const int kind = PyUnicode_KIND(str);
         const Py_UCS4 fill_char = PyUnicode_READ(kind, PyUnicode_DATA(str), 0);
-        void *to = PyUnicode_DATA(u);
-        if (kind == PyUnicode_1BYTE_KIND)
+        if (kind == PyUnicode_1BYTE_KIND) {
+            void *to = PyUnicode_DATA(u);
             memset(to, (unsigned char)fill_char, len);
-        else {
+        }
+        else if (kind == PyUnicode_2BYTE_KIND) {
+            Py_UCS2 *ucs2 = PyUnicode_2BYTE_DATA(u);
             for (n = 0; n < len; ++n)
-                PyUnicode_WRITE(kind, to, n, fill_char);
+                ucs2[n] = fill_char;
+        } else {
+            Py_UCS4 *ucs4 = PyUnicode_4BYTE_DATA(u);
+            assert(kind == PyUnicode_4BYTE_KIND);
+            for (n = 0; n < len; ++n)
+                ucs4[n] = fill_char;
         }
     }
     else {

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Dec 18 05:31:15 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 18 Dec 2011 05:31:15 +0100
Subject: [Python-checkins] Daily reference leaks (20f2f5193b04): sum=0
Message-ID: <E1Rc8P5-00041k-Vc@ap.vmr.nerim.net>

results for 20f2f5193b04 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogMs7WDq', '-x']

From python-checkins at python.org  Sun Dec 18 12:29:05 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 12:29:05 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTIy?=
 =?utf8?q?=3A_Fix_=5FPy=5Fco=5Fpow=28=29_documentation?=
Message-ID: <E1RcEvR-0004X7-QF@dinsdale.python.org>

http://hg.python.org/cpython/rev/edc981ce8748
changeset:   74029:edc981ce8748
branch:      3.2
parent:      74023:68cbf6551710
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 02:56:18 2011 +0100
summary:
  Issue #13522: Fix _Py_co_pow() documentation

Patch written by Arnaud Calmettes.

files:
  Doc/c-api/complex.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -72,7 +72,7 @@
    Return the exponentiation of *num* by *exp*, using the C :c:type:`Py_complex`
    representation.
 
-   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   If *num* is null and *exp* is not a positive real number,
    this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`.
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 12:29:27 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 12:29:27 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTIy?=
 =?utf8?q?=3A_Fix_=5FPy=5Fco=5Fpow=28=29_documentation?=
Message-ID: <E1RcEvn-0004YC-7n@dinsdale.python.org>

http://hg.python.org/cpython/rev/2863470caebb
changeset:   74030:2863470caebb
branch:      2.7
parent:      74025:059e4d752fbe
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 02:56:18 2011 +0100
summary:
  Issue #13522: Fix _Py_co_pow() documentation

Patch written by Arnaud Calmettes.

files:
  Doc/c-api/complex.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst
--- a/Doc/c-api/complex.rst
+++ b/Doc/c-api/complex.rst
@@ -72,7 +72,7 @@
    Return the exponentiation of *num* by *exp*, using the C :ctype:`Py_complex`
    representation.
 
-   If :attr:`exp.imag` is not null, or :attr:`exp.real` is negative,
+   If *num* is null and *exp* is not a positive real number,
    this method returns zero and sets :cdata:`errno` to :cdata:`EDOM`.
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 12:29:48 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 12:29:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Null_merge?=
Message-ID: <E1RcEw8-0004ZF-F7@dinsdale.python.org>

http://hg.python.org/cpython/rev/c7fc52997ec1
changeset:   74031:c7fc52997ec1
parent:      74028:20f2f5193b04
parent:      74029:edc981ce8748
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 12:28:18 2011 +0100
summary:
  Null merge

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 14:20:52 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 14:20:52 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313624=3A_Write_a_s?=
 =?utf8?q?pecialized_UTF-8_encoder_to_allow_more_optimization?=
Message-ID: <E1RcGfc-0003bX-EN@dinsdale.python.org>

http://hg.python.org/cpython/rev/fbd797fc3809
changeset:   74032:fbd797fc3809
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 14:22:26 2011 +0100
summary:
  Issue #13624: Write a specialized UTF-8 encoder to allow more optimization

The main bottleneck was the PyUnicode_READ() macro.

files:
  Doc/whatsnew/3.3.rst       |    4 +-
  Objects/stringlib/codecs.h |  197 +++++++++++++++++++++++++
  Objects/unicodeobject.c    |  163 +-------------------
  3 files changed, 213 insertions(+), 151 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -712,7 +712,9 @@
   * the memory footprint is divided by 2 to 4 depending on the text
   * encode an ASCII string to UTF-8 doesn't need to encode characters anymore,
     the UTF-8 representation is shared with the ASCII representation
-  * getting a substring of a latin1 strings is 4 times faster
+  * the UTF-8 encoder has been optimized
+  * repeating a single ASCII letter and getting a substring of a ASCII strings
+    is 4 times faster
 
 
 Build and C API Changes
diff --git a/Objects/stringlib/codecs.h b/Objects/stringlib/codecs.h
--- a/Objects/stringlib/codecs.h
+++ b/Objects/stringlib/codecs.h
@@ -153,4 +153,201 @@
 #undef LONG_PTR_MASK
 #undef ASCII_CHAR_MASK
 
+
+/* UTF-8 encoder specialized for a Unicode kind to avoid the slow
+   PyUnicode_READ() macro. Delete some parts of the code depending on the kind:
+   UCS-1 strings don't need to handle surrogates for example. */
+Py_LOCAL_INLINE(PyObject *)
+STRINGLIB(utf8_encoder)(PyObject *unicode,
+                        STRINGLIB_CHAR *data,
+                        Py_ssize_t size,
+                        const char *errors)
+{
+#define MAX_SHORT_UNICHARS 300  /* largest size we'll do on the stack */
+
+    Py_ssize_t i;                /* index into s of next input byte */
+    PyObject *result;            /* result string object */
+    char *p;                     /* next free byte in output buffer */
+    Py_ssize_t nallocated;      /* number of result bytes allocated */
+    Py_ssize_t nneeded;            /* number of result bytes needed */
+#if STRINGLIB_SIZEOF_CHAR > 1
+    PyObject *errorHandler = NULL;
+    PyObject *exc = NULL;
+    PyObject *rep = NULL;
+#endif
+#if STRINGLIB_SIZEOF_CHAR == 1
+    const Py_ssize_t max_char_size = 2;
+    char stackbuf[MAX_SHORT_UNICHARS * 2];
+#elif STRINGLIB_SIZEOF_CHAR == 2
+    const Py_ssize_t max_char_size = 3;
+    char stackbuf[MAX_SHORT_UNICHARS * 3];
+#else /*  STRINGLIB_SIZEOF_CHAR == 4 */
+    const Py_ssize_t max_char_size = 4;
+    char stackbuf[MAX_SHORT_UNICHARS * 4];
+#endif
+
+    assert(size >= 0);
+
+    if (size <= MAX_SHORT_UNICHARS) {
+        /* Write into the stack buffer; nallocated can't overflow.
+         * At the end, we'll allocate exactly as much heap space as it
+         * turns out we need.
+         */
+        nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
+        result = NULL;   /* will allocate after we're done */
+        p = stackbuf;
+    }
+    else {
+        if (size > PY_SSIZE_T_MAX / max_char_size) {
+            /* integer overflow */
+            return PyErr_NoMemory();
+        }
+        /* Overallocate on the heap, and give the excess back at the end. */
+        nallocated = size * max_char_size;
+        result = PyBytes_FromStringAndSize(NULL, nallocated);
+        if (result == NULL)
+            return NULL;
+        p = PyBytes_AS_STRING(result);
+    }
+
+    for (i = 0; i < size;) {
+        Py_UCS4 ch = data[i++];
+
+        if (ch < 0x80) {
+            /* Encode ASCII */
+            *p++ = (char) ch;
+
+        }
+        else
+#if STRINGLIB_SIZEOF_CHAR > 1
+        if (ch < 0x0800)
+#endif
+        {
+            /* Encode Latin-1 */
+            *p++ = (char)(0xc0 | (ch >> 6));
+            *p++ = (char)(0x80 | (ch & 0x3f));
+        }
+#if STRINGLIB_SIZEOF_CHAR > 1
+        else if (Py_UNICODE_IS_SURROGATE(ch)) {
+            Py_ssize_t newpos;
+            Py_ssize_t repsize, k, startpos;
+            startpos = i-1;
+            rep = unicode_encode_call_errorhandler(
+                  errors, &errorHandler, "utf-8", "surrogates not allowed",
+                  unicode, &exc, startpos, startpos+1, &newpos);
+            if (!rep)
+                goto error;
+
+            if (PyBytes_Check(rep))
+                repsize = PyBytes_GET_SIZE(rep);
+            else
+                repsize = PyUnicode_GET_LENGTH(rep);
+
+            if (repsize > max_char_size) {
+                Py_ssize_t offset;
+
+                if (result == NULL)
+                    offset = p - stackbuf;
+                else
+                    offset = p - PyBytes_AS_STRING(result);
+
+                if (nallocated > PY_SSIZE_T_MAX - repsize + max_char_size) {
+                    /* integer overflow */
+                    PyErr_NoMemory();
+                    goto error;
+                }
+                nallocated += repsize - max_char_size;
+                if (result != NULL) {
+                    if (_PyBytes_Resize(&result, nallocated) < 0)
+                        goto error;
+                } else {
+                    result = PyBytes_FromStringAndSize(NULL, nallocated);
+                    if (result == NULL)
+                        goto error;
+                    Py_MEMCPY(PyBytes_AS_STRING(result), stackbuf, offset);
+                }
+                p = PyBytes_AS_STRING(result) + offset;
+            }
+
+            if (PyBytes_Check(rep)) {
+                char *prep = PyBytes_AS_STRING(rep);
+                for(k = repsize; k > 0; k--)
+                    *p++ = *prep++;
+            } else /* rep is unicode */ {
+                enum PyUnicode_Kind repkind;
+                void *repdata;
+
+                if (PyUnicode_READY(rep) < 0)
+                    goto error;
+                repkind = PyUnicode_KIND(rep);
+                repdata = PyUnicode_DATA(rep);
+
+                for(k=0; k<repsize; k++) {
+                    Py_UCS4 c = PyUnicode_READ(repkind, repdata, k);
+                    if (0x80 <= c) {
+                        raise_encode_exception(&exc, "utf-8",
+                                               unicode,
+                                               i-1, i,
+                                               "surrogates not allowed");
+                        goto error;
+                    }
+                    *p++ = (char)c;
+                }
+            }
+            Py_CLEAR(rep);
+        }
+        else
+#if STRINGLIB_SIZEOF_CHAR > 2
+        if (ch < 0x10000)
+#endif
+        {
+            *p++ = (char)(0xe0 | (ch >> 12));
+            *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
+            *p++ = (char)(0x80 | (ch & 0x3f));
+        }
+#if STRINGLIB_SIZEOF_CHAR > 2
+        else /* ch >= 0x10000 */
+        {
+            assert(ch <= MAX_UNICODE);
+            /* Encode UCS4 Unicode ordinals */
+            *p++ = (char)(0xf0 | (ch >> 18));
+            *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
+            *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
+            *p++ = (char)(0x80 | (ch & 0x3f));
+        }
+#endif /* STRINGLIB_SIZEOF_CHAR > 2 */
+#endif /* STRINGLIB_SIZEOF_CHAR > 1 */
+    }
+
+    if (result == NULL) {
+        /* This was stack allocated. */
+        nneeded = p - stackbuf;
+        assert(nneeded <= nallocated);
+        result = PyBytes_FromStringAndSize(stackbuf, nneeded);
+    }
+    else {
+        /* Cut back to size actually needed. */
+        nneeded = p - PyBytes_AS_STRING(result);
+        assert(nneeded <= nallocated);
+        _PyBytes_Resize(&result, nneeded);
+    }
+
+#if STRINGLIB_SIZEOF_CHAR > 1
+    Py_XDECREF(errorHandler);
+    Py_XDECREF(exc);
+#endif
+    return result;
+
+#if STRINGLIB_SIZEOF_CHAR > 1
+ error:
+    Py_XDECREF(rep);
+    Py_XDECREF(errorHandler);
+    Py_XDECREF(exc);
+    Py_XDECREF(result);
+    return NULL;
+#endif
+
+#undef MAX_SHORT_UNICHARS
+}
+
 #endif /* STRINGLIB_IS_UNICODE */
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -4987,20 +4987,9 @@
 PyObject *
 _PyUnicode_AsUTF8String(PyObject *unicode, const char *errors)
 {
-#define MAX_SHORT_UNICHARS 300  /* largest size we'll do on the stack */
-
-    Py_ssize_t i;                /* index into s of next input byte */
-    PyObject *result;            /* result string object */
-    char *p;                     /* next free byte in output buffer */
-    Py_ssize_t nallocated;      /* number of result bytes allocated */
-    Py_ssize_t nneeded;            /* number of result bytes needed */
-    char stackbuf[MAX_SHORT_UNICHARS * 4];
-    PyObject *errorHandler = NULL;
-    PyObject *exc = NULL;
-    int kind;
+    enum PyUnicode_Kind kind;
     void *data;
     Py_ssize_t size;
-    PyObject *rep = NULL;
 
     if (!PyUnicode_Check(unicode)) {
         PyErr_BadArgument();
@@ -5018,144 +5007,18 @@
     data = PyUnicode_DATA(unicode);
     size = PyUnicode_GET_LENGTH(unicode);
 
-    assert(size >= 0);
-
-    if (size <= MAX_SHORT_UNICHARS) {
-        /* Write into the stack buffer; nallocated can't overflow.
-         * At the end, we'll allocate exactly as much heap space as it
-         * turns out we need.
-         */
-        nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
-        result = NULL;   /* will allocate after we're done */
-        p = stackbuf;
-    }
-    else {
-        /* Overallocate on the heap, and give the excess back at the end. */
-        nallocated = size * 4;
-        if (nallocated / 4 != size)  /* overflow! */
-            return PyErr_NoMemory();
-        result = PyBytes_FromStringAndSize(NULL, nallocated);
-        if (result == NULL)
-            return NULL;
-        p = PyBytes_AS_STRING(result);
-    }
-
-    for (i = 0; i < size;) {
-        Py_UCS4 ch = PyUnicode_READ(kind, data, i++);
-
-        if (ch < 0x80)
-            /* Encode ASCII */
-            *p++ = (char) ch;
-
-        else if (ch < 0x0800) {
-            /* Encode Latin-1 */
-            *p++ = (char)(0xc0 | (ch >> 6));
-            *p++ = (char)(0x80 | (ch & 0x3f));
-        } else if (Py_UNICODE_IS_SURROGATE(ch)) {
-            Py_ssize_t newpos;
-            Py_ssize_t repsize, k, startpos;
-            startpos = i-1;
-            rep = unicode_encode_call_errorhandler(
-                  errors, &errorHandler, "utf-8", "surrogates not allowed",
-                  unicode, &exc, startpos, startpos+1, &newpos);
-            if (!rep)
-                goto error;
-
-            if (PyBytes_Check(rep))
-                repsize = PyBytes_GET_SIZE(rep);
-            else
-                repsize = PyUnicode_GET_LENGTH(rep);
-
-            if (repsize > 4) {
-                Py_ssize_t offset;
-
-                if (result == NULL)
-                    offset = p - stackbuf;
-                else
-                    offset = p - PyBytes_AS_STRING(result);
-
-                if (nallocated > PY_SSIZE_T_MAX - repsize + 4) {
-                    /* integer overflow */
-                    PyErr_NoMemory();
-                    goto error;
-                }
-                nallocated += repsize - 4;
-                if (result != NULL) {
-                    if (_PyBytes_Resize(&result, nallocated) < 0)
-                        goto error;
-                } else {
-                    result = PyBytes_FromStringAndSize(NULL, nallocated);
-                    if (result == NULL)
-                        goto error;
-                    Py_MEMCPY(PyBytes_AS_STRING(result), stackbuf, offset);
-                }
-                p = PyBytes_AS_STRING(result) + offset;
-            }
-
-            if (PyBytes_Check(rep)) {
-                char *prep = PyBytes_AS_STRING(rep);
-                for(k = repsize; k > 0; k--)
-                    *p++ = *prep++;
-            } else /* rep is unicode */ {
-                enum PyUnicode_Kind repkind;
-                void *repdata;
-
-                if (PyUnicode_READY(rep) < 0)
-                    goto error;
-                repkind = PyUnicode_KIND(rep);
-                repdata = PyUnicode_DATA(rep);
-
-                for(k=0; k<repsize; k++) {
-                    Py_UCS4 c = PyUnicode_READ(repkind, repdata, k);
-                    if (0x80 <= c) {
-                        raise_encode_exception(&exc, "utf-8",
-                                               unicode,
-                                               i-1, i,
-                                               "surrogates not allowed");
-                        goto error;
-                    }
-                    *p++ = (char)c;
-                }
-            }
-            Py_CLEAR(rep);
-        } else if (ch < 0x10000) {
-            *p++ = (char)(0xe0 | (ch >> 12));
-            *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
-            *p++ = (char)(0x80 | (ch & 0x3f));
-        } else /* ch >= 0x10000 */ {
-            assert(ch <= MAX_UNICODE);
-            /* Encode UCS4 Unicode ordinals */
-            *p++ = (char)(0xf0 | (ch >> 18));
-            *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
-            *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
-            *p++ = (char)(0x80 | (ch & 0x3f));
-        }
-    }
-
-    if (result == NULL) {
-        /* This was stack allocated. */
-        nneeded = p - stackbuf;
-        assert(nneeded <= nallocated);
-        result = PyBytes_FromStringAndSize(stackbuf, nneeded);
-    }
-    else {
-        /* Cut back to size actually needed. */
-        nneeded = p - PyBytes_AS_STRING(result);
-        assert(nneeded <= nallocated);
-        _PyBytes_Resize(&result, nneeded);
-    }
-
-    Py_XDECREF(errorHandler);
-    Py_XDECREF(exc);
-    return result;
- error:
-    Py_XDECREF(rep);
-    Py_XDECREF(errorHandler);
-    Py_XDECREF(exc);
-    Py_XDECREF(result);
-    return NULL;
-
-#undef MAX_SHORT_UNICHARS
+    switch(kind) {
+    default:
+        assert(0);
+    case PyUnicode_1BYTE_KIND:
+        /* the string cannot be ASCII, or PyUnicode_UTF8() would be set */
+        assert(!PyUnicode_IS_ASCII(unicode));
+        return ucs1lib_utf8_encoder(unicode, data, size, errors);
+    case PyUnicode_2BYTE_KIND:
+        return ucs2lib_utf8_encoder(unicode, data, size, errors);
+    case PyUnicode_4BYTE_KIND:
+        return ucs4lib_utf8_encoder(unicode, data, size, errors);
+    }
 }
 
 PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 16:09:40 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 16:09:40 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzgwMzU6?=
 =?utf8?q?_urllib=3A_Fix_a_bug_where_the_client_could_remain_stuck_after_a?=
Message-ID: <E1RcIMu-00023A-D3@dinsdale.python.org>

http://hg.python.org/cpython/rev/038616802b65
changeset:   74033:038616802b65
branch:      2.7
parent:      74030:2863470caebb
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 15:52:48 2011 +0100
summary:
  Issue #8035: urllib: Fix a bug where the client could remain stuck after a
redirection or an error.

files:
  Lib/urllib.py |  2 --
  Misc/NEWS     |  3 +++
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib.py b/Lib/urllib.py
--- a/Lib/urllib.py
+++ b/Lib/urllib.py
@@ -374,7 +374,6 @@
 
     def http_error_default(self, url, fp, errcode, errmsg, headers):
         """Default error handler: close the connection and raise IOError."""
-        void = fp.read()
         fp.close()
         raise IOError, ('http error', errcode, errmsg, headers)
 
@@ -640,7 +639,6 @@
             newurl = headers['uri']
         else:
             return
-        void = fp.read()
         fp.close()
         # In case the server sent a relative URL, join with original:
         newurl = basejoin(self.type + ":" + url, newurl)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,9 @@
 Library
 -------
 
+- Issue #8035: urllib: Fix a bug where the client could remain stuck after a
+  redirection or an error.
+
 - Issue #4625: If IDLE cannot write to its recent file or breakpoint
   files, display a message popup and continue rather than crash.
   (original patch by Roger Serwy)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 16:10:11 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 16:10:11 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzgwMzU6?=
 =?utf8?q?_urllib=3A_Fix_a_bug_where_the_client_could_remain_stuck_after_a?=
Message-ID: <E1RcINP-0002AX-7e@dinsdale.python.org>

http://hg.python.org/cpython/rev/a420b27a86d9
changeset:   74034:a420b27a86d9
branch:      3.2
parent:      74029:edc981ce8748
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 16:05:07 2011 +0100
summary:
  Issue #8035: urllib: Fix a bug where the client could remain stuck after a
redirection or an error.

files:
  Lib/urllib/request.py |  2 --
  Misc/NEWS             |  3 +++
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -1732,7 +1732,6 @@
 
     def http_error_default(self, url, fp, errcode, errmsg, headers):
         """Default error handler: close the connection and raise IOError."""
-        void = fp.read()
         fp.close()
         raise HTTPError(url, errcode, errmsg, headers, None)
 
@@ -1923,7 +1922,6 @@
             newurl = headers['uri']
         else:
             return
-        void = fp.read()
         fp.close()
 
         # In case the server sent a relative URL, join with original:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #8035: urllib: Fix a bug where the client could remain stuck after a
+  redirection or an error.
+
 - Issue #10350: Read and save errno before calling a function which might
   overwrite it.  Original patch by Hallvard B Furuseth.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 16:10:42 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 16:10:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=238035=3A_urllib=3A_Fix_a_bug_where_the_client_could_?=
 =?utf8?q?remain_stuck_after_a?=
Message-ID: <E1RcINu-0002Ct-QQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/764234983120
changeset:   74035:764234983120
parent:      74032:fbd797fc3809
parent:      74034:a420b27a86d9
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 16:08:33 2011 +0100
summary:
  Issue #8035: urllib: Fix a bug where the client could remain stuck after a
redirection or an error.

files:
  Lib/urllib/request.py |  2 --
  Misc/NEWS             |  3 +++
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -1762,7 +1762,6 @@
 
     def http_error_default(self, url, fp, errcode, errmsg, headers):
         """Default error handler: close the connection and raise IOError."""
-        void = fp.read()
         fp.close()
         raise HTTPError(url, errcode, errmsg, headers, None)
 
@@ -1951,7 +1950,6 @@
             newurl = headers['uri']
         else:
             return
-        void = fp.read()
         fp.close()
 
         # In case the server sent a relative URL, join with original:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #8035: urllib: Fix a bug where the client could remain stuck after a
+  redirection or an error.
+
 - Issue #13560: os.strerror() now uses the current locale encoding instead of
   UTF-8.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 18:47:31 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 18:47:31 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzExODcw?=
 =?utf8?q?=3A_threading=3A_Properly_reinitialize_threads_internal_locks_an?=
 =?utf8?q?d?=
Message-ID: <E1RcKpf-0005yT-KI@dinsdale.python.org>

http://hg.python.org/cpython/rev/775319cebaa3
changeset:   74036:775319cebaa3
branch:      2.7
parent:      74033:038616802b65
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 18:22:24 2011 +0100
summary:
  Issue #11870: threading: Properly reinitialize threads internal locks and
condition variables to avoid deadlocks in child processes.

files:
  Lib/threading.py |  13 +++++--------
  Misc/NEWS        |   3 +++
  2 files changed, 8 insertions(+), 8 deletions(-)


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -878,22 +878,19 @@
     current = current_thread()
     with _active_limbo_lock:
         for thread in _active.itervalues():
+            # Any lock/condition variable may be currently locked or in an
+            # invalid state, so we reinitialize them.
+            if hasattr(thread, '_reset_internal_locks'):
+                thread._reset_internal_locks()
             if thread is current:
                 # There is only one active thread. We reset the ident to
                 # its new value since it can have changed.
                 ident = _get_ident()
                 thread._Thread__ident = ident
-                # Any condition variables hanging off of the active thread may
-                # be in an invalid state, so we reinitialize them.
-                if hasattr(thread, '_reset_internal_locks'):
-                    thread._reset_internal_locks()
                 new_active[ident] = thread
             else:
                 # All the others are already stopped.
-                # We don't call _Thread__stop() because it tries to acquire
-                # thread._Thread__block which could also have been held while
-                # we forked.
-                thread._Thread__stopped = True
+                thread._Thread__stop()
 
         _limbo.clear()
         _active.clear()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,9 @@
 Library
 -------
 
+- Issue #11870: threading: Properly reinitialize threads internal locks and
+  condition variables to avoid deadlocks in child processes.
+
 - Issue #8035: urllib: Fix a bug where the client could remain stuck after a
   redirection or an error.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 18:48:02 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 18:48:02 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExODcw?=
 =?utf8?q?=3A_threading=3A_Properly_reinitialize_threads_internal_locks_an?=
 =?utf8?q?d?=
Message-ID: <E1RcKqA-000612-3q@dinsdale.python.org>

http://hg.python.org/cpython/rev/de962ec0faaa
changeset:   74037:de962ec0faaa
branch:      3.2
parent:      74034:a420b27a86d9
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 18:35:09 2011 +0100
summary:
  Issue #11870: threading: Properly reinitialize threads internal locks and
condition variables to avoid deadlocks in child processes.

files:
  Lib/threading.py |  11 ++++-------
  Misc/NEWS        |   3 +++
  2 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -1068,21 +1068,18 @@
     current = current_thread()
     with _active_limbo_lock:
         for thread in _active.values():
+            # Any lock/condition variable may be currently locked or in an
+            # invalid state, so we reinitialize them.
+            thread._reset_internal_locks()
             if thread is current:
                 # There is only one active thread. We reset the ident to
                 # its new value since it can have changed.
                 ident = _get_ident()
                 thread._ident = ident
-                # Any condition variables hanging off of the active thread may
-                # be in an invalid state, so we reinitialize them.
-                thread._reset_internal_locks()
                 new_active[ident] = thread
             else:
                 # All the others are already stopped.
-                # We don't call _Thread__stop() because it tries to acquire
-                # thread._Thread__block which could also have been held while
-                # we forked.
-                thread._stopped = True
+                thread._stop()
 
         _limbo.clear()
         _active.clear()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #11870: threading: Properly reinitialize threads internal locks and
+  condition variables to avoid deadlocks in child processes.
+
 - Issue #8035: urllib: Fix a bug where the client could remain stuck after a
   redirection or an error.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 18:48:33 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 18:48:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2311870=3A_threading?=
 =?utf8?q?=3A_Properly_reinitialize_threads_internal_locks_and?=
Message-ID: <E1RcKqf-00065Q-IL@dinsdale.python.org>

http://hg.python.org/cpython/rev/cec0d77d01c4
changeset:   74038:cec0d77d01c4
parent:      74035:764234983120
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 18:45:16 2011 +0100
summary:
  Issue #11870: threading: Properly reinitialize threads internal locks and
condition variables to avoid deadlocks in child processes.

files:
  Lib/threading.py |  11 ++++-------
  Misc/NEWS        |   3 +++
  2 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -1047,21 +1047,18 @@
     current = current_thread()
     with _active_limbo_lock:
         for thread in _active.values():
+            # Any lock/condition variable may be currently locked or in an
+            # invalid state, so we reinitialize them.
+            thread._reset_internal_locks()
             if thread is current:
                 # There is only one active thread. We reset the ident to
                 # its new value since it can have changed.
                 ident = get_ident()
                 thread._ident = ident
-                # Any condition variables hanging off of the active thread may
-                # be in an invalid state, so we reinitialize them.
-                thread._reset_internal_locks()
                 new_active[ident] = thread
             else:
                 # All the others are already stopped.
-                # We don't call _Thread__stop() because it tries to acquire
-                # thread._Thread__block which could also have been held while
-                # we forked.
-                thread._stopped = True
+                thread._stop()
 
         _limbo.clear()
         _active.clear()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #11870: threading: Properly reinitialize threads internal locks and
+  condition variables to avoid deadlocks in child processes.
+
 - Issue #8035: urllib: Fix a bug where the client could remain stuck after a
   redirection or an error.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 18:49:48 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Sun, 18 Dec 2011 18:49:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Null_merge=2E?=
Message-ID: <E1RcKrs-0006DQ-FU@dinsdale.python.org>

http://hg.python.org/cpython/rev/336805423372
changeset:   74039:336805423372
parent:      74038:cec0d77d01c4
parent:      74037:de962ec0faaa
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Sun Dec 18 18:49:40 2011 +0100
summary:
  Null merge.

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:04:09 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:04:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_In_the_test_SSL_server=2C_a?=
	=?utf8?q?lso_output_the_cipher_name?=
Message-ID: <E1RcL5l-00088U-M8@dinsdale.python.org>

http://hg.python.org/cpython/rev/8335e6c084f6
changeset:   74040:8335e6c084f6
parent:      74032:fbd797fc3809
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:00:02 2011 +0100
summary:
  In the test SSL server, also output the cipher name

files:
  Lib/test/ssl_servers.py |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py
--- a/Lib/test/ssl_servers.py
+++ b/Lib/test/ssl_servers.py
@@ -94,7 +94,11 @@
         """Serve a GET request."""
         sock = self.rfile.raw._sock
         context = sock.context
-        body = pprint.pformat(context.session_stats())
+        stats = {
+            'session_cache': context.session_stats(),
+            'cipher': sock.cipher(),
+            }
+        body = pprint.pformat(stats)
         body = body.encode('utf-8')
         self.send_response(200)
         self.send_header("Content-type", "text/plain; charset=utf-8")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:04:30 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:04:30 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_Merge?=
Message-ID: <E1RcL66-0008BM-Uy@dinsdale.python.org>

http://hg.python.org/cpython/rev/99aaa0a4f23b
changeset:   74041:99aaa0a4f23b
parent:      74040:8335e6c084f6
parent:      74039:336805423372
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:00:16 2011 +0100
summary:
  Merge

files:
  Lib/threading.py      |  11 ++++-------
  Lib/urllib/request.py |   2 --
  Misc/NEWS             |   6 ++++++
  3 files changed, 10 insertions(+), 9 deletions(-)


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -1047,21 +1047,18 @@
     current = current_thread()
     with _active_limbo_lock:
         for thread in _active.values():
+            # Any lock/condition variable may be currently locked or in an
+            # invalid state, so we reinitialize them.
+            thread._reset_internal_locks()
             if thread is current:
                 # There is only one active thread. We reset the ident to
                 # its new value since it can have changed.
                 ident = get_ident()
                 thread._ident = ident
-                # Any condition variables hanging off of the active thread may
-                # be in an invalid state, so we reinitialize them.
-                thread._reset_internal_locks()
                 new_active[ident] = thread
             else:
                 # All the others are already stopped.
-                # We don't call _Thread__stop() because it tries to acquire
-                # thread._Thread__block which could also have been held while
-                # we forked.
-                thread._stopped = True
+                thread._stop()
 
         _limbo.clear()
         _active.clear()
diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -1762,7 +1762,6 @@
 
     def http_error_default(self, url, fp, errcode, errmsg, headers):
         """Default error handler: close the connection and raise IOError."""
-        void = fp.read()
         fp.close()
         raise HTTPError(url, errcode, errmsg, headers, None)
 
@@ -1951,7 +1950,6 @@
             newurl = headers['uri']
         else:
             return
-        void = fp.read()
         fp.close()
 
         # In case the server sent a relative URL, join with original:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,12 @@
 Library
 -------
 
+- Issue #11870: threading: Properly reinitialize threads internal locks and
+  condition variables to avoid deadlocks in child processes.
+
+- Issue #8035: urllib: Fix a bug where the client could remain stuck after a
+  redirection or an error.
+
 - Issue #13560: os.strerror() now uses the current locale encoding instead of
   UTF-8.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:29:03 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 19:29:03 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNjE3?=
 =?utf8?q?=3A_Document_that_the_result_of_the_conversion_of_a_Unicode_obje?=
 =?utf8?q?ct_to?=
Message-ID: <E1RcLTr-00020x-JF@dinsdale.python.org>

http://hg.python.org/cpython/rev/fa5c8cf29963
changeset:   74042:fa5c8cf29963
branch:      3.2
parent:      74037:de962ec0faaa
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 19:22:31 2011 +0100
summary:
  Issue #13617: Document that the result of the conversion of a Unicode object to
wchar*, Py_UNICODE* and bytes may contain embedded null characters/bytes.

Patch written by Arnaud Calmettes.

files:
  Doc/ACKS.txt          |   1 +
  Doc/c-api/unicode.rst |  28 +++++++++++++++++++---------
  2 files changed, 20 insertions(+), 9 deletions(-)


diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
--- a/Doc/ACKS.txt
+++ b/Doc/ACKS.txt
@@ -33,6 +33,7 @@
    * Keith Briggs
    * Ian Bruntlett
    * Lee Busby
+   * Arnaud Calmettes
    * Lorenzo M. Catucci
    * Carl Cerecke
    * Mauro Cicognini
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -338,16 +338,21 @@
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
-   Return a read-only pointer to the Unicode object's internal :c:type:`Py_UNICODE`
-   buffer, *NULL* if *unicode* is not a Unicode object.
+   Return a read-only pointer to the Unicode object's internal
+   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
+   Note that the resulting :c:type:`Py_UNICODE*` string may contain embedded
+   null characters, which would cause the string to be truncated when used in
+   most C functions.
 
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeCopy(PyObject *unicode)
 
    Create a copy of a Unicode string ending with a nul character. Return *NULL*
    and raise a :exc:`MemoryError` exception on memory allocation failure,
-   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free the
-   buffer).
+   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free
+   the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may contain
+   embedded null characters, which would cause the string to be truncated when
+   used in most C functions.
 
    .. versionadded:: 3.2
 
@@ -447,7 +452,8 @@
 
    Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the
    ``'surrogateescape'`` error handler, or ``'strict'`` on Windows, and return
-   :class:`bytes`.
+   :class:`bytes`. Note that the resulting :class:`bytes` object may contain
+   null bytes.
 
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
@@ -476,7 +482,9 @@
    copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t`
    string may or may not be 0-terminated.  It is the responsibility of the caller
    to make sure that the :c:type:`wchar_t` string is 0-terminated in case this is
-   required by the application.
+   required by the application. Also, note that the :c:type:`wchar_t*` string
+   might contain null characters, which would cause the string to be truncated
+   when used with most C functions.
 
 
 .. c:function:: wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size)
@@ -486,9 +494,11 @@
    of wide characters (excluding the trailing 0-termination character) into
    *\*size*.
 
-   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use :c:func:`PyMem_Free`
-   to free it) on success. On error, returns *NULL*, *\*size* is undefined and
-   raises a :exc:`MemoryError`.
+   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use
+   :c:func:`PyMem_Free` to free it) on success. On error, returns *NULL*,
+   *\*size* is undefined and raises a :exc:`MemoryError`. Note that the
+   resulting :c:type:`wchar_t*` string might contain null characters, which
+   would cause the string to be truncated when used with most C functions.
 
    .. versionadded:: 3.2
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:29:34 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 19:29:34 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313617=3A_Document_that_the_result_of_the_conversion?=
 =?utf8?q?_of_a_Unicode_object_to?=
Message-ID: <E1RcLUM-00022P-Bl@dinsdale.python.org>

http://hg.python.org/cpython/rev/f30ac7729f2b
changeset:   74043:f30ac7729f2b
parent:      74041:99aaa0a4f23b
parent:      74042:fa5c8cf29963
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 19:30:55 2011 +0100
summary:
  Issue #13617: Document that the result of the conversion of a Unicode object to
wchar*, Py_UNICODE* and bytes may contain embedded null characters/bytes.

Patch written by Arnaud Calmettes.

files:
  Doc/ACKS.txt          |   1 +
  Doc/c-api/unicode.rst |  37 ++++++++++++++++++++----------
  2 files changed, 25 insertions(+), 13 deletions(-)


diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
--- a/Doc/ACKS.txt
+++ b/Doc/ACKS.txt
@@ -33,6 +33,7 @@
    * Keith Briggs
    * Ian Bruntlett
    * Lee Busby
+   * Arnaud Calmettes
    * Lorenzo M. Catucci
    * Carl Cerecke
    * Mauro Cicognini
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -649,9 +649,11 @@
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
    Return a read-only pointer to the Unicode object's internal
-   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
-   This will create the :c:type:`Py_UNICODE` representation of the object if it
-   is not yet available.
+   :c:type:`Py_UNICODE` buffer, or *NULL* on error. This will create the
+   :c:type:`Py_UNICODE*` representation of the object if it is not yet
+   available. Note that the resulting :c:type:`Py_UNICODE` string may contain
+   embedded null characters, which would cause the string to be truncated when
+   used in most C functions.
 
    Please migrate to using :c:func:`PyUnicode_AsUCS4`,
    :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new
@@ -668,7 +670,9 @@
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
 
    Like :c:func:`PyUnicode_AsUnicode`, but also saves the :c:func:`Py_UNICODE`
-   array length in *size*.
+   array length in *size*. Note that the resulting :c:type:`Py_UNICODE*` string
+   may contain embedded null characters, which would cause the string to be
+   truncated when used in most C functions.
 
    .. versionadded:: 3.3
 
@@ -677,8 +681,10 @@
 
    Create a copy of a Unicode string ending with a nul character. Return *NULL*
    and raise a :exc:`MemoryError` exception on memory allocation failure,
-   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free the
-   buffer).
+   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free
+   the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may
+   contain embedded null characters, which would cause the string to be
+   truncated when used in most C functions.
 
    .. versionadded:: 3.2
 
@@ -817,7 +823,8 @@
 
    Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the
    ``'surrogateescape'`` error handler, or ``'strict'`` on Windows, and return
-   :class:`bytes`.
+   :class:`bytes`. Note that the resulting :class:`bytes` object may contain
+   null bytes.
 
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
@@ -850,10 +857,12 @@
    Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*.  At most
    *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing
    0-termination character).  Return the number of :c:type:`wchar_t` characters
-   copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t`
+   copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t*`
    string may or may not be 0-terminated.  It is the responsibility of the caller
-   to make sure that the :c:type:`wchar_t` string is 0-terminated in case this is
-   required by the application.
+   to make sure that the :c:type:`wchar_t*` string is 0-terminated in case this is
+   required by the application. Also, note that the :c:type:`wchar_t*` string
+   might contain null characters, which would cause the string to be truncated
+   when used with most C functions.
 
 
 .. c:function:: wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size)
@@ -863,9 +872,11 @@
    of wide characters (excluding the trailing 0-termination character) into
    *\*size*.
 
-   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use :c:func:`PyMem_Free`
-   to free it) on success. On error, returns *NULL*, *\*size* is undefined and
-   raises a :exc:`MemoryError`.
+   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use
+   :c:func:`PyMem_Free` to free it) on success. On error, returns *NULL*,
+   *\*size* is undefined and raises a :exc:`MemoryError`. Note that the
+   resulting :c:type:`wchar_t` string might contain null characters, which
+   would cause the string to be truncated when used with most C functions.
 
    .. versionadded:: 3.2
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:29:51 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 18 Dec 2011 19:29:51 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Small_clarifica?=
 =?utf8?q?tion_in_docstring_of_dict=2Eupdate=28=29=3A_the_positional_argum?=
 =?utf8?q?ent_is?=
Message-ID: <E1RcLUd-000233-1S@dinsdale.python.org>

http://hg.python.org/cpython/rev/393906d020fe
changeset:   74044:393906d020fe
branch:      3.2
parent:      74042:fa5c8cf29963
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 18 19:30:55 2011 +0100
summary:
  Small clarification in docstring of dict.update(): the positional argument is not required.

files:
  Objects/dictobject.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1973,9 +1973,9 @@
 2-tuple; but raise KeyError if D is empty.");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:30:21 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 18 Dec 2011 19:30:21 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_with_3=2E2=2E?=
Message-ID: <E1RcLV7-0002BP-FQ@dinsdale.python.org>

http://hg.python.org/cpython/rev/e49fdf7d149b
changeset:   74045:e49fdf7d149b
parent:      74043:f30ac7729f2b
parent:      74044:393906d020fe
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 18 19:32:37 2011 +0100
summary:
  Merge with 3.2.

files:
  Objects/dictobject.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1981,9 +1981,9 @@
 2-tuple; but raise KeyError if D is empty.");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:31:35 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 18 Dec 2011 19:31:35 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Small_clarifica?=
 =?utf8?q?tion_in_docstring_of_dict=2Eupdate=28=29=3A_the_positional_argum?=
 =?utf8?q?ent_is?=
Message-ID: <E1RcLWJ-0002GA-4e@dinsdale.python.org>

http://hg.python.org/cpython/rev/d9bb270feb58
changeset:   74046:d9bb270feb58
branch:      2.7
parent:      74036:775319cebaa3
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 18 19:30:55 2011 +0100
summary:
  Small clarification in docstring of dict.update(): the positional argument is not required.

files:
  Objects/dictobject.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2167,9 +2167,9 @@
 "D.values() -> list of D's values");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:32:12 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:32:12 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzc1MDI6?=
 =?utf8?q?_Fix_equality_comparison_for_DocTestCase_instances=2E?=
Message-ID: <E1RcLWu-0002I3-NA@dinsdale.python.org>

http://hg.python.org/cpython/rev/c478734ded4b
changeset:   74047:c478734ded4b
branch:      3.2
parent:      74034:a420b27a86d9
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:27:45 2011 +0100
summary:
  Issue #7502: Fix equality comparison for DocTestCase instances.
Patch by C?dric Krier.

files:
  Lib/doctest.py           |  41 ++++++++++++++++++++++++++++
  Lib/test/test_doctest.py |  40 +++++++++++++++++++++++++++
  Misc/ACKS                |   1 +
  Misc/NEWS                |   3 ++
  4 files changed, 85 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -440,6 +440,21 @@
         self.options = options
         self.exc_msg = exc_msg
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.source == other.source and \
+               self.want == other.want and \
+               self.lineno == other.lineno and \
+               self.indent == other.indent and \
+               self.options == other.options and \
+               self.exc_msg == other.exc_msg
+
+    def __ne__(self, other):
+        return not self == other
+
+
 class DocTest:
     """
     A collection of doctest examples that should be run in a single
@@ -488,6 +503,19 @@
         return ('<DocTest %s from %s:%s (%s)>' %
                 (self.name, self.filename, self.lineno, examples))
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.examples == other.examples and \
+               self.docstring == other.docstring and \
+               self.globs == other.globs and \
+               self.name == other.name and \
+               self.filename == other.filename and \
+               self.lineno == other.lineno
+
+    def __ne__(self, other):
+        return not self == other
 
     # This lets us sort tests by name:
     def __lt__(self, other):
@@ -2204,6 +2232,19 @@
     def id(self):
         return self._dt_test.name
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self._dt_test == other._dt_test and \
+               self._dt_optionflags == other._dt_optionflags and \
+               self._dt_setUp == other._dt_setUp and \
+               self._dt_tearDown == other._dt_tearDown and \
+               self._dt_checker == other._dt_checker
+
+    def __ne__(self, other):
+        return not self == other
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -347,6 +347,46 @@
     Traceback (most recent call last):
     ValueError: line 2 of the docstring for some_test lacks blank after ...: '...print(1)'
 
+Compare `DocTest`:
+
+    >>> docstring = '''
+    ...     >>> print 12
+    ...     12
+    ... '''
+    >>> test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                           'some_test', 20)
+    >>> same_test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                                'some_test', 20)
+    >>> test == same_test
+    True
+    >>> test != same_test
+    False
+    >>> docstring = '''
+    ...     >>> print 42
+    ...     42
+    ... '''
+    >>> other_test = parser.get_doctest(docstring, globs, 'other_test',
+    ...                                 'other_file', 10)
+    >>> test == other_test
+    False
+    >>> test != other_test
+    True
+
+Compare `DocTestCase`:
+
+    >>> DocTestCase = doctest.DocTestCase
+    >>> test_case = DocTestCase(test)
+    >>> same_test_case = DocTestCase(same_test)
+    >>> other_test_case = DocTestCase(other_test)
+    >>> test_case == same_test_case
+    True
+    >>> test_case != same_test_case
+    False
+    >>> test == other_test_case
+    False
+    >>> test != other_test_case
+    True
+
 """
 
 def test_DocTestFinder(): r"""
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -509,6 +509,7 @@
 Holger Krekel
 Michael Kremer
 Fabian Kreutz
+C?dric Krier
 Hannu Krosing
 Andrej Krpic
 Ivan Krsti?
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
+  C?dric Krier.
+
 - Issue #8035: urllib: Fix a bug where the client could remain stuck after a
   redirection or an error.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:32:52 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:32:52 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMik6?=
	=?utf8?q?_Merge?=
Message-ID: <E1RcLXY-0002Kd-7h@dinsdale.python.org>

http://hg.python.org/cpython/rev/a175284ede57
changeset:   74048:a175284ede57
branch:      3.2
parent:      74047:c478734ded4b
parent:      74037:de962ec0faaa
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:28:33 2011 +0100
summary:
  Merge

files:
  Lib/threading.py |  11 ++++-------
  Misc/NEWS        |   3 +++
  2 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -1068,21 +1068,18 @@
     current = current_thread()
     with _active_limbo_lock:
         for thread in _active.values():
+            # Any lock/condition variable may be currently locked or in an
+            # invalid state, so we reinitialize them.
+            thread._reset_internal_locks()
             if thread is current:
                 # There is only one active thread. We reset the ident to
                 # its new value since it can have changed.
                 ident = _get_ident()
                 thread._ident = ident
-                # Any condition variables hanging off of the active thread may
-                # be in an invalid state, so we reinitialize them.
-                thread._reset_internal_locks()
                 new_active[ident] = thread
             else:
                 # All the others are already stopped.
-                # We don't call _Thread__stop() because it tries to acquire
-                # thread._Thread__block which could also have been held while
-                # we forked.
-                thread._stopped = True
+                thread._stop()
 
         _limbo.clear()
         _active.clear()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -100,6 +100,9 @@
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
   C?dric Krier.
 
+- Issue #11870: threading: Properly reinitialize threads internal locks and
+  condition variables to avoid deadlocks in child processes.
+
 - Issue #8035: urllib: Fix a bug where the client could remain stuck after a
   redirection or an error.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:33:22 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:33:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=237502=3A_Fix_equality_comparison_for_DocTestCase_ins?=
 =?utf8?q?tances=2E?=
Message-ID: <E1RcLY2-0002NB-Np@dinsdale.python.org>

http://hg.python.org/cpython/rev/b8cb6f1e4981
changeset:   74049:b8cb6f1e4981
parent:      74041:99aaa0a4f23b
parent:      74048:a175284ede57
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:29:06 2011 +0100
summary:
  Issue #7502: Fix equality comparison for DocTestCase instances.
Patch by C?dric Krier.

files:
  Lib/doctest.py           |  41 ++++++++++++++++++++++++++++
  Lib/test/test_doctest.py |  40 +++++++++++++++++++++++++++
  Misc/ACKS                |   1 +
  Misc/NEWS                |   3 ++
  4 files changed, 85 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -440,6 +440,21 @@
         self.options = options
         self.exc_msg = exc_msg
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.source == other.source and \
+               self.want == other.want and \
+               self.lineno == other.lineno and \
+               self.indent == other.indent and \
+               self.options == other.options and \
+               self.exc_msg == other.exc_msg
+
+    def __ne__(self, other):
+        return not self == other
+
+
 class DocTest:
     """
     A collection of doctest examples that should be run in a single
@@ -488,6 +503,19 @@
         return ('<DocTest %s from %s:%s (%s)>' %
                 (self.name, self.filename, self.lineno, examples))
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.examples == other.examples and \
+               self.docstring == other.docstring and \
+               self.globs == other.globs and \
+               self.name == other.name and \
+               self.filename == other.filename and \
+               self.lineno == other.lineno
+
+    def __ne__(self, other):
+        return not self == other
 
     # This lets us sort tests by name:
     def __lt__(self, other):
@@ -2206,6 +2234,19 @@
     def id(self):
         return self._dt_test.name
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self._dt_test == other._dt_test and \
+               self._dt_optionflags == other._dt_optionflags and \
+               self._dt_setUp == other._dt_setUp and \
+               self._dt_tearDown == other._dt_tearDown and \
+               self._dt_checker == other._dt_checker
+
+    def __ne__(self, other):
+        return not self == other
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -348,6 +348,46 @@
     Traceback (most recent call last):
     ValueError: line 2 of the docstring for some_test lacks blank after ...: '...print(1)'
 
+Compare `DocTest`:
+
+    >>> docstring = '''
+    ...     >>> print 12
+    ...     12
+    ... '''
+    >>> test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                           'some_test', 20)
+    >>> same_test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                                'some_test', 20)
+    >>> test == same_test
+    True
+    >>> test != same_test
+    False
+    >>> docstring = '''
+    ...     >>> print 42
+    ...     42
+    ... '''
+    >>> other_test = parser.get_doctest(docstring, globs, 'other_test',
+    ...                                 'other_file', 10)
+    >>> test == other_test
+    False
+    >>> test != other_test
+    True
+
+Compare `DocTestCase`:
+
+    >>> DocTestCase = doctest.DocTestCase
+    >>> test_case = DocTestCase(test)
+    >>> same_test_case = DocTestCase(same_test)
+    >>> other_test_case = DocTestCase(other_test)
+    >>> test_case == same_test_case
+    True
+    >>> test_case != same_test_case
+    False
+    >>> test == other_test_case
+    False
+    >>> test != other_test_case
+    True
+
 """
 
 def test_DocTestFinder(): r"""
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -549,6 +549,7 @@
 Holger Krekel
 Michael Kremer
 Fabian Kreutz
+C?dric Krier
 Hannu Krosing
 Andrej Krpic
 Ivan Krsti?
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
+  C?dric Krier.
+
 - Issue #11870: threading: Properly reinitialize threads internal locks and
   condition variables to avoid deadlocks in child processes.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:33:56 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:33:56 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzc1MDI6?=
 =?utf8?q?_Fix_equality_comparison_for_DocTestCase_instances=2E?=
Message-ID: <E1RcLYa-0002PH-6g@dinsdale.python.org>

http://hg.python.org/cpython/rev/64d670a8b183
changeset:   74050:64d670a8b183
branch:      2.7
parent:      74036:775319cebaa3
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:27:45 2011 +0100
summary:
  Issue #7502: Fix equality comparison for DocTestCase instances.
Patch by C?dric Krier.

files:
  Lib/doctest.py           |  41 ++++++++++++++++++++++++++++
  Lib/test/test_doctest.py |  40 +++++++++++++++++++++++++++
  Misc/ACKS                |   1 +
  Misc/NEWS                |   3 ++
  4 files changed, 85 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -451,6 +451,21 @@
         self.options = options
         self.exc_msg = exc_msg
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.source == other.source and \
+               self.want == other.want and \
+               self.lineno == other.lineno and \
+               self.indent == other.indent and \
+               self.options == other.options and \
+               self.exc_msg == other.exc_msg
+
+    def __ne__(self, other):
+        return not self == other
+
+
 class DocTest:
     """
     A collection of doctest examples that should be run in a single
@@ -499,6 +514,19 @@
         return ('<DocTest %s from %s:%s (%s)>' %
                 (self.name, self.filename, self.lineno, examples))
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self.examples == other.examples and \
+               self.docstring == other.docstring and \
+               self.globs == other.globs and \
+               self.name == other.name and \
+               self.filename == other.filename and \
+               self.lineno == other.lineno
+
+    def __ne__(self, other):
+        return not self == other
 
     # This lets us sort tests by name:
     def __cmp__(self, other):
@@ -2252,6 +2280,19 @@
     def id(self):
         return self._dt_test.name
 
+    def __eq__(self, other):
+        if type(self) is not type(other):
+            return NotImplemented
+
+        return self._dt_test == other._dt_test and \
+               self._dt_optionflags == other._dt_optionflags and \
+               self._dt_setUp == other._dt_setUp and \
+               self._dt_tearDown == other._dt_tearDown and \
+               self._dt_checker == other._dt_checker
+
+    def __ne__(self, other):
+        return not self == other
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -347,6 +347,46 @@
     Traceback (most recent call last):
     ValueError: line 2 of the docstring for some_test lacks blank after ...: '...print 1'
 
+Compare `DocTest`:
+
+    >>> docstring = '''
+    ...     >>> print 12
+    ...     12
+    ... '''
+    >>> test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                           'some_test', 20)
+    >>> same_test = parser.get_doctest(docstring, globs, 'some_test',
+    ...                                'some_test', 20)
+    >>> test == same_test
+    True
+    >>> test != same_test
+    False
+    >>> docstring = '''
+    ...     >>> print 42
+    ...     42
+    ... '''
+    >>> other_test = parser.get_doctest(docstring, globs, 'other_test',
+    ...                                 'other_file', 10)
+    >>> test == other_test
+    False
+    >>> test != other_test
+    True
+
+Compare `DocTestCase`:
+
+    >>> DocTestCase = doctest.DocTestCase
+    >>> test_case = DocTestCase(test)
+    >>> same_test_case = DocTestCase(same_test)
+    >>> other_test_case = DocTestCase(other_test)
+    >>> test_case == same_test_case
+    True
+    >>> test_case != same_test_case
+    False
+    >>> test == other_test_case
+    False
+    >>> test != other_test_case
+    True
+
 """
 
 def test_DocTestFinder(): r"""
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -464,6 +464,7 @@
 Holger Krekel
 Michael Kremer
 Fabian Kreutz
+C?dric Krier
 Hannu Krosing
 Andrej Krpic
 Ivan Krsti?
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,9 @@
 Library
 -------
 
+- Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
+  C?dric Krier.
+
 - Issue #11870: threading: Properly reinitialize threads internal locks and
   condition variables to avoid deadlocks in child processes.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:34:17 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:34:17 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMik6?=
	=?utf8?q?_Merge?=
Message-ID: <E1RcLYv-0002PT-EW@dinsdale.python.org>

http://hg.python.org/cpython/rev/c5dbe4027d29
changeset:   74051:c5dbe4027d29
branch:      3.2
parent:      74048:a175284ede57
parent:      74044:393906d020fe
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:31:02 2011 +0100
summary:
  Merge

files:
  Doc/ACKS.txt          |   1 +
  Doc/c-api/unicode.rst |  28 +++++++++++++++++++---------
  Objects/dictobject.c  |   6 +++---
  3 files changed, 23 insertions(+), 12 deletions(-)


diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
--- a/Doc/ACKS.txt
+++ b/Doc/ACKS.txt
@@ -33,6 +33,7 @@
    * Keith Briggs
    * Ian Bruntlett
    * Lee Busby
+   * Arnaud Calmettes
    * Lorenzo M. Catucci
    * Carl Cerecke
    * Mauro Cicognini
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -338,16 +338,21 @@
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
-   Return a read-only pointer to the Unicode object's internal :c:type:`Py_UNICODE`
-   buffer, *NULL* if *unicode* is not a Unicode object.
+   Return a read-only pointer to the Unicode object's internal
+   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
+   Note that the resulting :c:type:`Py_UNICODE*` string may contain embedded
+   null characters, which would cause the string to be truncated when used in
+   most C functions.
 
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeCopy(PyObject *unicode)
 
    Create a copy of a Unicode string ending with a nul character. Return *NULL*
    and raise a :exc:`MemoryError` exception on memory allocation failure,
-   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free the
-   buffer).
+   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free
+   the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may contain
+   embedded null characters, which would cause the string to be truncated when
+   used in most C functions.
 
    .. versionadded:: 3.2
 
@@ -447,7 +452,8 @@
 
    Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the
    ``'surrogateescape'`` error handler, or ``'strict'`` on Windows, and return
-   :class:`bytes`.
+   :class:`bytes`. Note that the resulting :class:`bytes` object may contain
+   null bytes.
 
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
@@ -476,7 +482,9 @@
    copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t`
    string may or may not be 0-terminated.  It is the responsibility of the caller
    to make sure that the :c:type:`wchar_t` string is 0-terminated in case this is
-   required by the application.
+   required by the application. Also, note that the :c:type:`wchar_t*` string
+   might contain null characters, which would cause the string to be truncated
+   when used with most C functions.
 
 
 .. c:function:: wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size)
@@ -486,9 +494,11 @@
    of wide characters (excluding the trailing 0-termination character) into
    *\*size*.
 
-   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use :c:func:`PyMem_Free`
-   to free it) on success. On error, returns *NULL*, *\*size* is undefined and
-   raises a :exc:`MemoryError`.
+   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use
+   :c:func:`PyMem_Free` to free it) on success. On error, returns *NULL*,
+   *\*size* is undefined and raises a :exc:`MemoryError`. Note that the
+   resulting :c:type:`wchar_t*` string might contain null characters, which
+   would cause the string to be truncated when used with most C functions.
 
    .. versionadded:: 3.2
 
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1973,9 +1973,9 @@
 2-tuple; but raise KeyError if D is empty.");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:34:38 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:34:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_Merge?=
Message-ID: <E1RcLZG-0002PZ-M9@dinsdale.python.org>

http://hg.python.org/cpython/rev/6373cbafd6ec
changeset:   74052:6373cbafd6ec
parent:      74049:b8cb6f1e4981
parent:      74045:e49fdf7d149b
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:31:14 2011 +0100
summary:
  Merge

files:
  Doc/ACKS.txt          |   1 +
  Doc/c-api/unicode.rst |  37 ++++++++++++++++++++----------
  Objects/dictobject.c  |   6 ++--
  3 files changed, 28 insertions(+), 16 deletions(-)


diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
--- a/Doc/ACKS.txt
+++ b/Doc/ACKS.txt
@@ -33,6 +33,7 @@
    * Keith Briggs
    * Ian Bruntlett
    * Lee Busby
+   * Arnaud Calmettes
    * Lorenzo M. Catucci
    * Carl Cerecke
    * Mauro Cicognini
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -649,9 +649,11 @@
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
    Return a read-only pointer to the Unicode object's internal
-   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
-   This will create the :c:type:`Py_UNICODE` representation of the object if it
-   is not yet available.
+   :c:type:`Py_UNICODE` buffer, or *NULL* on error. This will create the
+   :c:type:`Py_UNICODE*` representation of the object if it is not yet
+   available. Note that the resulting :c:type:`Py_UNICODE` string may contain
+   embedded null characters, which would cause the string to be truncated when
+   used in most C functions.
 
    Please migrate to using :c:func:`PyUnicode_AsUCS4`,
    :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new
@@ -668,7 +670,9 @@
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
 
    Like :c:func:`PyUnicode_AsUnicode`, but also saves the :c:func:`Py_UNICODE`
-   array length in *size*.
+   array length in *size*. Note that the resulting :c:type:`Py_UNICODE*` string
+   may contain embedded null characters, which would cause the string to be
+   truncated when used in most C functions.
 
    .. versionadded:: 3.3
 
@@ -677,8 +681,10 @@
 
    Create a copy of a Unicode string ending with a nul character. Return *NULL*
    and raise a :exc:`MemoryError` exception on memory allocation failure,
-   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free the
-   buffer).
+   otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free
+   the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may
+   contain embedded null characters, which would cause the string to be
+   truncated when used in most C functions.
 
    .. versionadded:: 3.2
 
@@ -817,7 +823,8 @@
 
    Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the
    ``'surrogateescape'`` error handler, or ``'strict'`` on Windows, and return
-   :class:`bytes`.
+   :class:`bytes`. Note that the resulting :class:`bytes` object may contain
+   null bytes.
 
    If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the
    locale encoding.
@@ -850,10 +857,12 @@
    Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*.  At most
    *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing
    0-termination character).  Return the number of :c:type:`wchar_t` characters
-   copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t`
+   copied or -1 in case of an error.  Note that the resulting :c:type:`wchar_t*`
    string may or may not be 0-terminated.  It is the responsibility of the caller
-   to make sure that the :c:type:`wchar_t` string is 0-terminated in case this is
-   required by the application.
+   to make sure that the :c:type:`wchar_t*` string is 0-terminated in case this is
+   required by the application. Also, note that the :c:type:`wchar_t*` string
+   might contain null characters, which would cause the string to be truncated
+   when used with most C functions.
 
 
 .. c:function:: wchar_t* PyUnicode_AsWideCharString(PyObject *unicode, Py_ssize_t *size)
@@ -863,9 +872,11 @@
    of wide characters (excluding the trailing 0-termination character) into
    *\*size*.
 
-   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use :c:func:`PyMem_Free`
-   to free it) on success. On error, returns *NULL*, *\*size* is undefined and
-   raises a :exc:`MemoryError`.
+   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use
+   :c:func:`PyMem_Free` to free it) on success. On error, returns *NULL*,
+   *\*size* is undefined and raises a :exc:`MemoryError`. Note that the
+   resulting :c:type:`wchar_t` string might contain null characters, which
+   would cause the string to be truncated when used with most C functions.
 
    .. versionadded:: 3.2
 
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1981,9 +1981,9 @@
 2-tuple; but raise KeyError if D is empty.");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:35:18 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 19:35:18 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?=
	=?utf8?q?_Merge?=
Message-ID: <E1RcLZu-0002XV-3X@dinsdale.python.org>

http://hg.python.org/cpython/rev/27bf49316692
changeset:   74053:27bf49316692
branch:      2.7
parent:      74050:64d670a8b183
parent:      74046:d9bb270feb58
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 19:31:31 2011 +0100
summary:
  Merge

files:
  Objects/dictobject.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2167,9 +2167,9 @@
 "D.values() -> list of D's values");
 
 PyDoc_STRVAR(update__doc__,
-"D.update(E, **F) -> None.  Update D from dict/iterable E and F.\n"
-"If E has a .keys() method, does:     for k in E: D[k] = E[k]\n\
-If E lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
+"D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n"
+"If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n\
+If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n\
 In either case, this is followed by: for k in F: D[k] = F[k]");
 
 PyDoc_STRVAR(fromkeys__doc__,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 19:38:02 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 19:38:02 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNjE3?=
 =?utf8?q?=3A_Document_that_the_result_PyUnicode=5FAsUnicode=28=29_and?=
Message-ID: <E1RcLcY-0002dt-2q@dinsdale.python.org>

http://hg.python.org/cpython/rev/1c4d9534263e
changeset:   74054:1c4d9534263e
branch:      2.7
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 19:39:53 2011 +0100
summary:
  Issue #13617: Document that the result PyUnicode_AsUnicode() and
PyUnicode_AsWideChar() may contain embedded null characters.

Patch written by Arnaud Calmettes.

files:
  Doc/ACKS.txt          |   1 +
  Doc/c-api/unicode.rst |  11 ++++++++---
  2 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
--- a/Doc/ACKS.txt
+++ b/Doc/ACKS.txt
@@ -33,6 +33,7 @@
    * Keith Briggs
    * Ian Bruntlett
    * Lee Busby
+   * Arnaud Calmettes
    * Lorenzo M. Catucci
    * Carl Cerecke
    * Mauro Cicognini
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -327,8 +327,11 @@
 
 .. cfunction:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
-   Return a read-only pointer to the Unicode object's internal :ctype:`Py_UNICODE`
-   buffer, *NULL* if *unicode* is not a Unicode object.
+   Return a read-only pointer to the Unicode object's internal
+   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
+   Note that the resulting :c:type:`Py_UNICODE*` string may contain embedded
+   null characters, which would cause the string to be truncated when used in
+   most C functions.
 
 
 .. cfunction:: Py_ssize_t PyUnicode_GetSize(PyObject *unicode)
@@ -391,7 +394,9 @@
    copied or -1 in case of an error.  Note that the resulting :ctype:`wchar_t`
    string may or may not be 0-terminated.  It is the responsibility of the caller
    to make sure that the :ctype:`wchar_t` string is 0-terminated in case this is
-   required by the application.
+   required by the application. Also, note that the :c:type:`wchar_t*` string
+   might contain null characters, which would cause the string to be truncated
+   when used with most C functions.
 
    .. versionchanged:: 2.5
       This function returned an :ctype:`int` type and used an :ctype:`int`

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 20:20:53 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 20:20:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Followup_to_=23?=
 =?utf8?q?7502=3A_add_=5F=5Fhash=5F=5F_method_and_tests=2E?=
Message-ID: <E1RcMI1-0005yK-G2@dinsdale.python.org>

http://hg.python.org/cpython/rev/6a95820b9607
changeset:   74055:6a95820b9607
branch:      2.7
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 20:20:17 2011 +0100
summary:
  Followup to #7502: add __hash__ method and tests.

files:
  Lib/doctest.py           |  11 +++++++++++
  Lib/test/test_doctest.py |  19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -465,6 +465,10 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.source, self.want, self.lineno, self.indent,
+                     self.exc_msg))
+
 
 class DocTest:
     """
@@ -528,6 +532,9 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.docstring, self.name, self.filename, self.lineno))
+
     # This lets us sort tests by name:
     def __cmp__(self, other):
         if not isinstance(other, DocTest):
@@ -2293,6 +2300,10 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self._dt_optionflags, self._dt_setUp, self._dt_tearDown,
+                     self._dt_checker))
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -258,6 +258,21 @@
     >>> e = doctest.Example('raise X()', '', exc_msg)
     >>> e.exc_msg
     '\n'
+
+Compare `Example`:
+    >>> example = doctest.Example('print 1', '1\n')
+    >>> same_example = doctest.Example('print 1', '1\n')
+    >>> other_example = doctest.Example('print 42', '42\n')
+    >>> example == same_example
+    True
+    >>> example != same_example
+    False
+    >>> hash(example) == hash(same_example)
+    True
+    >>> example == other_example
+    False
+    >>> example != other_example
+    True
 """
 
 def test_DocTest(): r"""
@@ -361,6 +376,8 @@
     True
     >>> test != same_test
     False
+    >>> hash(test) == hash(same_test)
+    True
     >>> docstring = '''
     ...     >>> print 42
     ...     42
@@ -382,6 +399,8 @@
     True
     >>> test_case != same_test_case
     False
+    >>> hash(test_case) == hash(same_test_case)
+    True
     >>> test == other_test_case
     False
     >>> test != other_test_case

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 20:23:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 20:23:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Followup_to_=23?=
 =?utf8?q?7502=3A_add_=5F=5Fhash=5F=5F_method_and_tests=2E?=
Message-ID: <E1RcMKh-0006DD-Na@dinsdale.python.org>

http://hg.python.org/cpython/rev/e37c71698409
changeset:   74056:e37c71698409
branch:      3.2
parent:      74051:c5dbe4027d29
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 20:20:17 2011 +0100
summary:
  Followup to #7502: add __hash__ method and tests.

files:
  Lib/doctest.py           |  11 +++++++++++
  Lib/test/test_doctest.py |  19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -454,6 +454,10 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.source, self.want, self.lineno, self.indent,
+                     self.exc_msg))
+
 
 class DocTest:
     """
@@ -517,6 +521,9 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.docstring, self.name, self.filename, self.lineno))
+
     # This lets us sort tests by name:
     def __lt__(self, other):
         if not isinstance(other, DocTest):
@@ -2245,6 +2252,10 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self._dt_optionflags, self._dt_setUp, self._dt_tearDown,
+                     self._dt_checker))
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -258,6 +258,21 @@
     >>> e = doctest.Example('raise X()', '', exc_msg)
     >>> e.exc_msg
     '\n'
+
+Compare `Example`:
+    >>> example = doctest.Example('print 1', '1\n')
+    >>> same_example = doctest.Example('print 1', '1\n')
+    >>> other_example = doctest.Example('print 42', '42\n')
+    >>> example == same_example
+    True
+    >>> example != same_example
+    False
+    >>> hash(example) == hash(same_example)
+    True
+    >>> example == other_example
+    False
+    >>> example != other_example
+    True
 """
 
 def test_DocTest(): r"""
@@ -361,6 +376,8 @@
     True
     >>> test != same_test
     False
+    >>> hash(test) == hash(same_test)
+    True
     >>> docstring = '''
     ...     >>> print 42
     ...     42
@@ -382,6 +399,8 @@
     True
     >>> test_case != same_test_case
     False
+    >>> hash(test_case) == hash(same_test_case)
+    True
     >>> test == other_test_case
     False
     >>> test != other_test_case

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 20:24:01 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Sun, 18 Dec 2011 20:24:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Followup_to_=237502=3A_add_=5F=5Fhash=5F=5F_method_and_tests?=
 =?utf8?q?=2E?=
Message-ID: <E1RcML3-0006F5-15@dinsdale.python.org>

http://hg.python.org/cpython/rev/4ffa9992a7d8
changeset:   74057:4ffa9992a7d8
parent:      74052:6373cbafd6ec
parent:      74056:e37c71698409
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sun Dec 18 20:22:50 2011 +0100
summary:
  Followup to #7502: add __hash__ method and tests.

files:
  Lib/doctest.py           |  10 ++++++++++
  Lib/test/test_doctest.py |  19 +++++++++++++++++++
  2 files changed, 29 insertions(+), 0 deletions(-)


diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -454,6 +454,9 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.source, self.want, self.lineno, self.indent,
+                     self.exc_msg))
 
 class DocTest:
     """
@@ -517,6 +520,9 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self.docstring, self.name, self.filename, self.lineno))
+
     # This lets us sort tests by name:
     def __lt__(self, other):
         if not isinstance(other, DocTest):
@@ -2247,6 +2253,10 @@
     def __ne__(self, other):
         return not self == other
 
+    def __hash__(self):
+        return hash((self._dt_optionflags, self._dt_setUp, self._dt_tearDown,
+                     self._dt_checker))
+
     def __repr__(self):
         name = self._dt_test.name.split('.')
         return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -259,6 +259,21 @@
     >>> e = doctest.Example('raise X()', '', exc_msg)
     >>> e.exc_msg
     '\n'
+
+Compare `Example`:
+    >>> example = doctest.Example('print 1', '1\n')
+    >>> same_example = doctest.Example('print 1', '1\n')
+    >>> other_example = doctest.Example('print 42', '42\n')
+    >>> example == same_example
+    True
+    >>> example != same_example
+    False
+    >>> hash(example) == hash(same_example)
+    True
+    >>> example == other_example
+    False
+    >>> example != other_example
+    True
 """
 
 def test_DocTest(): r"""
@@ -362,6 +377,8 @@
     True
     >>> test != same_test
     False
+    >>> hash(test) == hash(same_test)
+    True
     >>> docstring = '''
     ...     >>> print 42
     ...     42
@@ -383,6 +400,8 @@
     True
     >>> test_case != same_test_case
     False
+    >>> hash(test_case) == hash(same_test_case)
+    True
     >>> test == other_test_case
     False
     >>> test != other_test_case

-- 
Repository URL: http://hg.python.org/cpython

From martin at v.loewis.de  Sun Dec 18 20:34:49 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 18 Dec 2011 20:34:49 +0100
Subject: [Python-checkins] cpython: Move PyUnicode_WCHAR_KIND outside
 PyUnicode_Kind enum
In-Reply-To: <E1Rc1xb-0001Rp-OL@dinsdale.python.org>
References: <E1Rc1xb-0001Rp-OL@dinsdale.python.org>
Message-ID: <4EEE4059.5040807@v.loewis.de>

>   Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum

What's the rationale for that change? It's a valid kind value, after
all, and the C convention is that an enumeration lists all valid values
(else there wouldn't be a need for an enumeration in the first place).

Regards,
Martin

From python-checkins at python.org  Sun Dec 18 21:03:46 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 21:03:46 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_import=2Ec_now_catchs_=5FPy?=
	=?utf8?q?=5Fstat=28=29_exceptions?=
Message-ID: <E1RcMxW-00025H-Hk@dinsdale.python.org>

http://hg.python.org/cpython/rev/2a140b01d343
changeset:   74058:2a140b01d343
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 20:47:30 2011 +0100
summary:
  import.c now catchs _Py_stat() exceptions

_Py_stat() now returns -2 if an exception was raised.

files:
  Modules/zipimport.c |   4 +-
  Python/fileutils.c  |   8 +++---
  Python/import.c     |  38 ++++++++++++++++++++++++++------
  3 files changed, 37 insertions(+), 13 deletions(-)


diff --git a/Modules/zipimport.c b/Modules/zipimport.c
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -102,6 +102,8 @@
         int rv;
 
         rv = _Py_stat(filename, &statbuf);
+        if (rv == -2)
+            goto error;
         if (rv == 0) {
             /* it exists */
             if (!S_ISREG(statbuf.st_mode))
@@ -109,8 +111,6 @@
                 Py_CLEAR(filename);
             break;
         }
-        else if (PyErr_Occurred())
-            goto error;
         Py_CLEAR(filename);
         /* back up one path element */
         flen = PyUnicode_FindChar(path, SEP, 0, flen, -1);
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -240,8 +240,8 @@
 /* Call _wstat() on Windows, or encode the path to the filesystem encoding and
    call stat() otherwise. Only fill st_mode attribute on Windows.
 
-   Return 0 on success, -1 on _wstat() / stat() error or (if PyErr_Occurred())
-   unicode error. */
+   Return 0 on success, -1 on _wstat() / stat() error, -2 if an exception was
+   raised. */
 
 int
 _Py_stat(PyObject *path, struct stat *statbuf)
@@ -253,7 +253,7 @@
 
     wpath = PyUnicode_AsUnicode(path);
     if (wpath == NULL)
-        return -1;
+        return -2;
     err = _wstat(wpath, &wstatbuf);
     if (!err)
         statbuf->st_mode = wstatbuf.st_mode;
@@ -262,7 +262,7 @@
     int ret;
     PyObject *bytes = PyUnicode_EncodeFSDefault(path);
     if (bytes == NULL)
-        return -1;
+        return -2;
     ret = stat(PyBytes_AS_STRING(bytes), statbuf);
     Py_DECREF(bytes);
     return ret;
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1527,6 +1527,7 @@
     Py_UCS4 *fileuni;
     PyObject *py;
     struct stat statbuf;
+    int err;
 
     len = PyUnicode_GET_LENGTH(filename);
     if (len == 0)
@@ -1554,7 +1555,10 @@
     if (py == NULL)
         goto error;
 
-    if (_Py_stat(py, &statbuf) == 0 && S_ISREG(statbuf.st_mode)) {
+    err = _Py_stat(py, &statbuf);
+    if (err == -2)
+        goto error;
+    if (err == 0 && S_ISREG(statbuf.st_mode)) {
         PyMem_Free(fileuni);
         return py;
     }
@@ -1760,7 +1764,7 @@
     Py_ssize_t len, pos;
     struct stat statbuf;
     static struct filedescr fd_package = {"", "", PKG_DIRECTORY};
-    int result, addsep;
+    int err, result, addsep;
 
     if (PyUnicode_Check(path)) {
         Py_INCREF(path);
@@ -1844,7 +1848,12 @@
     /* Check for package import (buf holds a directory name,
        and there's an __init__ module in that directory */
 #ifdef HAVE_STAT
-    if (_Py_stat(filename, &statbuf) == 0 &&         /* it exists */
+    err = _Py_stat(filename, &statbuf);
+    if (err == -2) {
+        result = -1;
+        goto out;
+    }
+    if (err == 0 &&         /* it exists */
         S_ISDIR(statbuf.st_mode))           /* it's a directory */
     {
         int match;
@@ -1905,6 +1914,7 @@
     FILE *fp = NULL;
     PyObject *prefix, *filename;
     int match;
+    int err;
 
     npath = PyList_Size(search_path_list);
     for (i = 0; i < npath; i++) {
@@ -1944,8 +1954,13 @@
             if (Py_VerboseFlag > 1)
                 PySys_FormatStderr("# trying %R\n", filename);
 
-            if (_Py_stat(filename, &statbuf) != 0 || S_ISDIR(statbuf.st_mode))
-            {
+            err = _Py_stat(filename, &statbuf);
+            if (err == -2) {
+                Py_DECREF(prefix);
+                Py_DECREF(filename);
+                return NULL;
+            }
+            if (err != 0 || S_ISDIR(statbuf.st_mode)) {
                 /* it doesn't exist, or it's a directory */
                 Py_DECREF(filename);
                 continue;
@@ -2345,11 +2360,15 @@
     struct stat statbuf;
     PyObject *filename;
     int match;
+    int err;
 
     filename = PyUnicode_FromFormat("%U%c__init__.py", directory, SEP);
     if (filename == NULL)
         return -1;
-    if (_Py_stat(filename, &statbuf) == 0) {
+    err = _Py_stat(filename, &statbuf);
+    if (err == -2)
+        return -1;
+    if (err == 0) {
         /* 3=len(".py") */
         match = case_ok(filename, -3, initstr);
         if (match < 0) {
@@ -2367,7 +2386,12 @@
         directory, SEP, Py_OptimizeFlag ? 'o' : 'c');
     if (filename == NULL)
         return -1;
-    if (_Py_stat(filename, &statbuf) == 0) {
+    err = _Py_stat(filename, &statbuf);
+    if (err == -2) {
+        Py_DECREF(filename);
+        return -1;
+    }
+    if (err == 0) {
         /* 4=len(".pyc") */
         match = case_ok(filename, -4, initstr);
         if (match < 0) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 21:04:17 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 21:04:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Handle_correctl?=
 =?utf8?q?y_=5FPy=5Ffopen=28=29_error=3A_don=27t_replace_the_exception?=
Message-ID: <E1RcMy1-00027E-Ar@dinsdale.python.org>

http://hg.python.org/cpython/rev/586bcc05f88f
changeset:   74059:586bcc05f88f
branch:      3.2
parent:      74056:e37c71698409
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 21:04:17 2011 +0100
summary:
  Handle correctly _Py_fopen() error: don't replace the exception

files:
  Modules/zipimport.c |  8 +++++---
  1 files changed, 5 insertions(+), 3 deletions(-)


diff --git a/Modules/zipimport.c b/Modules/zipimport.c
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -736,7 +736,8 @@
 
     fp = _Py_fopen(archive_obj, "rb");
     if (fp == NULL) {
-        PyErr_Format(ZipImportError, "can't open Zip file: '%U'", archive_obj);
+        if (!PyErr_Occurred())
+            PyErr_Format(ZipImportError, "can't open Zip file: '%U'", archive_obj);
         return NULL;
     }
     fseek(fp, -22, SEEK_END);
@@ -909,8 +910,9 @@
 
     fp = _Py_fopen(archive, "rb");
     if (!fp) {
-        PyErr_Format(PyExc_IOError,
-           "zipimport: can not open file %U", archive);
+        if (!PyErr_Occurred())
+            PyErr_Format(PyExc_IOError,
+               "zipimport: can not open file %U", archive);
         return NULL;
     }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 21:04:48 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 21:04:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=28Merge_3=2E2=29_Handle_correctly_=5FPy=5Ffopen=28=29_error?=
 =?utf8?q?=3A_don=27t_replace_the_exception?=
Message-ID: <E1RcMyW-00028w-7s@dinsdale.python.org>

http://hg.python.org/cpython/rev/f44ab94cd28b
changeset:   74060:f44ab94cd28b
parent:      74058:2a140b01d343
parent:      74059:586bcc05f88f
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 21:05:22 2011 +0100
summary:
  (Merge 3.2) Handle correctly _Py_fopen() error: don't replace the exception

files:
  Modules/zipimport.c |   8 +++++---
  Python/import.c     |  30 +++++++++++++++++++++---------
  2 files changed, 26 insertions(+), 12 deletions(-)


diff --git a/Modules/zipimport.c b/Modules/zipimport.c
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -742,7 +742,8 @@
 
     fp = _Py_fopen(archive, "rb");
     if (fp == NULL) {
-        PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
+        if (!PyErr_Occurred())
+            PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
         return NULL;
     }
     fseek(fp, -22, SEEK_END);
@@ -913,8 +914,9 @@
 
     fp = _Py_fopen(archive, "rb");
     if (!fp) {
-        PyErr_Format(PyExc_IOError,
-           "zipimport: can not open file %U", archive);
+        if (!PyErr_Occurred())
+            PyErr_Format(PyExc_IOError,
+               "zipimport: can not open file %U", archive);
         return NULL;
     }
 
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -3760,26 +3760,38 @@
         mode = "r" PY_STDIOTEXTMODE;
     if (fob == NULL) {
         fp = _Py_fopen(pathname, mode);
+        if (!fp) {
+            if (!PyErr_Occurred())
+                PyErr_SetFromErrno(PyExc_IOError);
+            return NULL;
+        }
+        return fp;
     }
     else {
         int fd = PyObject_AsFileDescriptor(fob);
         if (fd == -1)
             return NULL;
-        if (!_PyVerify_fd(fd))
-            goto error;
+        if (!_PyVerify_fd(fd)) {
+            PyErr_SetFromErrno(PyExc_IOError);
+            return NULL;
+        }
+
         /* the FILE struct gets a new fd, so that it can be closed
          * independently of the file descriptor given
          */
         fd = dup(fd);
-        if (fd == -1)
-            goto error;
+        if (fd == -1) {
+            PyErr_SetFromErrno(PyExc_IOError);
+            return NULL;
+        }
+
         fp = fdopen(fd, mode);
+        if (!fp) {
+            PyErr_SetFromErrno(PyExc_IOError);
+            return NULL;
+        }
+        return fp;
     }
-    if (fp)
-        return fp;
-error:
-    PyErr_SetFromErrno(PyExc_IOError);
-    return NULL;
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 21:36:57 2011
From: python-checkins at python.org (victor.stinner)
Date: Sun, 18 Dec 2011 21:36:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_iobench=2Epy=3A_add_more_in?=
	=?utf8?q?fo_in_the_header?=
Message-ID: <E1RcNTd-0004ai-T0@dinsdale.python.org>

http://hg.python.org/cpython/rev/03701ac95d25
changeset:   74061:03701ac95d25
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Sun Dec 18 21:38:51 2011 +0100
summary:
  iobench.py: add more info in the header

Write the Python version, Unicode implementation and the platform.

files:
  Tools/iobench/iobench.py |  19 +++++++++++++++----
  1 files changed, 15 insertions(+), 4 deletions(-)


diff --git a/Tools/iobench/iobench.py b/Tools/iobench/iobench.py
--- a/Tools/iobench/iobench.py
+++ b/Tools/iobench/iobench.py
@@ -1,13 +1,14 @@
 # -*- coding: utf-8 -*-
 # This file should be kept compatible with both Python 2.6 and Python >= 3.0.
 
-import time
+import functools
+import hashlib
+import itertools
 import os
+import platform
 import re
 import sys
-import hashlib
-import functools
-import itertools
+import time
 from optparse import OptionParser
 
 out = sys.stdout
@@ -307,6 +308,16 @@
         "large": 2,
     }
 
+    print("Python %s" % sys.version)
+    if sys.version_info < (3, 3):
+        if sys.maxunicode > 0xffff:
+            text = "UCS-4 (wide build)"
+        else:
+            text = "UTF-16 (narrow build)"
+    else:
+        text = "PEP 393"
+    print("Unicode: %s" % text)
+    print(platform.platform())
     binary_files = list(get_binary_files())
     text_files = list(get_text_files())
     if "b" in options:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 23:07:16 2011
From: python-checkins at python.org (michael.foord)
Date: Sun, 18 Dec 2011 23:07:16 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_inspect=2Eg?=
 =?utf8?q?etattr=5Fstatic_to_work_on_modules_=28again=29=2E?=
Message-ID: <E1RcOt2-0003cj-Jb@dinsdale.python.org>

http://hg.python.org/cpython/rev/56731ccf2e86
changeset:   74062:56731ccf2e86
branch:      3.2
parent:      74059:586bcc05f88f
user:        Michael Foord <michael at voidspace.org.uk>
date:        Sun Dec 18 22:01:40 2011 +0000
summary:
  Fix inspect.getattr_static to work on modules (again).
Closes issue 11813.

files:
  Lib/inspect.py           |  10 ++++++----
  Lib/test/test_inspect.py |   5 +++++
  Misc/NEWS                |   3 +++
  3 files changed, 14 insertions(+), 4 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -1084,7 +1084,7 @@
 
 def _check_class(klass, attr):
     for entry in _static_getmro(klass):
-        if not _shadowed_dict(type(entry)):
+        if _shadowed_dict(type(entry)) is _sentinel:
             try:
                 return entry.__dict__[attr]
             except KeyError:
@@ -1109,8 +1109,8 @@
             if not (type(class_dict) is types.GetSetDescriptorType and
                     class_dict.__name__ == "__dict__" and
                     class_dict.__objclass__ is entry):
-                return True
-    return False
+                return class_dict
+    return _sentinel
 
 def getattr_static(obj, attr, default=_sentinel):
     """Retrieve attributes without triggering dynamic lookup via the
@@ -1126,7 +1126,9 @@
     instance_result = _sentinel
     if not _is_type(obj):
         klass = type(obj)
-        if not _shadowed_dict(klass):
+        dict_attr = _shadowed_dict(klass)
+        if (dict_attr is _sentinel or
+            type(dict_attr) is types.MemberDescriptorType):
             instance_result = _check_instance(obj, attr)
     else:
         klass = obj
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -1004,6 +1004,11 @@
         self.assertEqual(inspect.getattr_static(instance, "spam"), 42)
         self.assertFalse(Thing.executed)
 
+    def test_module(self):
+        sentinel = object()
+        self.assertIsNot(inspect.getattr_static(sys, "version", sentinel),
+                         sentinel)
+
 class TestGetGeneratorState(unittest.TestCase):
 
     def setUp(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas 
+  St?hrk.
+
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
   C?dric Krier.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 18 23:09:33 2011
From: python-checkins at python.org (michael.foord)
Date: Sun, 18 Dec 2011 23:09:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_3=2E2?=
Message-ID: <E1RcOvF-0003gr-UB@dinsdale.python.org>

http://hg.python.org/cpython/rev/766136049b44
changeset:   74063:766136049b44
parent:      74061:03701ac95d25
parent:      74062:56731ccf2e86
user:        Michael Foord <michael at voidspace.org.uk>
date:        Sun Dec 18 22:09:27 2011 +0000
summary:
  Merge 3.2

files:
  Lib/inspect.py           |  10 ++++++----
  Lib/test/test_inspect.py |   5 +++++
  Misc/NEWS                |   3 +++
  3 files changed, 14 insertions(+), 4 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -1109,7 +1109,7 @@
 
 def _check_class(klass, attr):
     for entry in _static_getmro(klass):
-        if not _shadowed_dict(type(entry)):
+        if _shadowed_dict(type(entry)) is _sentinel:
             try:
                 return entry.__dict__[attr]
             except KeyError:
@@ -1134,8 +1134,8 @@
             if not (type(class_dict) is types.GetSetDescriptorType and
                     class_dict.__name__ == "__dict__" and
                     class_dict.__objclass__ is entry):
-                return True
-    return False
+                return class_dict
+    return _sentinel
 
 def getattr_static(obj, attr, default=_sentinel):
     """Retrieve attributes without triggering dynamic lookup via the
@@ -1151,7 +1151,9 @@
     instance_result = _sentinel
     if not _is_type(obj):
         klass = type(obj)
-        if not _shadowed_dict(klass):
+        dict_attr = _shadowed_dict(klass)
+        if (dict_attr is _sentinel or
+            type(dict_attr) is types.MemberDescriptorType):
             instance_result = _check_instance(obj, attr)
     else:
         klass = obj
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -1004,6 +1004,11 @@
         self.assertEqual(inspect.getattr_static(instance, "spam"), 42)
         self.assertFalse(Thing.executed)
 
+    def test_module(self):
+        sentinel = object()
+        self.assertIsNot(inspect.getattr_static(sys, "version", sentinel),
+                         sentinel)
+
 class TestGetGeneratorState(unittest.TestCase):
 
     def setUp(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas 
+  St?hrk.
+
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
   C?dric Krier.
 

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Dec 19 05:31:03 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 19 Dec 2011 05:31:03 +0100
Subject: [Python-checkins] Daily reference leaks (766136049b44): sum=0
Message-ID: <E1RcUsR-0003qC-Sa@ap.vmr.nerim.net>

results for 766136049b44 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog2Krztl', '-x']

From python-checkins at python.org  Mon Dec 19 06:07:19 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:07:19 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzMzg3OiByZXBo?=
	=?utf8?q?rase_unclear_sentence=2E?=
Message-ID: <E1RcVRX-00009k-PA@dinsdale.python.org>

http://hg.python.org/cpython/rev/88aacd3541ae
changeset:   74064:88aacd3541ae
branch:      2.7
parent:      74055:6a95820b9607
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:04:48 2011 +0200
summary:
  #13387: rephrase unclear sentence.

files:
  Doc/library/unittest.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -895,8 +895,7 @@
 
       Test that *obj* is (or is not) an instance of *cls* (which can be a
       class or a tuple of classes, as supported by :func:`isinstance`).
-      To check for a specific type (without including superclasses) use
-      :func:`assertIs(type(obj), cls) <assertIs>`.
+      To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
 
       .. versionadded:: 2.7
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:07:36 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:07:36 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzMzg3OiByZXBo?=
	=?utf8?q?rase_unclear_sentence=2E?=
Message-ID: <E1RcVRo-0000B1-3l@dinsdale.python.org>

http://hg.python.org/cpython/rev/eccb4795767b
changeset:   74065:eccb4795767b
branch:      3.2
parent:      74062:56731ccf2e86
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:04:48 2011 +0200
summary:
  #13387: rephrase unclear sentence.

files:
  Doc/library/unittest.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -929,8 +929,7 @@
 
       Test that *obj* is (or is not) an instance of *cls* (which can be a
       class or a tuple of classes, as supported by :func:`isinstance`).
-      To check for a specific type (without including superclasses) use
-      :func:`assertIs(type(obj), cls) <assertIs>`.
+      To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
 
       .. versionadded:: 3.2
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:07:41 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:07:41 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313387=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RcVRt-0000Bl-9B@dinsdale.python.org>

http://hg.python.org/cpython/rev/064854cef999
changeset:   74066:064854cef999
parent:      74063:766136049b44
parent:      74065:eccb4795767b
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:07:00 2011 +0200
summary:
  #13387: merge with 3.2.

files:
  Doc/library/unittest.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -933,8 +933,7 @@
 
       Test that *obj* is (or is not) an instance of *cls* (which can be a
       class or a tuple of classes, as supported by :func:`isinstance`).
-      To check for a specific type (without including superclasses) use
-      :func:`assertIs(type(obj), cls) <assertIs>`.
+      To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
 
       .. versionadded:: 3.2
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:17:28 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:17:28 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzM5MzI6IHN1Z2dl?=
 =?utf8?q?st_passing_unicode_to_HTMLParser=2Efeed=28=29=2E?=
Message-ID: <E1RcVbM-0001FI-EP@dinsdale.python.org>

http://hg.python.org/cpython/rev/978f45013c34
changeset:   74067:978f45013c34
branch:      2.7
parent:      74064:88aacd3541ae
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:15:26 2011 +0200
summary:
  #3932: suggest passing unicode to HTMLParser.feed().

files:
  Doc/library/htmlparser.rst |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Doc/library/htmlparser.rst b/Doc/library/htmlparser.rst
--- a/Doc/library/htmlparser.rst
+++ b/Doc/library/htmlparser.rst
@@ -64,7 +64,8 @@
 
    Feed some text to the parser.  It is processed insofar as it consists of
    complete elements; incomplete data is buffered until more data is fed or
-   :meth:`close` is called.
+   :meth:`close` is called.  *data* can be either :class:`unicode` or
+   :class:`str`, but passing :class:`unicode` is advised.
 
 
 .. method:: HTMLParser.close()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:17:59 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:17:59 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogcy86Yzp0eXBlOi86?=
	=?utf8?q?ctype=3A/g?=
Message-ID: <E1RcVbr-0001I1-7U@dinsdale.python.org>

http://hg.python.org/cpython/rev/443d96cb48b5
changeset:   74068:443d96cb48b5
branch:      2.7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:17:08 2011 +0200
summary:
  s/:c:type:/:ctype:/g

files:
  Doc/c-api/unicode.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -328,8 +328,8 @@
 .. cfunction:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
 
    Return a read-only pointer to the Unicode object's internal
-   :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
-   Note that the resulting :c:type:`Py_UNICODE*` string may contain embedded
+   :ctype:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object.
+   Note that the resulting :ctype:`Py_UNICODE*` string may contain embedded
    null characters, which would cause the string to be truncated when used in
    most C functions.
 
@@ -394,7 +394,7 @@
    copied or -1 in case of an error.  Note that the resulting :ctype:`wchar_t`
    string may or may not be 0-terminated.  It is the responsibility of the caller
    to make sure that the :ctype:`wchar_t` string is 0-terminated in case this is
-   required by the application. Also, note that the :c:type:`wchar_t*` string
+   required by the application. Also, note that the :ctype:`wchar_t*` string
    might contain null characters, which would cause the string to be truncated
    when used with most C functions.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:36:12 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:36:12 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogIzEzNTc2OiBhZGQg?=
 =?utf8?q?tests_about_the_handling_of_=28possibly_broken=29_condcoms=2E?=
Message-ID: <E1RcVtU-0003WS-QZ@dinsdale.python.org>

http://hg.python.org/cpython/rev/9c60fd12664f
changeset:   74069:9c60fd12664f
branch:      2.7
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:28:08 2011 +0200
summary:
  #13576: add tests about the handling of (possibly broken) condcoms.

files:
  Lib/markupbase.py           |   4 ++
  Lib/test/test_htmlparser.py |  42 +++++++++++++++++++++++++
  2 files changed, 46 insertions(+), 0 deletions(-)


diff --git a/Lib/markupbase.py b/Lib/markupbase.py
--- a/Lib/markupbase.py
+++ b/Lib/markupbase.py
@@ -108,6 +108,10 @@
                 if decltype == "doctype":
                     self.handle_decl(data)
                 else:
+                    # According to the HTML5 specs sections "8.2.4.44 Bogus
+                    # comment state" and "8.2.4.45 Markup declaration open
+                    # state", a comment token should be emitted.
+                    # Calling unknown_decl provides more flexibility though.
                     self.unknown_decl(data)
                 return j + 1
             if c in "\"'":
diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py
--- a/Lib/test/test_htmlparser.py
+++ b/Lib/test/test_htmlparser.py
@@ -449,6 +449,48 @@
                           [("href", "http://www.example.org/\">;")]),
                          ("data", "spam"), ("endtag", "a")])
 
+    def test_condcoms(self):
+        html = ('<!--[if IE & !(lte IE 8)]>aren\'t<![endif]-->'
+                '<!--[if IE 8]>condcoms<![endif]-->'
+                '<!--[if lte IE 7]>pretty?<![endif]-->')
+        expected = [('comment', "[if IE & !(lte IE 8)]>aren't<![endif]"),
+                    ('comment', '[if IE 8]>condcoms<![endif]'),
+                    ('comment', '[if lte IE 7]>pretty?<![endif]')]
+        self._run_check(html, expected)
+
+    def test_broken_condcoms(self):
+        # these condcoms are missing the '--' after '<!' and before the '>'
+        html = ('<![if !(IE)]>broken condcom<![endif]>'
+                '<![if ! IE]><link href="favicon.tiff"/><![endif]>'
+                '<![if !IE 6]><img src="firefox.png" /><![endif]>'
+                '<![if !ie 6]><b>foo</b><![endif]>'
+                '<![if (!IE)|(lt IE 9)]><img src="mammoth.bmp" /><![endif]>')
+        # According to the HTML5 specs sections "8.2.4.44 Bogus comment state"
+        # and "8.2.4.45 Markup declaration open state", comment tokens should
+        # be emitted instead of 'unknown decl', but calling unknown_decl
+        # provides more flexibility.
+        # See also Lib/_markupbase.py:parse_declaration
+        expected = [
+            ('unknown decl', 'if !(IE)'),
+            ('data', 'broken condcom'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if ! IE'),
+            ('startendtag', 'link', [('href', 'favicon.tiff')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !IE 6'),
+            ('startendtag', 'img', [('src', 'firefox.png')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !ie 6'),
+            ('starttag', 'b', []),
+            ('data', 'foo'),
+            ('endtag', 'b'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if (!IE)|(lt IE 9)'),
+            ('startendtag', 'img', [('src', 'mammoth.bmp')]),
+            ('unknown decl', 'endif')
+        ]
+        self._run_check(html, expected)
+
 
 def test_main():
     test_support.run_unittest(HTMLParserTestCase, AttributesTestCase)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:36:43 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:36:43 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogIzEzNTc2OiBhZGQg?=
 =?utf8?q?tests_about_the_handling_of_=28possibly_broken=29_condcoms=2E?=
Message-ID: <E1RcVtz-0003kz-UL@dinsdale.python.org>

http://hg.python.org/cpython/rev/4ddbb756b602
changeset:   74070:4ddbb756b602
branch:      3.2
parent:      74065:eccb4795767b
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:29:03 2011 +0200
summary:
  #13576: add tests about the handling of (possibly broken) condcoms.

files:
  Lib/_markupbase.py          |   4 ++
  Lib/test/test_htmlparser.py |  43 +++++++++++++++++++++++++
  2 files changed, 47 insertions(+), 0 deletions(-)


diff --git a/Lib/_markupbase.py b/Lib/_markupbase.py
--- a/Lib/_markupbase.py
+++ b/Lib/_markupbase.py
@@ -107,6 +107,10 @@
                 if decltype == "doctype":
                     self.handle_decl(data)
                 else:
+                    # According to the HTML5 specs sections "8.2.4.44 Bogus
+                    # comment state" and "8.2.4.45 Markup declaration open
+                    # state", a comment token should be emitted.
+                    # Calling unknown_decl provides more flexibility though.
                     self.unknown_decl(data)
                 return j + 1
             if c in "\"'":
diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py
--- a/Lib/test/test_htmlparser.py
+++ b/Lib/test/test_htmlparser.py
@@ -323,6 +323,16 @@
                                 ("endtag", element_lower)],
                             collector=Collector())
 
+    def test_condcoms(self):
+        html = ('<!--[if IE & !(lte IE 8)]>aren\'t<![endif]-->'
+                '<!--[if IE 8]>condcoms<![endif]-->'
+                '<!--[if lte IE 7]>pretty?<![endif]-->')
+        expected = [('comment', "[if IE & !(lte IE 8)]>aren't<![endif]"),
+                    ('comment', '[if IE 8]>condcoms<![endif]'),
+                    ('comment', '[if lte IE 7]>pretty?<![endif]')]
+        self._run_check(html, expected)
+
+
 class HTMLParserTolerantTestCase(HTMLParserStrictTestCase):
 
     def get_collector(self):
@@ -416,6 +426,39 @@
         # see #12888
         self.assertEqual(p.unescape('&#123; ' * 1050), '{ ' * 1050)
 
+    def test_broken_condcoms(self):
+        # these condcoms are missing the '--' after '<!' and before the '>'
+        html = ('<![if !(IE)]>broken condcom<![endif]>'
+                '<![if ! IE]><link href="favicon.tiff"/><![endif]>'
+                '<![if !IE 6]><img src="firefox.png" /><![endif]>'
+                '<![if !ie 6]><b>foo</b><![endif]>'
+                '<![if (!IE)|(lt IE 9)]><img src="mammoth.bmp" /><![endif]>')
+        # According to the HTML5 specs sections "8.2.4.44 Bogus comment state"
+        # and "8.2.4.45 Markup declaration open state", comment tokens should
+        # be emitted instead of 'unknown decl', but calling unknown_decl
+        # provides more flexibility.
+        # See also Lib/_markupbase.py:parse_declaration
+        expected = [
+            ('unknown decl', 'if !(IE)'),
+            ('data', 'broken condcom'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if ! IE'),
+            ('startendtag', 'link', [('href', 'favicon.tiff')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !IE 6'),
+            ('startendtag', 'img', [('src', 'firefox.png')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !ie 6'),
+            ('starttag', 'b', []),
+            ('data', 'foo'),
+            ('endtag', 'b'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if (!IE)|(lt IE 9)'),
+            ('startendtag', 'img', [('src', 'mammoth.bmp')]),
+            ('unknown decl', 'endif')
+        ]
+        self._run_check(html, expected)
+
 
 class AttributesStrictTestCase(TestCaseBase):
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 06:37:15 2011
From: python-checkins at python.org (ezio.melotti)
Date: Mon, 19 Dec 2011 06:37:15 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=2313576=3A_merge_with_3=2E2=2E?=
Message-ID: <E1RcVuV-0003rP-1l@dinsdale.python.org>

http://hg.python.org/cpython/rev/6452edbc5f12
changeset:   74071:6452edbc5f12
parent:      74066:064854cef999
parent:      74070:4ddbb756b602
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Mon Dec 19 07:35:21 2011 +0200
summary:
  #13576: merge with 3.2.

files:
  Lib/_markupbase.py          |   4 ++
  Lib/test/test_htmlparser.py |  43 +++++++++++++++++++++++++
  2 files changed, 47 insertions(+), 0 deletions(-)


diff --git a/Lib/_markupbase.py b/Lib/_markupbase.py
--- a/Lib/_markupbase.py
+++ b/Lib/_markupbase.py
@@ -107,6 +107,10 @@
                 if decltype == "doctype":
                     self.handle_decl(data)
                 else:
+                    # According to the HTML5 specs sections "8.2.4.44 Bogus
+                    # comment state" and "8.2.4.45 Markup declaration open
+                    # state", a comment token should be emitted.
+                    # Calling unknown_decl provides more flexibility though.
                     self.unknown_decl(data)
                 return j + 1
             if c in "\"'":
diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py
--- a/Lib/test/test_htmlparser.py
+++ b/Lib/test/test_htmlparser.py
@@ -323,6 +323,16 @@
                                 ("endtag", element_lower)],
                             collector=Collector())
 
+    def test_condcoms(self):
+        html = ('<!--[if IE & !(lte IE 8)]>aren\'t<![endif]-->'
+                '<!--[if IE 8]>condcoms<![endif]-->'
+                '<!--[if lte IE 7]>pretty?<![endif]-->')
+        expected = [('comment', "[if IE & !(lte IE 8)]>aren't<![endif]"),
+                    ('comment', '[if IE 8]>condcoms<![endif]'),
+                    ('comment', '[if lte IE 7]>pretty?<![endif]')]
+        self._run_check(html, expected)
+
+
 class HTMLParserTolerantTestCase(HTMLParserStrictTestCase):
 
     def get_collector(self):
@@ -416,6 +426,39 @@
         # see #12888
         self.assertEqual(p.unescape('&#123; ' * 1050), '{ ' * 1050)
 
+    def test_broken_condcoms(self):
+        # these condcoms are missing the '--' after '<!' and before the '>'
+        html = ('<![if !(IE)]>broken condcom<![endif]>'
+                '<![if ! IE]><link href="favicon.tiff"/><![endif]>'
+                '<![if !IE 6]><img src="firefox.png" /><![endif]>'
+                '<![if !ie 6]><b>foo</b><![endif]>'
+                '<![if (!IE)|(lt IE 9)]><img src="mammoth.bmp" /><![endif]>')
+        # According to the HTML5 specs sections "8.2.4.44 Bogus comment state"
+        # and "8.2.4.45 Markup declaration open state", comment tokens should
+        # be emitted instead of 'unknown decl', but calling unknown_decl
+        # provides more flexibility.
+        # See also Lib/_markupbase.py:parse_declaration
+        expected = [
+            ('unknown decl', 'if !(IE)'),
+            ('data', 'broken condcom'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if ! IE'),
+            ('startendtag', 'link', [('href', 'favicon.tiff')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !IE 6'),
+            ('startendtag', 'img', [('src', 'firefox.png')]),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if !ie 6'),
+            ('starttag', 'b', []),
+            ('data', 'foo'),
+            ('endtag', 'b'),
+            ('unknown decl', 'endif'),
+            ('unknown decl', 'if (!IE)|(lt IE 9)'),
+            ('startendtag', 'img', [('src', 'mammoth.bmp')]),
+            ('unknown decl', 'endif')
+        ]
+        self._run_check(html, expected)
+
 
 class AttributesStrictTestCase(TestCaseBase):
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 12:20:23 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 12:20:23 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzExODY3?=
 =?utf8?q?=3A_Make_test=5Fmailbox=2Etest=5Flock=5Fconflict_deterministic_?=
 =?utf8?q?=28and_fix_a?=
Message-ID: <E1RcbGZ-0007wD-OG@dinsdale.python.org>

http://hg.python.org/cpython/rev/c6d41dd60d2d
changeset:   74072:c6d41dd60d2d
branch:      2.7
parent:      74069:9c60fd12664f
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 12:17:16 2011 +0100
summary:
  Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
race condition).

files:
  Lib/test/test_mailbox.py |  38 +++++++++++++++++++--------
  1 files changed, 26 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -15,6 +15,10 @@
     import fcntl
 except ImportError:
     pass
+try:
+    import multiprocessing
+except ImportError:
+    multiprocessing = None
 
 # Silence Py3k warning
 rfc822 = test_support.import_module('rfc822', deprecated=True)
@@ -865,26 +869,36 @@
             self.assertEqual(contents, f.read())
         self._box = self._factory(self._path)
 
+    @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
+    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
     def test_lock_conflict(self):
-        # Fork off a subprocess that will lock the file for 2 seconds,
-        # unlock it, and then exit.
-        if not hasattr(os, 'fork'):
-            return
+        # Fork off a child process that will lock the mailbox temporarily,
+        # unlock it and exit.
+        ready = multiprocessing.Event()
+        done = multiprocessing.Event()
+
         pid = os.fork()
         if pid == 0:
-            # In the child, lock the mailbox.
-            self._box.lock()
-            time.sleep(2)
-            self._box.unlock()
-            os._exit(0)
+            # child
+            try:
+                # lock the mailbox, and signal the parent it can proceed
+                self._box.lock()
+                ready.set()
 
-        # In the parent, sleep a bit to give the child time to acquire
-        # the lock.
-        time.sleep(0.5)
+                # wait until the parent is done, and unlock the mailbox
+                done.wait(5)
+                self._box.unlock()
+            finally:
+                os._exit(0)
+
+        # In the parent, wait until the child signals it locked the mailbox.
+        ready.wait(5)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
+            # Signal the child it can now release the lock and exit.
+            done.set()
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 12:20:24 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 12:20:24 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExODY3?=
 =?utf8?q?=3A_Make_test=5Fmailbox=2Etest=5Flock=5Fconflict_deterministic_?=
 =?utf8?q?=28and_fix_a?=
Message-ID: <E1RcbGa-0007wM-GP@dinsdale.python.org>

http://hg.python.org/cpython/rev/0053b7c68a02
changeset:   74073:0053b7c68a02
branch:      3.2
parent:      74070:4ddbb756b602
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 12:18:55 2011 +0100
summary:
  Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
race condition).

files:
  Lib/test/test_mailbox.py |  30 +++++++++++++++++++--------
  1 files changed, 21 insertions(+), 9 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -17,6 +17,10 @@
     import fcntl
 except ImportError:
     pass
+try:
+    import multiprocessing
+except ImportError:
+    multiprocessing = None
 
 
 class TestBase(unittest.TestCase):
@@ -994,28 +998,36 @@
             self.assertEqual(contents, f.read())
         self._box = self._factory(self._path)
 
+    @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
+    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
     def test_lock_conflict(self):
-        # Fork off a subprocess that will lock the file for 2 seconds,
-        # unlock it, and then exit.
-        if not hasattr(os, 'fork'):
-            return
+        # Fork off a child process that will lock the mailbox temporarily,
+        # unlock it and exit.
+        ready = multiprocessing.Event()
+        done = multiprocessing.Event()
+
         pid = os.fork()
         if pid == 0:
-            # In the child, lock the mailbox.
+            # child
             try:
+                # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                time.sleep(2)
+                ready.set()
+
+                # wait until the parent is done, and unlock the mailbox
+                done.wait(5)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
-        # In the parent, sleep a bit to give the child time to acquire
-        # the lock.
-        time.sleep(0.5)
+        # In the parent, wait until the child signals it locked the mailbox.
+        ready.wait(5)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
+            # Signal the child it can now release the lock and exit.
+            done.set()
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 12:20:25 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 12:20:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2311867=3A_Make_test=5Fmailbox=2Etest=5Flock=5Fconfli?=
 =?utf8?q?ct_deterministic_=28and_fix_a?=
Message-ID: <E1RcbGb-0007wQ-DS@dinsdale.python.org>

http://hg.python.org/cpython/rev/020260ec44f2
changeset:   74074:020260ec44f2
parent:      74071:6452edbc5f12
parent:      74073:0053b7c68a02
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 12:19:52 2011 +0100
summary:
  Issue #11867: Make test_mailbox.test_lock_conflict deterministic (and fix a
race condition).

files:
  Lib/test/test_mailbox.py |  30 +++++++++++++++++++--------
  1 files changed, 21 insertions(+), 9 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -17,6 +17,10 @@
     import fcntl
 except ImportError:
     pass
+try:
+    import multiprocessing
+except ImportError:
+    multiprocessing = None
 
 
 class TestBase(unittest.TestCase):
@@ -993,28 +997,36 @@
             self.assertEqual(contents, f.read())
         self._box = self._factory(self._path)
 
+    @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
+    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
     def test_lock_conflict(self):
-        # Fork off a subprocess that will lock the file for 2 seconds,
-        # unlock it, and then exit.
-        if not hasattr(os, 'fork'):
-            return
+        # Fork off a child process that will lock the mailbox temporarily,
+        # unlock it and exit.
+        ready = multiprocessing.Event()
+        done = multiprocessing.Event()
+
         pid = os.fork()
         if pid == 0:
-            # In the child, lock the mailbox.
+            # child
             try:
+                # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                time.sleep(2)
+                ready.set()
+
+                # wait until the parent is done, and unlock the mailbox
+                done.wait(5)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
-        # In the parent, sleep a bit to give the child time to acquire
-        # the lock.
-        time.sleep(0.5)
+        # In the parent, wait until the child signals it locked the mailbox.
+        ready.wait(5)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
+            # Signal the child it can now release the lock and exit.
+            done.set()
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 13:27:49 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 19 Dec 2011 13:27:49 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313635=3A_Add_ssl?=
 =?utf8?q?=2EOP=5FCIPHER=5FSERVER=5FPREFERENCE=2C_so_that_SSL_servers?=
Message-ID: <E1RccJp-0001kD-L4@dinsdale.python.org>

http://hg.python.org/cpython/rev/c706f76c9ea8
changeset:   74075:c706f76c9ea8
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 19 13:27:11 2011 +0100
summary:
  Issue #13635: Add ssl.OP_CIPHER_SERVER_PREFERENCE, so that SSL servers
choose the cipher based on their own preferences, rather than on the
client's.

files:
  Doc/library/ssl.rst  |  7 +++++++
  Lib/ssl.py           |  5 ++++-
  Lib/test/test_ssl.py |  1 +
  Misc/NEWS            |  4 ++++
  Modules/_ssl.c       |  2 ++
  5 files changed, 18 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -421,6 +421,13 @@
 
    .. versionadded:: 3.2
 
+.. data:: OP_CIPHER_SERVER_PREFERENCE
+
+   Use the server's cipher ordering preference, rather than the client's.
+   This option has no effect on client sockets and SSLv2 server sockets.
+
+   .. versionadded:: 3.3
+
 .. data:: HAS_SNI
 
    Whether the OpenSSL library has built-in support for the *Server Name
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -66,7 +66,10 @@
     SSLSyscallError, SSLEOFError,
     )
 from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
-from _ssl import OP_ALL, OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1
+from _ssl import (
+    OP_ALL, OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1,
+    OP_CIPHER_SERVER_PREFERENCE,
+    )
 from _ssl import RAND_status, RAND_egd, RAND_add, RAND_bytes, RAND_pseudo_bytes
 from _ssl import (
     SSL_ERROR_ZERO_RETURN,
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -98,6 +98,7 @@
         ssl.CERT_NONE
         ssl.CERT_OPTIONAL
         ssl.CERT_REQUIRED
+        ssl.OP_CIPHER_SERVER_PREFERENCE
         self.assertIn(ssl.HAS_SNI, {True, False})
 
     def test_random(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,10 @@
 Library
 -------
 
+- Issue #13635: Add ssl.OP_CIPHER_SERVER_PREFERENCE, so that SSL servers
+  choose the cipher based on their own preferences, rather than on the
+  client's.
+
 - Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas 
   St?hrk.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2450,6 +2450,8 @@
     PyModule_AddIntConstant(m, "OP_NO_SSLv2", SSL_OP_NO_SSLv2);
     PyModule_AddIntConstant(m, "OP_NO_SSLv3", SSL_OP_NO_SSLv3);
     PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1);
+    PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE",
+                            SSL_OP_CIPHER_SERVER_PREFERENCE);
 
 #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
     r = Py_True;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 13:45:17 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 19 Dec 2011 13:45:17 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNjI4?=
 =?utf8?q?=3A_python-gdb=2Epy_is_now_able_to_retrieve_more_frames_in_the_P?=
 =?utf8?q?ython?=
Message-ID: <E1Rccaj-00009Q-Pj@dinsdale.python.org>

http://hg.python.org/cpython/rev/0b03cb97dac0
changeset:   74076:0b03cb97dac0
branch:      3.2
parent:      74073:0053b7c68a02
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 19 13:42:24 2011 +0100
summary:
  Issue #13628: python-gdb.py is now able to retrieve more frames in the Python
traceback if Python is optimized.

 * delay the lookup of the size_t type, it is not available at startup
 * The second argument of the PyFrameObjectPtr constructor is optional, as
   done in other constructors
 * iter_builtins() and iter_globals() methods of PyFrameObjectPtr returns
   an empty tuple instead of None if Python is optimized
 * Fix py-bt and py-bt-full to handle correctly "optimized" frames
 * Frame.get_pyop() tries to get the frame pointer from PyEval_EvalCodeEx()
   if the pointer is optimized out in PyEval_EvalFrameEx()

files:
  Lib/test/test_gdb.py   |  22 ++++++++++++--
  Misc/NEWS              |   8 ++++-
  Tools/gdb/libpython.py |  45 +++++++++++++++++++++--------
  3 files changed, 58 insertions(+), 17 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -529,6 +529,8 @@
                                  re.DOTALL),
                         'Unexpected gdb representation: %r\n%s' % (gdb_output, gdb_output))
 
+ at unittest.skipIf(python_is_optimized(),
+                 "Python was compiled with optimizations")
 class PyListTests(DebuggerTests):
     def assertListing(self, expected, actual):
         self.assertEndsWith(actual, expected)
@@ -571,6 +573,8 @@
 
 class StackNavigationTests(DebuggerTests):
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_pyup_command(self):
         'Verify that the "py-up" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -598,6 +602,8 @@
                             'Unable to find an older python frame\n')
 
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_up_then_down(self):
         'Verify "py-up" followed by "py-down"'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -611,6 +617,8 @@
 $''')
 
 class PyBtTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_bt(self):
         'Verify that the "py-bt" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -628,6 +636,8 @@
     foo\(1, 2, 3\)
 ''')
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_bt_full(self):
         'Verify that the "py-bt-full" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -639,10 +649,12 @@
 #[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\)
     bar\(a, b, c\)
 #[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 12, in <module> \(\)
-foo\(1, 2, 3\)
+    foo\(1, 2, 3\)
 ''')
 
 class PyPrintTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         'Verify that the "py-print" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -657,12 +669,16 @@
         self.assertMultilineMatches(bt,
                                     r".*\nlocal 'c' = 3\nlocal 'b' = 2\nlocal 'a' = 1\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_global(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print __name__'])
         self.assertMultilineMatches(bt,
                                     r".*\nglobal '__name__' = '__main__'\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_builtin(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print len'])
@@ -670,6 +686,8 @@
                                     r".*\nbuiltin 'len' = <built-in method len of module object at remote 0x[0-9a-f]+>\n.*")
 
 class PyLocalsTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-locals'])
@@ -684,8 +702,6 @@
                                     r".*\na = 1\nb = 2\nc = 3\n.*")
 
 def test_main():
-    if python_is_optimized():
-        raise unittest.SkipTest("Python was compiled with optimizations")
     run_unittest(PrettyPrintTests,
                  PyListTests,
                  StackNavigationTests,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,7 +97,7 @@
 Library
 -------
 
-- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas 
+- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas
   St?hrk.
 
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
@@ -314,6 +314,12 @@
 
 - Issue #13326: Clean __pycache__ directories correctly on OpenBSD.
 
+Tools/Demos
+-----------
+
+- Issue #13628: python-gdb.py is now able to retrieve more frames in the Python
+  traceback if Python is optimized.
+
 Tests
 -----
 
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -49,7 +49,6 @@
 _type_char_ptr = gdb.lookup_type('char').pointer() # char*
 _type_unsigned_char_ptr = gdb.lookup_type('unsigned char').pointer() # unsigned char*
 _type_void_ptr = gdb.lookup_type('void').pointer() # void*
-_type_size_t = gdb.lookup_type('size_t')
 
 SIZEOF_VOID_P = _type_void_ptr.sizeof
 
@@ -435,11 +434,15 @@
                                             self.address)
 
 def _PyObject_VAR_SIZE(typeobj, nitems):
+    if _PyObject_VAR_SIZE._type_size_t is None:
+        _PyObject_VAR_SIZE._type_size_t = gdb.lookup_type('size_t')
+
     return ( ( typeobj.field('tp_basicsize') +
                nitems * typeobj.field('tp_itemsize') +
                (SIZEOF_VOID_P - 1)
              ) & ~(SIZEOF_VOID_P - 1)
-           ).cast(_type_size_t)
+           ).cast(_PyObject_VAR_SIZE._type_size_t)
+_PyObject_VAR_SIZE._type_size_t = None
 
 class HeapTypeObjectPtr(PyObjectPtr):
     _typename = 'PyObject'
@@ -806,7 +809,7 @@
 class PyFrameObjectPtr(PyObjectPtr):
     _typename = 'PyFrameObject'
 
-    def __init__(self, gdbval, cast_to):
+    def __init__(self, gdbval, cast_to=None):
         PyObjectPtr.__init__(self, gdbval, cast_to)
 
         if not self.is_optimized_out():
@@ -840,7 +843,7 @@
         the global variables of this frame
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_globals = self.pyop_field('f_globals')
         return pyop_globals.iteritems()
@@ -851,7 +854,7 @@
         the builtin variables
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_builtins = self.pyop_field('f_builtins')
         return pyop_builtins.iteritems()
@@ -938,6 +941,7 @@
     def print_traceback(self):
         if self.is_optimized_out():
             sys.stdout.write('  (frame information optimized out)\n')
+            return
         visited = set()
         sys.stdout.write('  File "%s", line %i, in %s\n'
                   % (self.co_filename.proxyval(visited),
@@ -1403,7 +1407,20 @@
     def get_pyop(self):
         try:
             f = self._gdbframe.read_var('f')
-            return PyFrameObjectPtr.from_pyobject_ptr(f)
+            frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+            if not frame.is_optimized_out():
+                return frame
+            # gdb is unable to get the "f" argument of PyEval_EvalFrameEx()
+            # because it was "optimized out". Try to get "f" from the frame
+            # of the caller, PyEval_EvalCodeEx().
+            orig_frame = frame
+            caller = self._gdbframe.older()
+            if caller:
+                f = caller.read_var('f')
+                frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+                if not frame.is_optimized_out():
+                    return frame
+            return orig_frame
         except ValueError:
             return None
 
@@ -1434,9 +1451,10 @@
             if pyop:
                 line = pyop.get_truncated_repr(MAX_OUTPUT_LEN)
                 write_unicode(sys.stdout, '#%i %s\n' % (self.get_index(), line))
-                line = pyop.current_line()
-                if line is not None:
-                    sys.stdout.write(line)
+                if not pyop.is_optimized_out():
+                    line = pyop.current_line()
+                    if line is not None:
+                        sys.stdout.write('    %s\n' % line.strip())
             else:
                 sys.stdout.write('#%i (unable to read python frame information)\n' % self.get_index())
         else:
@@ -1447,9 +1465,10 @@
             pyop = self.get_pyop()
             if pyop:
                 pyop.print_traceback()
-                line = pyop.current_line()
-                if line is not None:
-                    sys.stdout.write('    %s\n' % line.strip())
+                if not pyop.is_optimized_out():
+                    line = pyop.current_line()
+                    if line is not None:
+                        sys.stdout.write('    %s\n' % line.strip())
             else:
                 sys.stdout.write('  (unable to read python frame information)\n')
         else:
@@ -1495,7 +1514,7 @@
             return
 
         pyop = frame.get_pyop()
-        if not pyop:
+        if not pyop or pyop.is_optimized_out():
             print 'Unable to read information on python frame'
             return
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 13:45:18 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 19 Dec 2011 13:45:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_=28Merge_3=2E2=29_Issue_=2313628=3A_python-gdb=2Epy_is_now_a?=
 =?utf8?q?ble_to_retrieve_more_frames_in?=
Message-ID: <E1Rccak-00009y-Lh@dinsdale.python.org>

http://hg.python.org/cpython/rev/5e3a172bba89
changeset:   74077:5e3a172bba89
parent:      74075:c706f76c9ea8
parent:      74076:0b03cb97dac0
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 19 13:47:10 2011 +0100
summary:
  (Merge 3.2) Issue #13628: python-gdb.py is now able to retrieve more frames in
the Python traceback if Python is optimized.

 * delay the lookup of the size_t type, it is not available at startup
 * The second argument of the PyFrameObjectPtr constructor is optional, as
   done in other constructors
 * iter_builtins() and iter_globals() methods of PyFrameObjectPtr returns
   an empty tuple instead of None if Python is optimized
 * Fix py-bt and py-bt-full to handle correctly "optimized" frames
 * Frame.get_pyop() tries to get the frame pointer from PyEval_EvalCodeEx()
   if the pointer is optimized out in PyEval_EvalFrameEx()

files:
  Lib/test/test_gdb.py   |  22 ++++++++++++--
  Misc/NEWS              |   5 ++-
  Tools/gdb/libpython.py |  45 +++++++++++++++++++++--------
  3 files changed, 55 insertions(+), 17 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -529,6 +529,8 @@
                                  re.DOTALL),
                         'Unexpected gdb representation: %r\n%s' % (gdb_output, gdb_output))
 
+ at unittest.skipIf(python_is_optimized(),
+                 "Python was compiled with optimizations")
 class PyListTests(DebuggerTests):
     def assertListing(self, expected, actual):
         self.assertEndsWith(actual, expected)
@@ -571,6 +573,8 @@
 
 class StackNavigationTests(DebuggerTests):
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_pyup_command(self):
         'Verify that the "py-up" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -598,6 +602,8 @@
                             'Unable to find an older python frame\n')
 
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_up_then_down(self):
         'Verify "py-up" followed by "py-down"'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -611,6 +617,8 @@
 $''')
 
 class PyBtTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_bt(self):
         'Verify that the "py-bt" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -628,6 +636,8 @@
     foo\(1, 2, 3\)
 ''')
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_bt_full(self):
         'Verify that the "py-bt-full" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -639,10 +649,12 @@
 #[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\)
     bar\(a, b, c\)
 #[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 12, in <module> \(\)
-foo\(1, 2, 3\)
+    foo\(1, 2, 3\)
 ''')
 
 class PyPrintTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         'Verify that the "py-print" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -657,12 +669,16 @@
         self.assertMultilineMatches(bt,
                                     r".*\nlocal 'c' = 3\nlocal 'b' = 2\nlocal 'a' = 1\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_global(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print __name__'])
         self.assertMultilineMatches(bt,
                                     r".*\nglobal '__name__' = '__main__'\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_builtin(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print len'])
@@ -670,6 +686,8 @@
                                     r".*\nbuiltin 'len' = <built-in method len of module object at remote 0x-?[0-9a-f]+>\n.*")
 
 class PyLocalsTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-locals'])
@@ -684,8 +702,6 @@
                                     r".*\na = 1\nb = 2\nc = 3\n.*")
 
 def test_main():
-    if python_is_optimized():
-        raise unittest.SkipTest("Python was compiled with optimizations")
     run_unittest(PrettyPrintTests,
                  PyListTests,
                  StackNavigationTests,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -423,7 +423,7 @@
   choose the cipher based on their own preferences, rather than on the
   client's.
 
-- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas 
+- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas
   St?hrk.
 
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
@@ -1692,6 +1692,9 @@
 Tools/Demos
 -----------
 
+- Issue #13628: python-gdb.py is now able to retrieve more frames in the Python
+  traceback if Python is optimized.
+
 - Issue #11996: libpython (gdb), replace "py-bt" command by "py-bt-full" and
   add a smarter "py-bt" command printing a classic Python traceback.
 
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -49,7 +49,6 @@
 _type_char_ptr = gdb.lookup_type('char').pointer() # char*
 _type_unsigned_char_ptr = gdb.lookup_type('unsigned char').pointer() # unsigned char*
 _type_void_ptr = gdb.lookup_type('void').pointer() # void*
-_type_size_t = gdb.lookup_type('size_t')
 _type_unsigned_short_ptr = gdb.lookup_type('unsigned short').pointer()
 _type_unsigned_int_ptr = gdb.lookup_type('unsigned int').pointer()
 
@@ -439,11 +438,15 @@
                                             self.address)
 
 def _PyObject_VAR_SIZE(typeobj, nitems):
+    if _PyObject_VAR_SIZE._type_size_t is None:
+        _PyObject_VAR_SIZE._type_size_t = gdb.lookup_type('size_t')
+
     return ( ( typeobj.field('tp_basicsize') +
                nitems * typeobj.field('tp_itemsize') +
                (SIZEOF_VOID_P - 1)
              ) & ~(SIZEOF_VOID_P - 1)
-           ).cast(_type_size_t)
+           ).cast(_PyObject_VAR_SIZE._type_size_t)
+_PyObject_VAR_SIZE._type_size_t = None
 
 class HeapTypeObjectPtr(PyObjectPtr):
     _typename = 'PyObject'
@@ -772,7 +775,7 @@
 class PyFrameObjectPtr(PyObjectPtr):
     _typename = 'PyFrameObject'
 
-    def __init__(self, gdbval, cast_to):
+    def __init__(self, gdbval, cast_to=None):
         PyObjectPtr.__init__(self, gdbval, cast_to)
 
         if not self.is_optimized_out():
@@ -806,7 +809,7 @@
         the global variables of this frame
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_globals = self.pyop_field('f_globals')
         return pyop_globals.iteritems()
@@ -817,7 +820,7 @@
         the builtin variables
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_builtins = self.pyop_field('f_builtins')
         return pyop_builtins.iteritems()
@@ -904,6 +907,7 @@
     def print_traceback(self):
         if self.is_optimized_out():
             sys.stdout.write('  (frame information optimized out)\n')
+            return
         visited = set()
         sys.stdout.write('  File "%s", line %i, in %s\n'
                   % (self.co_filename.proxyval(visited),
@@ -1400,7 +1404,20 @@
     def get_pyop(self):
         try:
             f = self._gdbframe.read_var('f')
-            return PyFrameObjectPtr.from_pyobject_ptr(f)
+            frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+            if not frame.is_optimized_out():
+                return frame
+            # gdb is unable to get the "f" argument of PyEval_EvalFrameEx()
+            # because it was "optimized out". Try to get "f" from the frame
+            # of the caller, PyEval_EvalCodeEx().
+            orig_frame = frame
+            caller = self._gdbframe.older()
+            if caller:
+                f = caller.read_var('f')
+                frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+                if not frame.is_optimized_out():
+                    return frame
+            return orig_frame
         except ValueError:
             return None
 
@@ -1431,9 +1448,10 @@
             if pyop:
                 line = pyop.get_truncated_repr(MAX_OUTPUT_LEN)
                 write_unicode(sys.stdout, '#%i %s\n' % (self.get_index(), line))
-                line = pyop.current_line()
-                if line is not None:
-                    sys.stdout.write(line)
+                if not pyop.is_optimized_out():
+                    line = pyop.current_line()
+                    if line is not None:
+                        sys.stdout.write('    %s\n' % line.strip())
             else:
                 sys.stdout.write('#%i (unable to read python frame information)\n' % self.get_index())
         else:
@@ -1444,9 +1462,10 @@
             pyop = self.get_pyop()
             if pyop:
                 pyop.print_traceback()
-                line = pyop.current_line()
-                if line is not None:
-                    sys.stdout.write('    %s\n' % line.strip())
+                if not pyop.is_optimized_out():
+                    line = pyop.current_line()
+                    if line is not None:
+                        sys.stdout.write('    %s\n' % line.strip())
             else:
                 sys.stdout.write('  (unable to read python frame information)\n')
         else:
@@ -1492,7 +1511,7 @@
             return
 
         pyop = frame.get_pyop()
-        if not pyop:
+        if not pyop or pyop.is_optimized_out():
             print 'Unable to read information on python frame'
             return
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 13:58:24 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 19 Dec 2011 13:58:24 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNjI4?=
 =?utf8?q?=3A_python-gdb=2Epy_is_now_able_to_retrieve_more_frames_in_the_P?=
 =?utf8?q?ython?=
Message-ID: <E1RccnQ-0001gb-1z@dinsdale.python.org>

http://hg.python.org/cpython/rev/1cc8e9565339
changeset:   74078:1cc8e9565339
branch:      2.7
parent:      74072:c6d41dd60d2d
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 19 13:59:58 2011 +0100
summary:
  Issue #13628: python-gdb.py is now able to retrieve more frames in the Python
traceback if Python is optimized.

 * delay the lookup of the size_t type, it is not available at startup
 * The second argument of the PyFrameObjectPtr constructor is optional, as
   done in other constructors
 * iter_builtins() and iter_globals() methods of PyFrameObjectPtr returns
   an empty tuple instead of None if Python is optimized
 * Fix py-bt to handle correctly "optimized" frames
 * Frame.get_pyop() tries to get the frame pointer from PyEval_EvalCodeEx()
   if the pointer is optimized out in PyEval_EvalFrameEx()

files:
  Lib/test/test_gdb.py   |  39 ++++++++++++++++++++++-------
  Misc/NEWS              |   6 ++++
  Tools/gdb/libpython.py |  34 ++++++++++++++++++++------
  3 files changed, 61 insertions(+), 18 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -32,6 +32,14 @@
 if gdbpy_version == '':
     raise unittest.SkipTest("gdb not built with embedded python support")
 
+def python_is_optimized():
+    cflags = sysconfig.get_config_vars()['PY_CFLAGS']
+    final_opt = ""
+    for opt in cflags.split():
+        if opt.startswith('-O'):
+            final_opt = opt
+    return (final_opt and final_opt != '-O0')
+
 def gdb_has_frame_select():
     # Does this build of gdb have gdb.Frame.select ?
     cmd = "--eval-command=python print(dir(gdb.Frame))"
@@ -543,6 +551,8 @@
                                  re.DOTALL),
                         'Unexpected gdb representation: %r\n%s' % (gdb_output, gdb_output))
 
+ at unittest.skipIf(python_is_optimized(),
+                 "Python was compiled with optimizations")
 class PyListTests(DebuggerTests):
     def assertListing(self, expected, actual):
         self.assertEndsWith(actual, expected)
@@ -585,6 +595,8 @@
 
 class StackNavigationTests(DebuggerTests):
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_pyup_command(self):
         'Verify that the "py-up" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -612,6 +624,8 @@
                             'Unable to find an older python frame\n')
 
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_up_then_down(self):
         'Verify "py-up" followed by "py-down"'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -625,6 +639,8 @@
 $''')
 
 class PyBtTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         'Verify that the "py-bt" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -636,10 +652,12 @@
 #[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\)
     bar\(a, b, c\)
 #[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 12, in <module> \(\)
-foo\(1, 2, 3\)
+    foo\(1, 2, 3\)
 ''')
 
 class PyPrintTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         'Verify that the "py-print" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
@@ -648,18 +666,24 @@
                                     r".*\nlocal 'args' = \(1, 2, 3\)\n.*")
 
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_print_after_up(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-up', 'py-print c', 'py-print b', 'py-print a'])
         self.assertMultilineMatches(bt,
                                     r".*\nlocal 'c' = 3\nlocal 'b' = 2\nlocal 'a' = 1\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_global(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print __name__'])
         self.assertMultilineMatches(bt,
                                     r".*\nglobal '__name__' = '__main__'\n.*")
 
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_printing_builtin(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-print len'])
@@ -667,6 +691,8 @@
                                     r".*\nbuiltin 'len' = <built-in function len>\n.*")
 
 class PyLocalsTests(DebuggerTests):
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_basic_command(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-locals'])
@@ -674,6 +700,8 @@
                                     r".*\nargs = \(1, 2, 3\)\n.*")
 
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
+    @unittest.skipIf(python_is_optimized(),
+                     "Python was compiled with optimizations")
     def test_locals_after_up(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
                                   cmds_after_breakpoint=['py-up', 'py-locals'])
@@ -681,15 +709,6 @@
                                     r".*\na = 1\nb = 2\nc = 3\n.*")
 
 def test_main():
-    cflags = sysconfig.get_config_vars()['PY_CFLAGS']
-    final_opt = ""
-    for opt in cflags.split():
-        if opt.startswith('-O'):
-            final_opt = opt
-    if final_opt and final_opt != '-O0':
-        raise unittest.SkipTest("Python was built with compiler optimizations, "
-                                "tests can't reliably succeed")
-
     run_unittest(PrettyPrintTests,
                  PyListTests,
                  StackNavigationTests,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -404,6 +404,12 @@
 - Issue #10639: reindent.py no longer converts newlines and will raise
   an error if attempting to convert a file with mixed newlines.
 
+Tools/Demos
+-----------
+
+- Issue #13628: python-gdb.py is now able to retrieve more frames in the Python
+  traceback if Python is optimized.
+
 Tests
 -----
 
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -47,7 +47,6 @@
 _type_char_ptr = gdb.lookup_type('char').pointer() # char*
 _type_unsigned_char_ptr = gdb.lookup_type('unsigned char').pointer() # unsigned char*
 _type_void_ptr = gdb.lookup_type('void').pointer() # void*
-_type_size_t = gdb.lookup_type('size_t')
 
 SIZEOF_VOID_P = _type_void_ptr.sizeof
 
@@ -410,11 +409,15 @@
                                             self.address)
 
 def _PyObject_VAR_SIZE(typeobj, nitems):
+    if _PyObject_VAR_SIZE._type_size_t is None:
+        _PyObject_VAR_SIZE._type_size_t = gdb.lookup_type('size_t')
+
     return ( ( typeobj.field('tp_basicsize') +
                nitems * typeobj.field('tp_itemsize') +
                (SIZEOF_VOID_P - 1)
              ) & ~(SIZEOF_VOID_P - 1)
-           ).cast(_type_size_t)
+           ).cast(_PyObject_VAR_SIZE._type_size_t)
+_PyObject_VAR_SIZE._type_size_t = None
 
 class HeapTypeObjectPtr(PyObjectPtr):
     _typename = 'PyObject'
@@ -786,7 +789,7 @@
 class PyFrameObjectPtr(PyObjectPtr):
     _typename = 'PyFrameObject'
 
-    def __init__(self, gdbval, cast_to):
+    def __init__(self, gdbval, cast_to=None):
         PyObjectPtr.__init__(self, gdbval, cast_to)
 
         if not self.is_optimized_out():
@@ -820,7 +823,7 @@
         the global variables of this frame
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_globals = self.pyop_field('f_globals')
         return pyop_globals.iteritems()
@@ -831,7 +834,7 @@
         the builtin variables
         '''
         if self.is_optimized_out():
-            return
+            return ()
 
         pyop_builtins = self.pyop_field('f_builtins')
         return pyop_builtins.iteritems()
@@ -1205,7 +1208,20 @@
     def get_pyop(self):
         try:
             f = self._gdbframe.read_var('f')
-            return PyFrameObjectPtr.from_pyobject_ptr(f)
+            frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+            if not frame.is_optimized_out():
+                return frame
+            # gdb is unable to get the "f" argument of PyEval_EvalFrameEx()
+            # because it was "optimized out". Try to get "f" from the frame
+            # of the caller, PyEval_EvalCodeEx().
+            orig_frame = frame
+            caller = self._gdbframe.older()
+            if caller:
+                f = caller.read_var('f')
+                frame = PyFrameObjectPtr.from_pyobject_ptr(f)
+                if not frame.is_optimized_out():
+                    return frame
+            return orig_frame
         except ValueError:
             return None
 
@@ -1235,7 +1251,9 @@
             pyop = self.get_pyop()
             if pyop:
                 sys.stdout.write('#%i %s\n' % (self.get_index(), pyop.get_truncated_repr(MAX_OUTPUT_LEN)))
-                sys.stdout.write(pyop.current_line())
+                if not pyop.is_optimized_out():
+                    line = pyop.current_line()
+                    sys.stdout.write('    %s\n' % line.strip())
             else:
                 sys.stdout.write('#%i (unable to read python frame information)\n' % self.get_index())
         else:
@@ -1281,7 +1299,7 @@
             return
 
         pyop = frame.get_pyop()
-        if not pyop:
+        if not pyop or pyop.is_optimized_out():
             print 'Unable to read information on python frame'
             return
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 16:14:49 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 16:14:49 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Try_to_increase_some_socket_timeouts_to_make_some_buildbots_?=
 =?utf8?q?stop?=
Message-ID: <E1RcevR-0001NG-2W@dinsdale.python.org>

http://hg.python.org/cpython/rev/2228d985fdcc
changeset:   74079:2228d985fdcc
branch:      2.7
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 16:11:04 2011 +0100
summary:
  Issue #13453: Try to increase some socket timeouts to make some buildbots stop
failing.

files:
  Lib/test/test_ftplib.py    |  2 +-
  Lib/test/test_telnetlib.py |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py
--- a/Lib/test/test_ftplib.py
+++ b/Lib/test/test_ftplib.py
@@ -667,7 +667,7 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(10)
         self.port = test_support.bind_port(self.sock)
         threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
         # Wait for the server to be ready.
diff --git a/Lib/test/test_telnetlib.py b/Lib/test/test_telnetlib.py
--- a/Lib/test/test_telnetlib.py
+++ b/Lib/test/test_telnetlib.py
@@ -95,7 +95,7 @@
     self.evt = threading.Event()
     self.dataq = Queue.Queue()
     self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-    self.sock.settimeout(3)
+    self.sock.settimeout(10)
     self.port = test_support.bind_port(self.sock)
     self.thread = threading.Thread(target=server, args=(self.evt,self.sock, self.dataq))
     self.thread.start()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 16:14:49 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 16:14:49 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNDUz?=
 =?utf8?q?=3A_Try_to_increase_some_socket_timeouts_to_make_some_buildbots_?=
 =?utf8?q?stop?=
Message-ID: <E1RcevR-0001OD-Sb@dinsdale.python.org>

http://hg.python.org/cpython/rev/d7daf98c068e
changeset:   74080:d7daf98c068e
branch:      3.2
parent:      74076:0b03cb97dac0
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 16:12:23 2011 +0100
summary:
  Issue #13453: Try to increase some socket timeouts to make some buildbots stop
failing.

files:
  Lib/test/test_ftplib.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py
--- a/Lib/test/test_ftplib.py
+++ b/Lib/test/test_ftplib.py
@@ -767,7 +767,7 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(10)
         self.port = support.bind_port(self.sock)
         threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
         # Wait for the server to be ready.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 16:14:50 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Mon, 19 Dec 2011 16:14:50 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313453=3A_Try_to_increase_some_socket_timeouts_to_ma?=
 =?utf8?q?ke_some_buildbots_stop?=
Message-ID: <E1RcevS-0001OJ-Lt@dinsdale.python.org>

http://hg.python.org/cpython/rev/307d698c3ece
changeset:   74081:307d698c3ece
parent:      74077:5e3a172bba89
parent:      74080:d7daf98c068e
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Mon Dec 19 16:12:59 2011 +0100
summary:
  Issue #13453: Try to increase some socket timeouts to make some buildbots stop
failing.

files:
  Lib/test/test_ftplib.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py
--- a/Lib/test/test_ftplib.py
+++ b/Lib/test/test_ftplib.py
@@ -894,7 +894,7 @@
     def setUp(self):
         self.evt = threading.Event()
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.sock.settimeout(3)
+        self.sock.settimeout(10)
         self.port = support.bind_port(self.sock)
         threading.Thread(target=self.server, args=(self.evt,self.sock)).start()
         # Wait for the server to be ready.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 16:23:14 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 19 Dec 2011 16:23:14 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzU0MjQ6?=
 =?utf8?q?_add_tests_for_inet=5Fntoa=2C_inet=5Fntop=2C_inet=5Faton_and_ine?=
 =?utf8?q?t=5Fpton=2E?=
Message-ID: <E1Rcf3a-0002Vx-Cw@dinsdale.python.org>

http://hg.python.org/cpython/rev/71e5a083f9b1
changeset:   74082:71e5a083f9b1
branch:      3.2
parent:      74080:d7daf98c068e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 19 16:21:21 2011 +0100
summary:
  Issue #5424: add tests for inet_ntoa, inet_ntop, inet_aton and inet_pton.
Patch by Philipp Hagemeister.

files:
  Lib/test/test_socket.py |  65 +++++++++++++++++++++++++++++
  Misc/ACKS               |   1 +
  2 files changed, 66 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -499,16 +499,30 @@
         from socket import inet_aton as f, inet_pton, AF_INET
         g = lambda a: inet_pton(AF_INET, a)
 
+        assertInvalid = lambda func,a: self.assertRaises(
+            (socket.error, ValueError), func, a
+        )
+
         self.assertEqual(b'\x00\x00\x00\x00', f('0.0.0.0'))
         self.assertEqual(b'\xff\x00\xff\x00', f('255.0.255.0'))
         self.assertEqual(b'\xaa\xaa\xaa\xaa', f('170.170.170.170'))
         self.assertEqual(b'\x01\x02\x03\x04', f('1.2.3.4'))
         self.assertEqual(b'\xff\xff\xff\xff', f('255.255.255.255'))
+        assertInvalid(f, '0.0.0.')
+        assertInvalid(f, '300.0.0.0')
+        assertInvalid(f, 'a.0.0.0')
+        assertInvalid(f, '1.2.3.4.5')
+        assertInvalid(f, '::1')
 
         self.assertEqual(b'\x00\x00\x00\x00', g('0.0.0.0'))
         self.assertEqual(b'\xff\x00\xff\x00', g('255.0.255.0'))
         self.assertEqual(b'\xaa\xaa\xaa\xaa', g('170.170.170.170'))
         self.assertEqual(b'\xff\xff\xff\xff', g('255.255.255.255'))
+        assertInvalid(g, '0.0.0.')
+        assertInvalid(g, '300.0.0.0')
+        assertInvalid(g, 'a.0.0.0')
+        assertInvalid(g, '1.2.3.4.5')
+        assertInvalid(g, '::1')
 
     def testIPv6toString(self):
         if not hasattr(socket, 'inet_pton'):
@@ -520,6 +534,9 @@
         except ImportError:
             return
         f = lambda a: inet_pton(AF_INET6, a)
+        assertInvalid = lambda a: self.assertRaises(
+            (socket.error, ValueError), f, a
+        )
 
         self.assertEqual(b'\x00' * 16, f('::'))
         self.assertEqual(b'\x00' * 16, f('0::0'))
@@ -528,21 +545,62 @@
             b'\x45\xef\x76\xcb\x00\x1a\x56\xef\xaf\xeb\x0b\xac\x19\x24\xae\xae',
             f('45ef:76cb:1a:56ef:afeb:bac:1924:aeae')
         )
+        self.assertEqual(
+            b'\xad\x42\x0a\xbc' + b'\x00' * 4 + b'\x01\x27\x00\x00\x02\x54\x00\x02',
+            f('ad42:abc::127:0:254:2')
+        )
+        self.assertEqual(b'\x00\x12\x00\x0a' + b'\x00' * 12, f('12:a::'))
+        assertInvalid('0x20::')
+        assertInvalid(':::')
+        assertInvalid('::0::')
+        assertInvalid('1::abc::')
+        assertInvalid('1::abc::def')
+        assertInvalid('1:2:3:4:5:6:')
+        assertInvalid('1:2:3:4:5:6')
+        assertInvalid('1:2:3:4:5:6:7:8:')
+        assertInvalid('1:2:3:4:5:6:7:8:0')
+
+        self.assertEqual(b'\x00' * 12 + b'\xfe\x2a\x17\x40',
+            f('::254.42.23.64')
+        )
+        self.assertEqual(
+            b'\x00\x42' + b'\x00' * 8 + b'\xa2\x9b\xfe\x2a\x17\x40',
+            f('42::a29b:254.42.23.64')
+        )
+        self.assertEqual(
+            b'\x00\x42\xa8\xb9\x00\x00\x00\x02\xff\xff\xa2\x9b\xfe\x2a\x17\x40',
+            f('42:a8b9:0:2:ffff:a29b:254.42.23.64')
+        )
+        assertInvalid('255.254.253.252')
+        assertInvalid('1::260.2.3.0')
+        assertInvalid('1::0.be.e.0')
+        assertInvalid('1:2:3:4:5:6:7:1.2.3.4')
+        assertInvalid('::1.2.3.4:0')
+        assertInvalid('0.100.200.0:3:4:5:6:7:8')
 
     def testStringToIPv4(self):
         if not hasattr(socket, 'inet_ntop'):
             return # No inet_ntop() on this platform
         from socket import inet_ntoa as f, inet_ntop, AF_INET
         g = lambda a: inet_ntop(AF_INET, a)
+        assertInvalid = lambda func,a: self.assertRaises(
+            (socket.error, ValueError), func, a
+        )
 
         self.assertEqual('1.0.1.0', f(b'\x01\x00\x01\x00'))
         self.assertEqual('170.85.170.85', f(b'\xaa\x55\xaa\x55'))
         self.assertEqual('255.255.255.255', f(b'\xff\xff\xff\xff'))
         self.assertEqual('1.2.3.4', f(b'\x01\x02\x03\x04'))
+        assertInvalid(f, b'\x00' * 3)
+        assertInvalid(f, b'\x00' * 5)
+        assertInvalid(f, b'\x00' * 16)
 
         self.assertEqual('1.0.1.0', g(b'\x01\x00\x01\x00'))
         self.assertEqual('170.85.170.85', g(b'\xaa\x55\xaa\x55'))
         self.assertEqual('255.255.255.255', g(b'\xff\xff\xff\xff'))
+        assertInvalid(g, b'\x00' * 3)
+        assertInvalid(g, b'\x00' * 5)
+        assertInvalid(g, b'\x00' * 16)
 
     def testStringToIPv6(self):
         if not hasattr(socket, 'inet_ntop'):
@@ -554,6 +612,9 @@
         except ImportError:
             return
         f = lambda a: inet_ntop(AF_INET6, a)
+        assertInvalid = lambda a: self.assertRaises(
+            (socket.error, ValueError), f, a
+        )
 
         self.assertEqual('::', f(b'\x00' * 16))
         self.assertEqual('::1', f(b'\x00' * 15 + b'\x01'))
@@ -562,6 +623,10 @@
             f(b'\x0a\xef\x0b\x01\x05\x06\x10\x01\xff\xff\x99\x97\x00\x55\x01\x70')
         )
 
+        assertInvalid(b'\x12' * 15)
+        assertInvalid(b'\x12' * 17)
+        assertInvalid(b'\x12' * 4)
+
     # XXX The following don't test module-level functionality...
 
     def testSockName(self):
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -353,6 +353,7 @@
 Lars Gust?bel
 Thomas G?ttler
 Barry Haddow
+Philipp Hagemeister
 Paul ten Hagen
 Rasmus Hahn
 Peter Haight

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 16:23:15 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 19 Dec 2011 16:23:15 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=235424=3A_add_tests_for_inet=5Fntoa=2C_inet=5Fntop=2C?=
 =?utf8?q?_inet=5Faton_and_inet=5Fpton=2E?=
Message-ID: <E1Rcf3b-0002WU-5s@dinsdale.python.org>

http://hg.python.org/cpython/rev/a3d5f522065f
changeset:   74083:a3d5f522065f
parent:      74081:307d698c3ece
parent:      74082:71e5a083f9b1
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 19 16:22:26 2011 +0100
summary:
  Issue #5424: add tests for inet_ntoa, inet_ntop, inet_aton and inet_pton.
Patch by Philipp Hagemeister.

files:
  Lib/test/test_socket.py |  65 +++++++++++++++++++++++++++++
  Misc/ACKS               |   1 +
  2 files changed, 66 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -895,16 +895,30 @@
         from socket import inet_aton as f, inet_pton, AF_INET
         g = lambda a: inet_pton(AF_INET, a)
 
+        assertInvalid = lambda func,a: self.assertRaises(
+            (socket.error, ValueError), func, a
+        )
+
         self.assertEqual(b'\x00\x00\x00\x00', f('0.0.0.0'))
         self.assertEqual(b'\xff\x00\xff\x00', f('255.0.255.0'))
         self.assertEqual(b'\xaa\xaa\xaa\xaa', f('170.170.170.170'))
         self.assertEqual(b'\x01\x02\x03\x04', f('1.2.3.4'))
         self.assertEqual(b'\xff\xff\xff\xff', f('255.255.255.255'))
+        assertInvalid(f, '0.0.0.')
+        assertInvalid(f, '300.0.0.0')
+        assertInvalid(f, 'a.0.0.0')
+        assertInvalid(f, '1.2.3.4.5')
+        assertInvalid(f, '::1')
 
         self.assertEqual(b'\x00\x00\x00\x00', g('0.0.0.0'))
         self.assertEqual(b'\xff\x00\xff\x00', g('255.0.255.0'))
         self.assertEqual(b'\xaa\xaa\xaa\xaa', g('170.170.170.170'))
         self.assertEqual(b'\xff\xff\xff\xff', g('255.255.255.255'))
+        assertInvalid(g, '0.0.0.')
+        assertInvalid(g, '300.0.0.0')
+        assertInvalid(g, 'a.0.0.0')
+        assertInvalid(g, '1.2.3.4.5')
+        assertInvalid(g, '::1')
 
     def testIPv6toString(self):
         if not hasattr(socket, 'inet_pton'):
@@ -916,6 +930,9 @@
         except ImportError:
             return
         f = lambda a: inet_pton(AF_INET6, a)
+        assertInvalid = lambda a: self.assertRaises(
+            (socket.error, ValueError), f, a
+        )
 
         self.assertEqual(b'\x00' * 16, f('::'))
         self.assertEqual(b'\x00' * 16, f('0::0'))
@@ -924,21 +941,62 @@
             b'\x45\xef\x76\xcb\x00\x1a\x56\xef\xaf\xeb\x0b\xac\x19\x24\xae\xae',
             f('45ef:76cb:1a:56ef:afeb:bac:1924:aeae')
         )
+        self.assertEqual(
+            b'\xad\x42\x0a\xbc' + b'\x00' * 4 + b'\x01\x27\x00\x00\x02\x54\x00\x02',
+            f('ad42:abc::127:0:254:2')
+        )
+        self.assertEqual(b'\x00\x12\x00\x0a' + b'\x00' * 12, f('12:a::'))
+        assertInvalid('0x20::')
+        assertInvalid(':::')
+        assertInvalid('::0::')
+        assertInvalid('1::abc::')
+        assertInvalid('1::abc::def')
+        assertInvalid('1:2:3:4:5:6:')
+        assertInvalid('1:2:3:4:5:6')
+        assertInvalid('1:2:3:4:5:6:7:8:')
+        assertInvalid('1:2:3:4:5:6:7:8:0')
+
+        self.assertEqual(b'\x00' * 12 + b'\xfe\x2a\x17\x40',
+            f('::254.42.23.64')
+        )
+        self.assertEqual(
+            b'\x00\x42' + b'\x00' * 8 + b'\xa2\x9b\xfe\x2a\x17\x40',
+            f('42::a29b:254.42.23.64')
+        )
+        self.assertEqual(
+            b'\x00\x42\xa8\xb9\x00\x00\x00\x02\xff\xff\xa2\x9b\xfe\x2a\x17\x40',
+            f('42:a8b9:0:2:ffff:a29b:254.42.23.64')
+        )
+        assertInvalid('255.254.253.252')
+        assertInvalid('1::260.2.3.0')
+        assertInvalid('1::0.be.e.0')
+        assertInvalid('1:2:3:4:5:6:7:1.2.3.4')
+        assertInvalid('::1.2.3.4:0')
+        assertInvalid('0.100.200.0:3:4:5:6:7:8')
 
     def testStringToIPv4(self):
         if not hasattr(socket, 'inet_ntop'):
             return # No inet_ntop() on this platform
         from socket import inet_ntoa as f, inet_ntop, AF_INET
         g = lambda a: inet_ntop(AF_INET, a)
+        assertInvalid = lambda func,a: self.assertRaises(
+            (socket.error, ValueError), func, a
+        )
 
         self.assertEqual('1.0.1.0', f(b'\x01\x00\x01\x00'))
         self.assertEqual('170.85.170.85', f(b'\xaa\x55\xaa\x55'))
         self.assertEqual('255.255.255.255', f(b'\xff\xff\xff\xff'))
         self.assertEqual('1.2.3.4', f(b'\x01\x02\x03\x04'))
+        assertInvalid(f, b'\x00' * 3)
+        assertInvalid(f, b'\x00' * 5)
+        assertInvalid(f, b'\x00' * 16)
 
         self.assertEqual('1.0.1.0', g(b'\x01\x00\x01\x00'))
         self.assertEqual('170.85.170.85', g(b'\xaa\x55\xaa\x55'))
         self.assertEqual('255.255.255.255', g(b'\xff\xff\xff\xff'))
+        assertInvalid(g, b'\x00' * 3)
+        assertInvalid(g, b'\x00' * 5)
+        assertInvalid(g, b'\x00' * 16)
 
     def testStringToIPv6(self):
         if not hasattr(socket, 'inet_ntop'):
@@ -950,6 +1008,9 @@
         except ImportError:
             return
         f = lambda a: inet_ntop(AF_INET6, a)
+        assertInvalid = lambda a: self.assertRaises(
+            (socket.error, ValueError), f, a
+        )
 
         self.assertEqual('::', f(b'\x00' * 16))
         self.assertEqual('::1', f(b'\x00' * 15 + b'\x01'))
@@ -958,6 +1019,10 @@
             f(b'\x0a\xef\x0b\x01\x05\x06\x10\x01\xff\xff\x99\x97\x00\x55\x01\x70')
         )
 
+        assertInvalid(b'\x12' * 15)
+        assertInvalid(b'\x12' * 17)
+        assertInvalid(b'\x12' * 4)
+
     # XXX The following don't test module-level functionality...
 
     def testSockName(self):
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -382,6 +382,7 @@
 Thomas G?ttler
 Jonas H.
 Barry Haddow
+Philipp Hagemeister
 Paul ten Hagen
 Rasmus Hahn
 Peter Haight

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 17:18:23 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 19 Dec 2011 17:18:23 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313627=3A_Add_suppo?=
 =?utf8?q?rt_for_SSL_Elliptic_Curve-based_Diffie-Hellman?=
Message-ID: <E1Rcfux-0001Ij-61@dinsdale.python.org>

http://hg.python.org/cpython/rev/8b729d65cfd2
changeset:   74084:8b729d65cfd2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 19 17:16:51 2011 +0100
summary:
  Issue #13627: Add support for SSL Elliptic Curve-based Diffie-Hellman
key exchange, through the SSLContext.set_ecdh_curve() method and the
ssl.OP_SINGLE_ECDH_USE option.

files:
  Doc/library/ssl.rst     |  26 +++++++++++++++++++++++++
  Lib/ssl.py              |   2 +-
  Lib/test/ssl_servers.py |   5 ++++
  Lib/test/test_ssl.py    |  10 +++++++++
  Misc/NEWS               |   4 +++
  Modules/_ssl.c          |  30 +++++++++++++++++++++++++++++
  6 files changed, 76 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -428,6 +428,14 @@
 
    .. versionadded:: 3.3
 
+.. data:: OP_SINGLE_ECDH_USE
+
+   Prevents re-use of the same ECDH key for several SSL sessions.  This
+   improves forward secrecy but requires more computational resources.
+   This option only applies to server sockets.
+
+   .. versionadded:: 3.3
+
 .. data:: HAS_SNI
 
    Whether the OpenSSL library has built-in support for the *Server Name
@@ -672,6 +680,24 @@
       when connected, the :meth:`SSLSocket.cipher` method of SSL sockets will
       give the currently selected cipher.
 
+.. method:: SSLContext.set_ecdh_curve(curve_name)
+
+   Set the curve name for Elliptic Curve-based Diffie-Hellman (abbreviated
+   ECDH) key exchange.  Using Diffie-Hellman key exchange improves forward
+   secrecy at the expense of computational resources (both on the server and
+   on the client).  The *curve_name* parameter should be a string describing
+   a well-known elliptic curve, for example ``prime256v1`` for a widely
+   supported curve.
+
+   This setting doesn't apply to client sockets.  You can also use the
+   :data:`OP_SINGLE_ECDH_USE` option to further improve security.
+
+   .. versionadded:: 3.3
+
+   .. seealso::
+      `SSL/TLS & Perfect Forward Secrecy <http://vincent.bernat.im/en/blog/2011-ssl-perfect-forward-secrecy.html>`_
+         Vincent Bernat.
+
 .. method:: SSLContext.wrap_socket(sock, server_side=False, \
       do_handshake_on_connect=True, suppress_ragged_eofs=True, \
       server_hostname=None)
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -68,7 +68,7 @@
 from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
 from _ssl import (
     OP_ALL, OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1,
-    OP_CIPHER_SERVER_PREFERENCE,
+    OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_ECDH_USE,
     )
 from _ssl import RAND_status, RAND_egd, RAND_add, RAND_bytes, RAND_pseudo_bytes
 from _ssl import (
diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py
--- a/Lib/test/ssl_servers.py
+++ b/Lib/test/ssl_servers.py
@@ -176,6 +176,9 @@
                         action='store_false', help='be less verbose')
     parser.add_argument('-s', '--stats', dest='use_stats_handler', default=False,
                         action='store_true', help='always return stats page')
+    parser.add_argument('--curve-name', dest='curve_name', type=str,
+                        action='store',
+                        help='curve name for EC-based Diffie-Hellman')
     args = parser.parse_args()
 
     support.verbose = args.verbose
@@ -186,6 +189,8 @@
         handler_class.root = os.getcwd()
     context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
     context.load_cert_chain(CERTFILE)
+    if args.curve_name:
+        context.set_ecdh_curve(args.curve_name)
 
     server = HTTPSServer(("", args.port), handler_class, context)
     if args.verbose:
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -99,6 +99,7 @@
         ssl.CERT_OPTIONAL
         ssl.CERT_REQUIRED
         ssl.OP_CIPHER_SERVER_PREFERENCE
+        ssl.OP_SINGLE_ECDH_USE
         self.assertIn(ssl.HAS_SNI, {True, False})
 
     def test_random(self):
@@ -558,6 +559,15 @@
         ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
         ctx.set_default_verify_paths()
 
+    def test_set_ecdh_curve(self):
+        ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+        ctx.set_ecdh_curve("prime256v1")
+        ctx.set_ecdh_curve(b"prime256v1")
+        self.assertRaises(TypeError, ctx.set_ecdh_curve)
+        self.assertRaises(TypeError, ctx.set_ecdh_curve, None)
+        self.assertRaises(ValueError, ctx.set_ecdh_curve, "foo")
+        self.assertRaises(ValueError, ctx.set_ecdh_curve, b"foo")
+
 
 class NetworkedTests(unittest.TestCase):
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,10 @@
 Library
 -------
 
+- Issue #13627: Add support for SSL Elliptic Curve-based Diffie-Hellman
+  key exchange, through the SSLContext.set_ecdh_curve() method and the
+  ssl.OP_SINGLE_ECDH_USE option.
+
 - Issue #13635: Add ssl.OP_CIPHER_SERVER_PREFERENCE, so that SSL servers
   choose the cipher based on their own preferences, rather than on the
   client's.
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1986,6 +1986,33 @@
     Py_RETURN_NONE;
 }
 
+static PyObject *
+set_ecdh_curve(PySSLContext *self, PyObject *name)
+{
+    PyObject *name_bytes;
+    int nid;
+    EC_KEY *key;
+
+    if (!PyUnicode_FSConverter(name, &name_bytes))
+        return NULL;
+    assert(PyBytes_Check(name_bytes));
+    nid = OBJ_sn2nid(PyBytes_AS_STRING(name_bytes));
+    Py_DECREF(name_bytes);
+    if (nid == 0) {
+        PyErr_Format(PyExc_ValueError,
+                     "unknown elliptic curve name %R", name);
+        return NULL;
+    }
+    key = EC_KEY_new_by_curve_name(nid);
+    if (key == NULL) {
+        _setSSLError(NULL, 0, __FILE__, __LINE__);
+        return NULL;
+    }
+    SSL_CTX_set_tmp_ecdh(self->ctx, key);
+    EC_KEY_free(key);
+    Py_RETURN_NONE;
+}
+
 static PyGetSetDef context_getsetlist[] = {
     {"options", (getter) get_options,
                 (setter) set_options, NULL},
@@ -2007,6 +2034,8 @@
                       METH_NOARGS, NULL},
     {"set_default_verify_paths", (PyCFunction) set_default_verify_paths,
                                  METH_NOARGS, NULL},
+    {"set_ecdh_curve", (PyCFunction) set_ecdh_curve,
+                       METH_O, NULL},
     {NULL, NULL}        /* sentinel */
 };
 
@@ -2452,6 +2481,7 @@
     PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1);
     PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE",
                             SSL_OP_CIPHER_SERVER_PREFERENCE);
+    PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE);
 
 #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
     r = Py_True;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 18:34:12 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Mon, 19 Dec 2011 18:34:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_=5FPy=5Ffopen_now_allows_by?=
 =?utf8?q?tes_filenames_under_non-Windows_platforms=2E?=
Message-ID: <E1Rch6K-0001C2-Oo@dinsdale.python.org>

http://hg.python.org/cpython/rev/a882aa30a713
changeset:   74085:a882aa30a713
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Mon Dec 19 18:19:06 2011 +0100
summary:
  _Py_fopen now allows bytes filenames under non-Windows platforms.

files:
  Python/fileutils.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -321,8 +321,8 @@
     return _wfopen(wpath, wmode);
 #else
     FILE *f;
-    PyObject *bytes = PyUnicode_EncodeFSDefault(path);
-    if (bytes == NULL)
+    PyObject *bytes;
+    if (!PyUnicode_FSConverter(path, &bytes))
         return NULL;
     f = fopen(PyBytes_AS_STRING(bytes), mode);
     Py_DECREF(bytes);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 19:12:07 2011
From: python-checkins at python.org (giampaolo.rodola)
Date: Mon, 19 Dec 2011 19:12:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_=28bug_=238684=29_fix_=27fe?=
 =?utf8?q?dora_without_thread_buildbot=27_as_per?=
Message-ID: <E1Rchh1-0005dB-GY@dinsdale.python.org>

http://hg.python.org/cpython/rev/50267d2bb320
changeset:   74086:50267d2bb320
user:        Giampaolo Rodola' <g.rodola at gmail.com>
date:        Mon Dec 19 19:12:01 2011 +0100
summary:
  (bug #8684) fix 'fedora without thread buildbot' as per http://bugs.python.org/issue8684

files:
  Lib/sched.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/sched.py b/Lib/sched.py
--- a/Lib/sched.py
+++ b/Lib/sched.py
@@ -30,8 +30,11 @@
 
 import time
 import heapq
-import threading
 from collections import namedtuple
+try:
+    import threading
+except ImportError:
+    import dummy_threading as threading
 
 __all__ = ["scheduler"]
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 22:13:59 2011
From: python-checkins at python.org (victor.stinner)
Date: Mon, 19 Dec 2011 22:13:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_backout_7876cd49300d=3A_Mov?=
 =?utf8?q?e_PyUnicode=5FWCHAR=5FKIND_outside_PyUnicode=5FKind_enum?=
Message-ID: <E1RckX1-0007sx-CD@dinsdale.python.org>

http://hg.python.org/cpython/rev/d2df3396fef0
changeset:   74087:d2df3396fef0
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Mon Dec 19 22:14:45 2011 +0100
summary:
  backout 7876cd49300d: Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum

files:
  Include/unicodeobject.h |  7 +++----
  1 files changed, 3 insertions(+), 4 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -445,13 +445,12 @@
 #define PyUnicode_IS_COMPACT_ASCII(op)                 \
     (((PyASCIIObject*)op)->state.ascii && PyUnicode_IS_COMPACT(op))
 
+enum PyUnicode_Kind {
 /* String contains only wstr byte characters.  This is only possible
    when the string was created with a legacy API and _PyUnicode_Ready()
    has not been called yet.  */
-#define PyUnicode_WCHAR_KIND 0
-
-/* Return values of the PyUnicode_KIND() macro */
-enum PyUnicode_Kind {
+    PyUnicode_WCHAR_KIND = 0,
+/* Return values of the PyUnicode_KIND() macro: */
     PyUnicode_1BYTE_KIND = 1,
     PyUnicode_2BYTE_KIND = 2,
     PyUnicode_4BYTE_KIND = 4

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 22:42:43 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 19 Dec 2011 22:42:43 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_don=27t_mention?=
	=?utf8?q?_implementation_detail?=
Message-ID: <E1Rckyp-0002W0-7n@dinsdale.python.org>

http://hg.python.org/cpython/rev/d85efd73b0e1
changeset:   74088:d85efd73b0e1
branch:      3.2
parent:      74082:71e5a083f9b1
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Dec 19 16:41:11 2011 -0500
summary:
  don't mention implementation detail

files:
  Doc/library/operator.rst |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,11 +12,11 @@
    from operator import itemgetter, iadd
 
 
-The :mod:`operator` module exports a set of functions implemented in C
-corresponding to the intrinsic operators of Python.  For example,
-``operator.add(x, y)`` is equivalent to the expression ``x+y``.  The function
-names are those used for special class methods; variants without leading and
-trailing ``__`` are also provided for convenience.
+The :mod:`operator` module exports a set of functions corresponding to the
+intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+equivalent to the expression ``x+y``.  The function names are those used for
+special class methods; variants without leading and trailing ``__`` are also
+provided for convenience.
 
 The functions fall into categories that perform object comparisons, logical
 operations, mathematical operations and sequence operations.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 22:42:44 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 19 Dec 2011 22:42:44 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_don=27t_mention?=
	=?utf8?q?_implementation_detail?=
Message-ID: <E1Rckyq-0002WB-1S@dinsdale.python.org>

http://hg.python.org/cpython/rev/355466216029
changeset:   74089:355466216029
branch:      2.7
parent:      74079:2228d985fdcc
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Dec 19 16:41:11 2011 -0500
summary:
  don't mention implementation detail

files:
  Doc/library/operator.rst |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,11 +12,11 @@
    from operator import itemgetter
 
 
-The :mod:`operator` module exports a set of functions implemented in C
-corresponding to the intrinsic operators of Python.  For example,
-``operator.add(x, y)`` is equivalent to the expression ``x+y``.  The function
-names are those used for special class methods; variants without leading and
-trailing ``__`` are also provided for convenience.
+The :mod:`operator` module exports a set of functions corresponding to the
+intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+equivalent to the expression ``x+y``.  The function names are those used for
+special class methods; variants without leading and trailing ``__`` are also
+provided for convenience.
 
 The functions fall into categories that perform object comparisons, logical
 operations, mathematical operations, sequence operations, and abstract type

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 19 22:42:44 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 19 Dec 2011 22:42:44 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1Rckyq-0002WF-Qn@dinsdale.python.org>

http://hg.python.org/cpython/rev/ab5d39caad6f
changeset:   74090:ab5d39caad6f
parent:      74087:d2df3396fef0
parent:      74088:d85efd73b0e1
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Dec 19 16:42:28 2011 -0500
summary:
  merge 3.2

files:
  Doc/library/operator.rst |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,11 +12,11 @@
    from operator import itemgetter, iadd
 
 
-The :mod:`operator` module exports a set of functions implemented in C
-corresponding to the intrinsic operators of Python.  For example,
-``operator.add(x, y)`` is equivalent to the expression ``x+y``.  The function
-names are those used for special class methods; variants without leading and
-trailing ``__`` are also provided for convenience.
+The :mod:`operator` module exports a set of functions corresponding to the
+intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+equivalent to the expression ``x+y``.  The function names are those used for
+special class methods; variants without leading and trailing ``__`` are also
+provided for convenience.
 
 The functions fall into categories that perform object comparisons, logical
 operations, mathematical operations and sequence operations.

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Dec 20 05:30:54 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 20 Dec 2011 05:30:54 +0100
Subject: [Python-checkins] Daily reference leaks (ab5d39caad6f): sum=0
Message-ID: <E1RcrLq-00062O-Nh@ap.vmr.nerim.net>

results for ab5d39caad6f on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogUFhL2R', '-x']

From python-checkins at python.org  Tue Dec 20 10:14:21 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 20 Dec 2011 10:14:21 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313634=3A_Add_suppo?=
 =?utf8?q?rt_for_querying_and_disabling_SSL_compression=2E?=
Message-ID: <E1Rcvm9-0000Nx-7Y@dinsdale.python.org>

http://hg.python.org/cpython/rev/20b52be99b5d
changeset:   74091:20b52be99b5d
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 20 10:13:40 2011 +0100
summary:
  Issue #13634: Add support for querying and disabling SSL compression.

files:
  Doc/library/ssl.rst     |  19 +++++++++++++++++++
  Lib/ssl.py              |  11 +++++++++++
  Lib/test/ssl_servers.py |   1 +
  Lib/test/test_ssl.py    |  26 ++++++++++++++++++++++++++
  Misc/NEWS               |   2 ++
  Modules/_ssl.c          |  24 ++++++++++++++++++++++++
  6 files changed, 83 insertions(+), 0 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -436,6 +436,15 @@
 
    .. versionadded:: 3.3
 
+.. data:: OP_NO_COMPRESSION
+
+   Disable compression on the SSL channel.  This is useful if the application
+   protocol supports its own compression scheme.
+
+   This option is only available with OpenSSL 1.0.0 and later.
+
+   .. versionadded:: 3.3
+
 .. data:: HAS_SNI
 
    Whether the OpenSSL library has built-in support for the *Server Name
@@ -561,6 +570,16 @@
    version of the SSL protocol that defines its use, and the number of secret
    bits being used.  If no connection has been established, returns ``None``.
 
+.. method:: SSLSocket.compression()
+
+   Return the compression algorithm being used as a string, or ``None``
+   if the connection isn't compressed.
+
+   If the higher-level protocol supports its own compression mechanism,
+   you can use :data:`OP_NO_COMPRESSION` to disable SSL-level compression.
+
+   .. versionadded:: 3.3
+
 .. method:: SSLSocket.get_channel_binding(cb_type="tls-unique")
 
    Get channel binding data for current connection, as a bytes object.  Returns
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -70,6 +70,10 @@
     OP_ALL, OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1,
     OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_ECDH_USE,
     )
+try:
+    from _ssl import OP_NO_COMPRESSION
+except ImportError:
+    pass
 from _ssl import RAND_status, RAND_egd, RAND_add, RAND_bytes, RAND_pseudo_bytes
 from _ssl import (
     SSL_ERROR_ZERO_RETURN,
@@ -330,6 +334,13 @@
         else:
             return self._sslobj.cipher()
 
+    def compression(self):
+        self._checkClosed()
+        if not self._sslobj:
+            return None
+        else:
+            return self._sslobj.compression()
+
     def send(self, data, flags=0):
         self._checkClosed()
         if self._sslobj:
diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py
--- a/Lib/test/ssl_servers.py
+++ b/Lib/test/ssl_servers.py
@@ -97,6 +97,7 @@
         stats = {
             'session_cache': context.session_stats(),
             'cipher': sock.cipher(),
+            'compression': sock.compression(),
             }
         body = pprint.pformat(stats)
         body = body.encode('utf-8')
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -100,6 +100,8 @@
         ssl.CERT_REQUIRED
         ssl.OP_CIPHER_SERVER_PREFERENCE
         ssl.OP_SINGLE_ECDH_USE
+        if ssl.OPENSSL_VERSION_INFO >= (1, 0):
+            ssl.OP_NO_COMPRESSION
         self.assertIn(ssl.HAS_SNI, {True, False})
 
     def test_random(self):
@@ -1185,7 +1187,12 @@
             if connectionchatty:
                 if support.verbose:
                     sys.stdout.write(" client:  closing connection.\n")
+            stats = {
+                'compression': s.compression(),
+                'cipher': s.cipher(),
+            }
             s.close()
+            return stats
         finally:
             server.stop()
             server.join()
@@ -1814,6 +1821,25 @@
                 server.stop()
                 server.join()
 
+        def test_compression(self):
+            context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            context.load_cert_chain(CERTFILE)
+            stats = server_params_test(context, context,
+                                       chatty=True, connectionchatty=True)
+            if support.verbose:
+                sys.stdout.write(" got compression: {!r}\n".format(stats['compression']))
+            self.assertIn(stats['compression'], { None, 'ZLIB', 'RLE' })
+
+        @unittest.skipUnless(hasattr(ssl, 'OP_NO_COMPRESSION'),
+                             "ssl.OP_NO_COMPRESSION needed for this test")
+        def test_compression_disabled(self):
+            context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            context.load_cert_chain(CERTFILE)
+            stats = server_params_test(context, context,
+                                       chatty=True, connectionchatty=True)
+            self.assertIs(stats['compression'], None)
+
+
 def test_main(verbose=False):
     if support.verbose:
         plats = {
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,8 @@
 Library
 -------
 
+- Issue #13634: Add support for querying and disabling SSL compression.
+
 - Issue #13627: Add support for SSL Elliptic Curve-based Diffie-Hellman
   key exchange, through the SSLContext.set_ecdh_curve() method and the
   ssl.OP_SINGLE_ECDH_USE option.
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -999,6 +999,25 @@
     return NULL;
 }
 
+static PyObject *PySSL_compression(PySSLSocket *self) {
+#ifdef OPENSSL_NO_COMP
+    Py_RETURN_NONE;
+#else
+    const COMP_METHOD *comp_method;
+    const char *short_name;
+
+    if (self->ssl == NULL)
+        Py_RETURN_NONE;
+    comp_method = SSL_get_current_compression(self->ssl);
+    if (comp_method == NULL || comp_method->type == NID_undef)
+        Py_RETURN_NONE;
+    short_name = OBJ_nid2sn(comp_method->type);
+    if (short_name == NULL)
+        Py_RETURN_NONE;
+    return PyUnicode_DecodeFSDefault(short_name);
+#endif
+}
+
 static void PySSL_dealloc(PySSLSocket *self)
 {
     if (self->peer_cert)        /* Possible not to have one? */
@@ -1452,6 +1471,7 @@
     {"peer_certificate", (PyCFunction)PySSL_peercert, METH_VARARGS,
      PySSL_peercert_doc},
     {"cipher", (PyCFunction)PySSL_cipher, METH_NOARGS},
+    {"compression", (PyCFunction)PySSL_compression, METH_NOARGS},
     {"shutdown", (PyCFunction)PySSL_SSLshutdown, METH_NOARGS,
      PySSL_SSLshutdown_doc},
 #if HAVE_OPENSSL_FINISHED
@@ -2482,6 +2502,10 @@
     PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE",
                             SSL_OP_CIPHER_SERVER_PREFERENCE);
     PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE);
+#ifdef SSL_OP_NO_COMPRESSION
+    PyModule_AddIntConstant(m, "OP_NO_COMPRESSION",
+                            SSL_OP_NO_COMPRESSION);
+#endif
 
 #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
     r = Py_True;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 10:48:22 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 20 Dec 2011 10:48:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_for_buggy_test?=
Message-ID: <E1RcwJ4-00043l-1P@dinsdale.python.org>

http://hg.python.org/cpython/rev/40f8e840bc24
changeset:   74092:40f8e840bc24
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 20 10:47:42 2011 +0100
summary:
  Fix for buggy test

files:
  Lib/test/test_ssl.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -1835,6 +1835,7 @@
         def test_compression_disabled(self):
             context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
             context.load_cert_chain(CERTFILE)
+            context.options |= ssl.OP_NO_COMPRESSION
             stats = server_params_test(context, context,
                                        chatty=True, connectionchatty=True)
             self.assertIs(stats['compression'], None)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 11:50:02 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Tue, 20 Dec 2011 11:50:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Followup_to_iss?=
 =?utf8?q?ue_=2311867=3A_Use_socketpair=28=29=2C_since_FreeBSD_=3C_8_doesn?=
 =?utf8?q?=27t_really?=
Message-ID: <E1RcxGk-0000rG-7n@dinsdale.python.org>

http://hg.python.org/cpython/rev/c9facd251725
changeset:   74093:c9facd251725
branch:      2.7
parent:      74089:355466216029
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Tue Dec 20 11:47:23 2011 +0100
summary:
  Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
support multiprocessing.Event.

files:
  Lib/test/test_mailbox.py |  19 ++++++++-----------
  1 files changed, 8 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -15,10 +15,6 @@
     import fcntl
 except ImportError:
     pass
-try:
-    import multiprocessing
-except ImportError:
-    multiprocessing = None
 
 # Silence Py3k warning
 rfc822 = test_support.import_module('rfc822', deprecated=True)
@@ -870,12 +866,13 @@
         self._box = self._factory(self._path)
 
     @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
-    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
+    @unittest.skipUnless(hasattr(socket, 'socketpair'), "Test needs socketpair().")
     def test_lock_conflict(self):
         # Fork off a child process that will lock the mailbox temporarily,
         # unlock it and exit.
-        ready = multiprocessing.Event()
-        done = multiprocessing.Event()
+        c, p = socket.socketpair()
+        self.addCleanup(c.close)
+        self.addCleanup(p.close)
 
         pid = os.fork()
         if pid == 0:
@@ -883,22 +880,22 @@
             try:
                 # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                ready.set()
+                c.send(b'c')
 
                 # wait until the parent is done, and unlock the mailbox
-                done.wait(5)
+                c.recv(1)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
         # In the parent, wait until the child signals it locked the mailbox.
-        ready.wait(5)
+        p.recv(1)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
             # Signal the child it can now release the lock and exit.
-            done.set()
+            p.send(b'p')
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 11:50:41 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Tue, 20 Dec 2011 11:50:41 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Followup_to_iss?=
 =?utf8?q?ue_=2311867=3A_Use_socketpair=28=29=2C_since_FreeBSD_=3C_8_doesn?=
 =?utf8?q?=27t_really?=
Message-ID: <E1RcxHN-00010C-P4@dinsdale.python.org>

http://hg.python.org/cpython/rev/9dee8a095faf
changeset:   74094:9dee8a095faf
branch:      3.2
parent:      74088:d85efd73b0e1
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Tue Dec 20 11:48:22 2011 +0100
summary:
  Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
support multiprocessing.Event.

files:
  Lib/test/test_mailbox.py |  19 ++++++++-----------
  1 files changed, 8 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -17,10 +17,6 @@
     import fcntl
 except ImportError:
     pass
-try:
-    import multiprocessing
-except ImportError:
-    multiprocessing = None
 
 
 class TestBase(unittest.TestCase):
@@ -999,12 +995,13 @@
         self._box = self._factory(self._path)
 
     @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
-    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
+    @unittest.skipUnless(hasattr(socket, 'socketpair'), "Test needs socketpair().")
     def test_lock_conflict(self):
         # Fork off a child process that will lock the mailbox temporarily,
         # unlock it and exit.
-        ready = multiprocessing.Event()
-        done = multiprocessing.Event()
+        c, p = socket.socketpair()
+        self.addCleanup(c.close)
+        self.addCleanup(p.close)
 
         pid = os.fork()
         if pid == 0:
@@ -1012,22 +1009,22 @@
             try:
                 # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                ready.set()
+                c.send(b'c')
 
                 # wait until the parent is done, and unlock the mailbox
-                done.wait(5)
+                c.recv(1)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
         # In the parent, wait until the child signals it locked the mailbox.
-        ready.wait(5)
+        p.recv(1)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
             # Signal the child it can now release the lock and exit.
-            done.set()
+            p.send(b'p')
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 11:51:03 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Tue, 20 Dec 2011 11:51:03 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Followup_to_issue_=2311867=3A_Use_socketpair=28=29=2C_since_?=
 =?utf8?q?FreeBSD_=3C_8_doesn=27t_really?=
Message-ID: <E1RcxHj-00011G-JD@dinsdale.python.org>

http://hg.python.org/cpython/rev/9014e0cc5589
changeset:   74095:9014e0cc5589
parent:      74092:40f8e840bc24
parent:      74094:9dee8a095faf
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Tue Dec 20 11:49:25 2011 +0100
summary:
  Followup to issue #11867: Use socketpair(), since FreeBSD < 8 doesn't really
support multiprocessing.Event.

files:
  Lib/test/test_mailbox.py |  19 ++++++++-----------
  1 files changed, 8 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -17,10 +17,6 @@
     import fcntl
 except ImportError:
     pass
-try:
-    import multiprocessing
-except ImportError:
-    multiprocessing = None
 
 
 class TestBase(unittest.TestCase):
@@ -998,12 +994,13 @@
         self._box = self._factory(self._path)
 
     @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
-    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
+    @unittest.skipUnless(hasattr(socket, 'socketpair'), "Test needs socketpair().")
     def test_lock_conflict(self):
         # Fork off a child process that will lock the mailbox temporarily,
         # unlock it and exit.
-        ready = multiprocessing.Event()
-        done = multiprocessing.Event()
+        c, p = socket.socketpair()
+        self.addCleanup(c.close)
+        self.addCleanup(p.close)
 
         pid = os.fork()
         if pid == 0:
@@ -1011,22 +1008,22 @@
             try:
                 # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                ready.set()
+                c.send(b'c')
 
                 # wait until the parent is done, and unlock the mailbox
-                done.wait(5)
+                c.recv(1)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
         # In the parent, wait until the child signals it locked the mailbox.
-        ready.wait(5)
+        p.recv(1)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
             # Signal the child it can now release the lock and exit.
-            done.set()
+            p.send(b'p')
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 14:00:53 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 20 Dec 2011 14:00:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313637=3A_=22a2b=22?=
 =?utf8?q?_functions_in_the_binascii_module_now_accept_ASCII-only?=
Message-ID: <E1RczJN-00058k-LF@dinsdale.python.org>

http://hg.python.org/cpython/rev/eb8d62706d5f
changeset:   74096:eb8d62706d5f
parent:      74092:40f8e840bc24
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 20 13:58:41 2011 +0100
summary:
  Issue #13637: "a2b" functions in the binascii module now accept ASCII-only unicode strings.

files:
  Doc/library/binascii.rst  |   9 +++-
  Lib/test/test_binascii.py |  34 ++++++++++++++++-
  Misc/NEWS                 |   3 +
  Modules/binascii.c        |  50 +++++++++++++++++++++++---
  4 files changed, 85 insertions(+), 11 deletions(-)


diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst
--- a/Doc/library/binascii.rst
+++ b/Doc/library/binascii.rst
@@ -20,8 +20,13 @@
 
 .. note::
 
-   Encoding and decoding functions do not accept Unicode strings.  Only bytestring
-   and bytearray objects can be processed.
+   ``a2b_*`` functions accept Unicode strings containing only ASCII characters.
+   Other functions only accept bytes and bytes-compatible objects (such as
+   bytearray objects and other objects implementing the buffer API).
+
+   .. versionchanged:: 3.3
+      ASCII-only unicode strings are now accepted by the ``a2b_*`` functions.
+
 
 The :mod:`binascii` module defines the following functions:
 
diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py
--- a/Lib/test/test_binascii.py
+++ b/Lib/test/test_binascii.py
@@ -208,9 +208,9 @@
             except Exception as err:
                 self.fail("{}({!r}) raises {!r}".format(func, empty, err))
 
-    def test_unicode_strings(self):
-        # Unicode strings are not accepted.
-        for func in all_functions:
+    def test_unicode_b2a(self):
+        # Unicode strings are not accepted by b2a_* functions.
+        for func in set(all_functions) - set(a2b_functions) | {'rledecode_hqx'}:
             try:
                 self.assertRaises(TypeError, getattr(binascii, func), "test")
             except Exception as err:
@@ -218,6 +218,34 @@
         # crc_hqx needs 2 arguments
         self.assertRaises(TypeError, binascii.crc_hqx, "test", 0)
 
+    def test_unicode_a2b(self):
+        # Unicode strings are accepted by a2b_* functions.
+        MAX_ALL = 45
+        raw = self.rawdata[:MAX_ALL]
+        for fa, fb in zip(a2b_functions, b2a_functions):
+            if fa == 'rledecode_hqx':
+                # Takes non-ASCII data
+                continue
+            a2b = getattr(binascii, fa)
+            b2a = getattr(binascii, fb)
+            try:
+                a = b2a(self.type2test(raw))
+                binary_res = a2b(a)
+                a = a.decode('ascii')
+                res = a2b(a)
+            except Exception as err:
+                self.fail("{}/{} conversion raises {!r}".format(fb, fa, err))
+            if fb == 'b2a_hqx':
+                # b2a_hqx returns a tuple
+                res, _ = res
+                binary_res, _ = binary_res
+            self.assertEqual(res, raw, "{}/{} conversion: "
+                             "{!r} != {!r}".format(fb, fa, res, raw))
+            self.assertEqual(res, binary_res)
+            self.assertIsInstance(res, bytes)
+            # non-ASCII string
+            self.assertRaises(ValueError, a2b, "\x80")
+
 
 class ArrayBinASCIITest(BinASCIITest):
     def type2test(self, s):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #13637: "a2b" functions in the binascii module now accept ASCII-only
+  unicode strings.
+
 - Issue #13634: Add support for querying and disabling SSL compression.
 
 - Issue #13627: Add support for SSL Elliptic Curve-based Diffie-Hellman
diff --git a/Modules/binascii.c b/Modules/binascii.c
--- a/Modules/binascii.c
+++ b/Modules/binascii.c
@@ -183,6 +183,44 @@
     0x6e17, 0x7e36, 0x4e55, 0x5e74, 0x2e93, 0x3eb2, 0x0ed1, 0x1ef0,
 };
 
+static int
+ascii_buffer_converter(PyObject *arg, Py_buffer *buf)
+{
+    if (arg == NULL) {
+        PyBuffer_Release(buf);
+        return 1;
+    }
+    if (PyUnicode_Check(arg)) {
+        if (PyUnicode_READY(arg) < 0)
+            return 0;
+        if (!PyUnicode_IS_ASCII(arg)) {
+            PyErr_SetString(PyExc_ValueError,
+                            "string argument should contain only ASCII characters");
+            return 0;
+        }
+        assert(PyUnicode_KIND(arg) == PyUnicode_1BYTE_KIND);
+        buf->buf = (void *) PyUnicode_1BYTE_DATA(arg);
+        buf->len = PyUnicode_GET_LENGTH(arg);
+        buf->obj = NULL;
+        return 1;
+    }
+    if (PyObject_GetBuffer(arg, buf, PyBUF_SIMPLE) != 0) {
+        PyErr_Format(PyExc_TypeError,
+                     "argument should be bytes, buffer or ASCII string, "
+                     "not %R", Py_TYPE(arg));
+        return 0;
+    }
+    if (!PyBuffer_IsContiguous(buf, 'C')) {
+        PyErr_Format(PyExc_TypeError,
+                     "argument should be a contiguous buffer, "
+                     "not %R", Py_TYPE(arg));
+        PyBuffer_Release(buf);
+        return 0;
+    }
+    return Py_CLEANUP_SUPPORTED;
+}
+
+
 PyDoc_STRVAR(doc_a2b_uu, "(ascii) -> bin. Decode a line of uuencoded data");
 
 static PyObject *
@@ -196,7 +234,7 @@
     PyObject *rv;
     Py_ssize_t ascii_len, bin_len;
 
-    if ( !PyArg_ParseTuple(args, "y*:a2b_uu", &pascii) )
+    if ( !PyArg_ParseTuple(args, "O&:a2b_uu", ascii_buffer_converter, &pascii) )
         return NULL;
     ascii_data = pascii.buf;
     ascii_len = pascii.len;
@@ -370,7 +408,7 @@
     Py_ssize_t ascii_len, bin_len;
     int quad_pos = 0;
 
-    if ( !PyArg_ParseTuple(args, "y*:a2b_base64", &pascii) )
+    if ( !PyArg_ParseTuple(args, "O&:a2b_base64", ascii_buffer_converter, &pascii) )
         return NULL;
     ascii_data = pascii.buf;
     ascii_len = pascii.len;
@@ -546,7 +584,7 @@
     Py_ssize_t len;
     int done = 0;
 
-    if ( !PyArg_ParseTuple(args, "y*:a2b_hqx", &pascii) )
+    if ( !PyArg_ParseTuple(args, "O&:a2b_hqx", ascii_buffer_converter, &pascii) )
         return NULL;
     ascii_data = pascii.buf;
     len = pascii.len;
@@ -1119,7 +1157,7 @@
     char* retbuf;
     Py_ssize_t i, j;
 
-    if (!PyArg_ParseTuple(args, "y*:a2b_hex", &parg))
+    if (!PyArg_ParseTuple(args, "O&:a2b_hex", ascii_buffer_converter, &parg))
         return NULL;
     argbuf = parg.buf;
     arglen = parg.len;
@@ -1197,8 +1235,8 @@
     static char *kwlist[] = {"data", "header", NULL};
     int header = 0;
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|i", kwlist, &pdata,
-          &header))
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|i:a2b_qp", kwlist,
+                                     ascii_buffer_converter, &pdata, &header))
         return NULL;
     data = pdata.buf;
     datalen = pdata.len;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 14:01:33 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 20 Dec 2011 14:01:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_Merge?=
Message-ID: <E1RczK1-0005Dd-4g@dinsdale.python.org>

http://hg.python.org/cpython/rev/a6e1d926cd98
changeset:   74097:a6e1d926cd98
parent:      74096:eb8d62706d5f
parent:      74095:9014e0cc5589
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Dec 20 14:00:12 2011 +0100
summary:
  Merge

files:
  Lib/test/test_mailbox.py |  19 ++++++++-----------
  1 files changed, 8 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -17,10 +17,6 @@
     import fcntl
 except ImportError:
     pass
-try:
-    import multiprocessing
-except ImportError:
-    multiprocessing = None
 
 
 class TestBase(unittest.TestCase):
@@ -998,12 +994,13 @@
         self._box = self._factory(self._path)
 
     @unittest.skipUnless(hasattr(os, 'fork'), "Test needs fork().")
-    @unittest.skipUnless(multiprocessing, "Test needs multiprocessing.")
+    @unittest.skipUnless(hasattr(socket, 'socketpair'), "Test needs socketpair().")
     def test_lock_conflict(self):
         # Fork off a child process that will lock the mailbox temporarily,
         # unlock it and exit.
-        ready = multiprocessing.Event()
-        done = multiprocessing.Event()
+        c, p = socket.socketpair()
+        self.addCleanup(c.close)
+        self.addCleanup(p.close)
 
         pid = os.fork()
         if pid == 0:
@@ -1011,22 +1008,22 @@
             try:
                 # lock the mailbox, and signal the parent it can proceed
                 self._box.lock()
-                ready.set()
+                c.send(b'c')
 
                 # wait until the parent is done, and unlock the mailbox
-                done.wait(5)
+                c.recv(1)
                 self._box.unlock()
             finally:
                 os._exit(0)
 
         # In the parent, wait until the child signals it locked the mailbox.
-        ready.wait(5)
+        p.recv(1)
         try:
             self.assertRaises(mailbox.ExternalClashError,
                               self._box.lock)
         finally:
             # Signal the child it can now release the lock and exit.
-            done.set()
+            p.send(b'p')
             # Wait for child to exit.  Locking should now succeed.
             exited_pid, status = os.waitpid(pid, 0)
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 17:13:33 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 20 Dec 2011 17:13:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_note_the_blindi?=
 =?utf8?q?ng_speed_of_these_functions?=
Message-ID: <E1Rd2Jp-0000im-Ac@dinsdale.python.org>

http://hg.python.org/cpython/rev/b68373aeb335
changeset:   74098:b68373aeb335
branch:      3.2
parent:      74094:9dee8a095faf
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 10:12:41 2011 -0600
summary:
  note the blinding speed of these functions

files:
  Doc/library/operator.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,8 +12,8 @@
    from operator import itemgetter, iadd
 
 
-The :mod:`operator` module exports a set of functions corresponding to the
-intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+The :mod:`operator` module exports a set of efficient functions corresponding to
+the intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
 equivalent to the expression ``x+y``.  The function names are those used for
 special class methods; variants without leading and trailing ``__`` are also
 provided for convenience.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 17:13:54 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 20 Dec 2011 17:13:54 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_note_the_blindi?=
 =?utf8?q?ng_speed_of_these_functions?=
Message-ID: <E1Rd2KA-0000n6-K1@dinsdale.python.org>

http://hg.python.org/cpython/rev/5e939912f9f8
changeset:   74099:5e939912f9f8
branch:      2.7
parent:      74093:c9facd251725
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 10:12:41 2011 -0600
summary:
  note the blinding speed of these functions

files:
  Doc/library/operator.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,8 +12,8 @@
    from operator import itemgetter
 
 
-The :mod:`operator` module exports a set of functions corresponding to the
-intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+The :mod:`operator` module exports a set of efficient functions corresponding to
+the intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
 equivalent to the expression ``x+y``.  The function names are those used for
 special class methods; variants without leading and trailing ``__`` are also
 provided for convenience.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 17:14:15 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 20 Dec 2011 17:14:15 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1Rd2KV-0000nn-RM@dinsdale.python.org>

http://hg.python.org/cpython/rev/ba29dcb3403f
changeset:   74100:ba29dcb3403f
parent:      74097:a6e1d926cd98
parent:      74098:b68373aeb335
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 10:13:05 2011 -0600
summary:
  merge 3.2

files:
  Doc/library/operator.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -12,8 +12,8 @@
    from operator import itemgetter, iadd
 
 
-The :mod:`operator` module exports a set of functions corresponding to the
-intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
+The :mod:`operator` module exports a set of efficient functions corresponding to
+the intrinsic operators of Python.  For example, ``operator.add(x, y)`` is
 equivalent to the expression ``x+y``.  The function names are those used for
 special class methods; variants without leading and trailing ``__`` are also
 provided for convenience.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 20:32:02 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 20 Dec 2011 20:32:02 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_possible_if?=
	=?utf8?q?_unlikely_leak?=
Message-ID: <E1Rd5Pu-0003BV-Kk@dinsdale.python.org>

http://hg.python.org/cpython/rev/40a211eb4fda
changeset:   74101:40a211eb4fda
branch:      3.2
parent:      74098:b68373aeb335
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 13:29:45 2011 -0600
summary:
  fix possible if unlikely leak

files:
  Objects/unicodeobject.c |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -8888,9 +8888,13 @@
         /* create entries for translating chars in x to those in y */
         for (i = 0; i < PyUnicode_GET_SIZE(x); i++) {
             key = PyLong_FromLong(PyUnicode_AS_UNICODE(x)[i]);
+            if (!key)
+                goto err;
             value = PyLong_FromLong(PyUnicode_AS_UNICODE(y)[i]);
-            if (!key || !value)
+            if (!value) {
+                Py_DECREF(key);
                 goto err;
+            }
             res = PyDict_SetItem(new, key, value);
             Py_DECREF(key);
             Py_DECREF(value);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 20 20:33:17 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 20 Dec 2011 20:33:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1Rd5R7-0003Hv-Po@dinsdale.python.org>

http://hg.python.org/cpython/rev/ac29dc61873c
changeset:   74102:ac29dc61873c
parent:      74100:ba29dcb3403f
parent:      74101:40a211eb4fda
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 13:32:50 2011 -0600
summary:
  merge 3.2

files:
  Objects/unicodeobject.c |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -12591,9 +12591,13 @@
         y_data = PyUnicode_DATA(y);
         for (i = 0; i < PyUnicode_GET_LENGTH(x); i++) {
             key = PyLong_FromLong(PyUnicode_READ(x_kind, x_data, i));
+            if (!key)
+                goto err;
             value = PyLong_FromLong(PyUnicode_READ(y_kind, y_data, i));
-            if (!key || !value)
+            if (!value) {
+                Py_DECREF(key);
                 goto err;
+            }
             res = PyDict_SetItem(new, key, value);
             Py_DECREF(key);
             Py_DECREF(value);

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Wed Dec 21 05:31:58 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 21 Dec 2011 05:31:58 +0100
Subject: [Python-checkins] Daily reference leaks (ac29dc61873c): sum=0
Message-ID: <E1RdDqQ-0006hi-Pe@ap.vmr.nerim.net>

results for ac29dc61873c on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog8Jn0HX', '-x']

From python-checkins at python.org  Wed Dec 21 09:28:24 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 09:28:24 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Fix_ssl_module_compilation_?=
 =?utf8?q?if_ECDH_support_was_disabled_in_the_OpenSSL_build=2E?=
Message-ID: <E1RdHXE-0003oV-AY@dinsdale.python.org>

http://hg.python.org/cpython/rev/ec44f2e82707
changeset:   74103:ec44f2e82707
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 09:27:41 2011 +0100
summary:
  Fix ssl module compilation if ECDH support was disabled in the OpenSSL build.
(followup to issue #13627)

files:
  Doc/library/ssl.rst  |  10 ++++++++++
  Lib/ssl.py           |   2 +-
  Lib/test/test_ssl.py |   2 ++
  Modules/_ssl.c       |  12 ++++++++++++
  4 files changed, 25 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -445,6 +445,14 @@
 
    .. versionadded:: 3.3
 
+.. data:: HAS_ECDH
+
+   Whether the OpenSSL library has built-in support for Elliptic Curve-based
+   Diffie-Hellman key exchange.  This should be true unless the feature was
+   explicitly disabled by the distributor.
+
+   .. versionadded:: 3.3
+
 .. data:: HAS_SNI
 
    Whether the OpenSSL library has built-in support for the *Server Name
@@ -711,6 +719,8 @@
    This setting doesn't apply to client sockets.  You can also use the
    :data:`OP_SINGLE_ECDH_USE` option to further improve security.
 
+   This method is not available if :data:`HAS_ECDH` is False.
+
    .. versionadded:: 3.3
 
    .. seealso::
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -86,7 +86,7 @@
     SSL_ERROR_EOF,
     SSL_ERROR_INVALID_ERROR_CODE,
     )
-from _ssl import HAS_SNI
+from _ssl import HAS_SNI, HAS_ECDH
 from _ssl import (PROTOCOL_SSLv3, PROTOCOL_SSLv23,
                   PROTOCOL_TLSv1)
 from _ssl import _OPENSSL_API_VERSION
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -103,6 +103,7 @@
         if ssl.OPENSSL_VERSION_INFO >= (1, 0):
             ssl.OP_NO_COMPRESSION
         self.assertIn(ssl.HAS_SNI, {True, False})
+        self.assertIn(ssl.HAS_ECDH, {True, False})
 
     def test_random(self):
         v = ssl.RAND_status()
@@ -561,6 +562,7 @@
         ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
         ctx.set_default_verify_paths()
 
+    @unittest.skipUnless(ssl.HAS_ECDH, "ECDH disabled on this OpenSSL build")
     def test_set_ecdh_curve(self):
         ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
         ctx.set_ecdh_curve("prime256v1")
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2006,6 +2006,7 @@
     Py_RETURN_NONE;
 }
 
+#ifndef OPENSSL_NO_ECDH
 static PyObject *
 set_ecdh_curve(PySSLContext *self, PyObject *name)
 {
@@ -2032,6 +2033,7 @@
     EC_KEY_free(key);
     Py_RETURN_NONE;
 }
+#endif
 
 static PyGetSetDef context_getsetlist[] = {
     {"options", (getter) get_options,
@@ -2054,8 +2056,10 @@
                       METH_NOARGS, NULL},
     {"set_default_verify_paths", (PyCFunction) set_default_verify_paths,
                                  METH_NOARGS, NULL},
+#ifndef OPENSSL_NO_ECDH
     {"set_ecdh_curve", (PyCFunction) set_ecdh_curve,
                        METH_O, NULL},
+#endif
     {NULL, NULL}        /* sentinel */
 };
 
@@ -2523,6 +2527,14 @@
     Py_INCREF(r);
     PyModule_AddObject(m, "HAS_TLS_UNIQUE", r);
 
+#ifdef OPENSSL_NO_ECDH
+    r = Py_False;
+#else
+    r = Py_True;
+#endif
+    Py_INCREF(r);
+    PyModule_AddObject(m, "HAS_ECDH", r);
+
     /* OpenSSL version */
     /* SSLeay() gives us the version of the library linked against,
        which could be different from the headers version.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 10:01:09 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 10:01:09 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzE3ODU6?=
 =?utf8?q?_Fix_inspect_and_pydoc_with_misbehaving_descriptors=2E?=
Message-ID: <E1RdI2v-0006PV-8N@dinsdale.python.org>

http://hg.python.org/cpython/rev/902f694a7b0e
changeset:   74104:902f694a7b0e
branch:      3.2
parent:      74101:40a211eb4fda
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 09:57:40 2011 +0100
summary:
  Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
Also fixes issue #13581: `help(type)` wouldn't display anything.

files:
  Lib/inspect.py           |  79 ++++++++++++++++-----------
  Lib/pydoc.py             |  29 ++++++++-
  Lib/test/test_inspect.py |  79 ++++++++++++++++++++++++++++
  Misc/NEWS                |   2 +
  4 files changed, 151 insertions(+), 38 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -100,11 +100,11 @@
     tests return false from the ismethoddescriptor() test, simply because
     the other tests promise more -- you can, e.g., count on having the
     __func__ attribute (etc) when an object passes ismethod()."""
-    return (hasattr(object, "__get__")
-            and not hasattr(object, "__set__") # else it's a data descriptor
-            and not ismethod(object)           # mutual exclusion
-            and not isfunction(object)
-            and not isclass(object))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__get__") and not hasattr(tp, "__set__")
 
 def isdatadescriptor(object):
     """Return true if the object is a data descriptor.
@@ -114,7 +114,11 @@
     Typically, data descriptors will also have __name__ and __doc__ attributes
     (properties, getsets, and members have both of these attributes), but this
     is not guaranteed."""
-    return (hasattr(object, "__set__") and hasattr(object, "__get__"))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__set__") and hasattr(tp, "__get__")
 
 if hasattr(types, 'MemberDescriptorType'):
     # CPython and equivalent
@@ -254,12 +258,23 @@
 def getmembers(object, predicate=None):
     """Return all members of an object as (name, value) pairs sorted by name.
     Optionally, only return members that satisfy a given predicate."""
+    if isclass(object):
+        mro = (object,) + getmro(object)
+    else:
+        mro = ()
     results = []
     for key in dir(object):
-        try:
-            value = getattr(object, key)
-        except AttributeError:
-            continue
+        # First try to get the value via __dict__. Some descriptors don't
+        # like calling their __get__ (see bug #1785).
+        for base in mro:
+            if key in base.__dict__:
+                value = base.__dict__[key]
+                break
+        else:
+            try:
+                value = getattr(object, key)
+            except AttributeError:
+                continue
         if not predicate or predicate(value):
             results.append((key, value))
     results.sort()
@@ -295,30 +310,21 @@
     names = dir(cls)
     result = []
     for name in names:
-        # Get the object associated with the name.
+        # Get the object associated with the name, and where it was defined.
         # Getting an obj from the __dict__ sometimes reveals more than
         # using getattr.  Static and class methods are dramatic examples.
-        if name in cls.__dict__:
-            obj = cls.__dict__[name]
+        # Furthermore, some objects may raise an Exception when fetched with
+        # getattr(). This is the case with some descriptors (bug #1785).
+        # Thus, we only use getattr() as a last resort.
+        homecls = None
+        for base in (cls,) + mro:
+            if name in base.__dict__:
+                obj = base.__dict__[name]
+                homecls = base
+                break
         else:
             obj = getattr(cls, name)
-
-        # Figure out where it was defined.
-        homecls = getattr(obj, "__objclass__", None)
-        if homecls is None:
-            # search the dicts.
-            for base in mro:
-                if name in base.__dict__:
-                    homecls = base
-                    break
-
-        # Get the object again, in order to get it from the defining
-        # __dict__ instead of via getattr (if possible).
-        if homecls is not None and name in homecls.__dict__:
-            obj = homecls.__dict__[name]
-
-        # Also get the object via getattr.
-        obj_via_getattr = getattr(cls, name)
+            homecls = getattr(obj, "__objclass__", homecls)
 
         # Classify the object.
         if isinstance(obj, staticmethod):
@@ -327,11 +333,18 @@
             kind = "class method"
         elif isinstance(obj, property):
             kind = "property"
-        elif (isfunction(obj_via_getattr) or
-              ismethoddescriptor(obj_via_getattr)):
+        elif ismethoddescriptor(obj):
             kind = "method"
+        elif isdatadescriptor(obj):
+            kind = "data"
         else:
-            kind = "data"
+            obj_via_getattr = getattr(cls, name)
+            if (isfunction(obj_via_getattr) or
+                ismethoddescriptor(obj_via_getattr)):
+                kind = "method"
+            else:
+                kind = "data"
+            obj = obj_via_getattr
 
         result.append(Attribute(name, kind, homecls, obj))
 
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -754,8 +754,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name), name, mod,
-                                       funcs, classes, mdict, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value, name, mod,
+                                        funcs, classes, mdict, object))
                     push('\n')
             return attrs
 
@@ -796,7 +803,12 @@
         mdict = {}
         for key, kind, homecls, value in attrs:
             mdict[key] = anchor = '#' + name + '-' + key
-            value = getattr(object, key)
+            try:
+                value = getattr(object, name)
+            except Exception:
+                # Some descriptors may meet a failure in their __get__.
+                # (bug #1785)
+                pass
             try:
                 # The value may not be hashable (e.g., a data attr with
                 # a dict or list value).
@@ -1180,8 +1192,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name),
-                                       name, mod, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value,
+                                        name, mod, object))
             return attrs
 
         def spilldescriptors(msg, attrs, predicate):
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -425,10 +425,37 @@
     def test_class(self):
         self.assertSourceEqual(self.fodderModule.X, 1, 2)
 
+
+class _BrokenDataDescriptor(object):
+    """
+    A broken data descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ data descriptors")
+
+    def __set__(*args):
+        raise RuntimeError
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ data descriptors")
+
+
+class _BrokenMethodDescriptor(object):
+    """
+    A broken method descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ method descriptors")
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ method descriptors")
+
+
 # Helper for testing classify_class_attrs.
 def attrs_wo_objs(cls):
     return [t[:3] for t in inspect.classify_class_attrs(cls)]
 
+
 class TestClassesAndFunctions(unittest.TestCase):
     def test_newstyle_mro(self):
         # The same w/ new-class MRO.
@@ -525,6 +552,9 @@
 
             datablob = '1'
 
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
         attrs = attrs_wo_objs(A)
         self.assertIn(('s', 'static method', A), attrs, 'missing static method')
         self.assertIn(('c', 'class method', A), attrs, 'missing class method')
@@ -533,6 +563,8 @@
                       'missing plain method: %r' % attrs)
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class B(A):
 
@@ -545,6 +577,8 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
 
         class C(A):
@@ -559,6 +593,8 @@
         self.assertIn(('m', 'method', C), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class D(B, C):
 
@@ -571,6 +607,49 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', D), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
+
+    def test_classify_builtin_types(self):
+        # Simple sanity check that all built-in types can have their
+        # attributes classified.
+        for name in dir(__builtins__):
+            builtin = getattr(__builtins__, name)
+            if isinstance(builtin, type):
+                inspect.classify_class_attrs(builtin)
+
+    def test_getmembers_descriptors(self):
+        class A(object):
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
+        def pred_wrapper(pred):
+            # A quick'n'dirty way to discard standard attributes of new-style
+            # classes.
+            class Empty(object):
+                pass
+            def wrapped(x):
+                if '__name__' in dir(x) and hasattr(Empty, x.__name__):
+                    return False
+                return pred(x)
+            return wrapped
+
+        ismethoddescriptor = pred_wrapper(inspect.ismethoddescriptor)
+        isdatadescriptor = pred_wrapper(inspect.isdatadescriptor)
+
+        self.assertEqual(inspect.getmembers(A, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(A, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        class B(A):
+            pass
+
+        self.assertEqual(inspect.getmembers(B, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(B, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
 
 class TestGetcallargsFunctions(unittest.TestCase):
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,8 @@
 Library
 -------
 
+- Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
+
 - Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas
   St?hrk.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 10:01:10 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 10:01:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=231785=3A_Fix_inspect_and_pydoc_with_misbehaving_desc?=
 =?utf8?q?riptors=2E?=
Message-ID: <E1RdI2w-0006Pj-8v@dinsdale.python.org>

http://hg.python.org/cpython/rev/b08bf8df8eec
changeset:   74105:b08bf8df8eec
parent:      74103:ec44f2e82707
parent:      74104:902f694a7b0e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 09:59:49 2011 +0100
summary:
  Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
Also fixes issue #13581: `help(type)` wouldn't display anything.

files:
  Lib/inspect.py           |  79 ++++++++++++++++-----------
  Lib/pydoc.py             |  29 ++++++++-
  Lib/test/test_inspect.py |  79 ++++++++++++++++++++++++++++
  Misc/NEWS                |   2 +
  4 files changed, 151 insertions(+), 38 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -99,11 +99,11 @@
     tests return false from the ismethoddescriptor() test, simply because
     the other tests promise more -- you can, e.g., count on having the
     __func__ attribute (etc) when an object passes ismethod()."""
-    return (hasattr(object, "__get__")
-            and not hasattr(object, "__set__") # else it's a data descriptor
-            and not ismethod(object)           # mutual exclusion
-            and not isfunction(object)
-            and not isclass(object))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__get__") and not hasattr(tp, "__set__")
 
 def isdatadescriptor(object):
     """Return true if the object is a data descriptor.
@@ -113,7 +113,11 @@
     Typically, data descriptors will also have __name__ and __doc__ attributes
     (properties, getsets, and members have both of these attributes), but this
     is not guaranteed."""
-    return (hasattr(object, "__set__") and hasattr(object, "__get__"))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__set__") and hasattr(tp, "__get__")
 
 if hasattr(types, 'MemberDescriptorType'):
     # CPython and equivalent
@@ -253,12 +257,23 @@
 def getmembers(object, predicate=None):
     """Return all members of an object as (name, value) pairs sorted by name.
     Optionally, only return members that satisfy a given predicate."""
+    if isclass(object):
+        mro = (object,) + getmro(object)
+    else:
+        mro = ()
     results = []
     for key in dir(object):
-        try:
-            value = getattr(object, key)
-        except AttributeError:
-            continue
+        # First try to get the value via __dict__. Some descriptors don't
+        # like calling their __get__ (see bug #1785).
+        for base in mro:
+            if key in base.__dict__:
+                value = base.__dict__[key]
+                break
+        else:
+            try:
+                value = getattr(object, key)
+            except AttributeError:
+                continue
         if not predicate or predicate(value):
             results.append((key, value))
     results.sort()
@@ -294,30 +309,21 @@
     names = dir(cls)
     result = []
     for name in names:
-        # Get the object associated with the name.
+        # Get the object associated with the name, and where it was defined.
         # Getting an obj from the __dict__ sometimes reveals more than
         # using getattr.  Static and class methods are dramatic examples.
-        if name in cls.__dict__:
-            obj = cls.__dict__[name]
+        # Furthermore, some objects may raise an Exception when fetched with
+        # getattr(). This is the case with some descriptors (bug #1785).
+        # Thus, we only use getattr() as a last resort.
+        homecls = None
+        for base in (cls,) + mro:
+            if name in base.__dict__:
+                obj = base.__dict__[name]
+                homecls = base
+                break
         else:
             obj = getattr(cls, name)
-
-        # Figure out where it was defined.
-        homecls = getattr(obj, "__objclass__", None)
-        if homecls is None:
-            # search the dicts.
-            for base in mro:
-                if name in base.__dict__:
-                    homecls = base
-                    break
-
-        # Get the object again, in order to get it from the defining
-        # __dict__ instead of via getattr (if possible).
-        if homecls is not None and name in homecls.__dict__:
-            obj = homecls.__dict__[name]
-
-        # Also get the object via getattr.
-        obj_via_getattr = getattr(cls, name)
+            homecls = getattr(obj, "__objclass__", homecls)
 
         # Classify the object.
         if isinstance(obj, staticmethod):
@@ -326,11 +332,18 @@
             kind = "class method"
         elif isinstance(obj, property):
             kind = "property"
-        elif (isfunction(obj_via_getattr) or
-              ismethoddescriptor(obj_via_getattr)):
+        elif ismethoddescriptor(obj):
             kind = "method"
+        elif isdatadescriptor(obj):
+            kind = "data"
         else:
-            kind = "data"
+            obj_via_getattr = getattr(cls, name)
+            if (isfunction(obj_via_getattr) or
+                ismethoddescriptor(obj_via_getattr)):
+                kind = "method"
+            else:
+                kind = "data"
+            obj = obj_via_getattr
 
         result.append(Attribute(name, kind, homecls, obj))
 
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -748,8 +748,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name), name, mod,
-                                       funcs, classes, mdict, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value, name, mod,
+                                        funcs, classes, mdict, object))
                     push('\n')
             return attrs
 
@@ -790,7 +797,12 @@
         mdict = {}
         for key, kind, homecls, value in attrs:
             mdict[key] = anchor = '#' + name + '-' + key
-            value = getattr(object, key)
+            try:
+                value = getattr(object, name)
+            except Exception:
+                # Some descriptors may meet a failure in their __get__.
+                # (bug #1785)
+                pass
             try:
                 # The value may not be hashable (e.g., a data attr with
                 # a dict or list value).
@@ -1177,8 +1189,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name),
-                                       name, mod, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value,
+                                        name, mod, object))
             return attrs
 
         def spilldescriptors(msg, attrs, predicate):
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -425,10 +425,37 @@
     def test_class(self):
         self.assertSourceEqual(self.fodderModule.X, 1, 2)
 
+
+class _BrokenDataDescriptor(object):
+    """
+    A broken data descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ data descriptors")
+
+    def __set__(*args):
+        raise RuntimeError
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ data descriptors")
+
+
+class _BrokenMethodDescriptor(object):
+    """
+    A broken method descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ method descriptors")
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ method descriptors")
+
+
 # Helper for testing classify_class_attrs.
 def attrs_wo_objs(cls):
     return [t[:3] for t in inspect.classify_class_attrs(cls)]
 
+
 class TestClassesAndFunctions(unittest.TestCase):
     def test_newstyle_mro(self):
         # The same w/ new-class MRO.
@@ -525,6 +552,9 @@
 
             datablob = '1'
 
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
         attrs = attrs_wo_objs(A)
         self.assertIn(('s', 'static method', A), attrs, 'missing static method')
         self.assertIn(('c', 'class method', A), attrs, 'missing class method')
@@ -533,6 +563,8 @@
                       'missing plain method: %r' % attrs)
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class B(A):
 
@@ -545,6 +577,8 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
 
         class C(A):
@@ -559,6 +593,8 @@
         self.assertIn(('m', 'method', C), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class D(B, C):
 
@@ -571,6 +607,49 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', D), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
+
+    def test_classify_builtin_types(self):
+        # Simple sanity check that all built-in types can have their
+        # attributes classified.
+        for name in dir(__builtins__):
+            builtin = getattr(__builtins__, name)
+            if isinstance(builtin, type):
+                inspect.classify_class_attrs(builtin)
+
+    def test_getmembers_descriptors(self):
+        class A(object):
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
+        def pred_wrapper(pred):
+            # A quick'n'dirty way to discard standard attributes of new-style
+            # classes.
+            class Empty(object):
+                pass
+            def wrapped(x):
+                if '__name__' in dir(x) and hasattr(Empty, x.__name__):
+                    return False
+                return pred(x)
+            return wrapped
+
+        ismethoddescriptor = pred_wrapper(inspect.ismethoddescriptor)
+        isdatadescriptor = pred_wrapper(inspect.isdatadescriptor)
+
+        self.assertEqual(inspect.getmembers(A, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(A, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        class B(A):
+            pass
+
+        self.assertEqual(inspect.getmembers(B, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(B, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
 
 class TestGetcallargsFunctions(unittest.TestCase):
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,8 @@
 Library
 -------
 
+- Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
+
 - Issue #13637: "a2b" functions in the binascii module now accept ASCII-only
   unicode strings.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 10:18:05 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 10:18:05 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE3ODU6?=
 =?utf8?q?_Fix_inspect_and_pydoc_with_misbehaving_descriptors=2E?=
Message-ID: <E1RdIJJ-0008JT-8V@dinsdale.python.org>

http://hg.python.org/cpython/rev/13f56cd8dec1
changeset:   74106:13f56cd8dec1
branch:      2.7
parent:      74099:5e939912f9f8
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 10:16:14 2011 +0100
summary:
  Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
Also fixes issue #13581: `help(type)` wouldn't display anything.

files:
  Lib/inspect.py           |  63 ++++++++++-------
  Lib/pydoc.py             |  29 +++++++-
  Lib/test/test_inspect.py |  96 ++++++++++++++++++++++++++++
  Misc/NEWS                |   2 +
  4 files changed, 158 insertions(+), 32 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -247,12 +247,23 @@
 def getmembers(object, predicate=None):
     """Return all members of an object as (name, value) pairs sorted by name.
     Optionally, only return members that satisfy a given predicate."""
+    if isclass(object):
+        mro = (object,) + getmro(object)
+    else:
+        mro = ()
     results = []
     for key in dir(object):
-        try:
-            value = getattr(object, key)
-        except AttributeError:
-            continue
+        # First try to get the value via __dict__. Some descriptors don't
+        # like calling their __get__ (see bug #1785).
+        for base in mro:
+            if key in base.__dict__:
+                value = base.__dict__[key]
+                break
+        else:
+            try:
+                value = getattr(object, key)
+            except AttributeError:
+                continue
         if not predicate or predicate(value):
             results.append((key, value))
     results.sort()
@@ -288,30 +299,21 @@
     names = dir(cls)
     result = []
     for name in names:
-        # Get the object associated with the name.
+        # Get the object associated with the name, and where it was defined.
         # Getting an obj from the __dict__ sometimes reveals more than
         # using getattr.  Static and class methods are dramatic examples.
-        if name in cls.__dict__:
-            obj = cls.__dict__[name]
+        # Furthermore, some objects may raise an Exception when fetched with
+        # getattr(). This is the case with some descriptors (bug #1785).
+        # Thus, we only use getattr() as a last resort.
+        homecls = None
+        for base in (cls,) + mro:
+            if name in base.__dict__:
+                obj = base.__dict__[name]
+                homecls = base
+                break
         else:
             obj = getattr(cls, name)
-
-        # Figure out where it was defined.
-        homecls = getattr(obj, "__objclass__", None)
-        if homecls is None:
-            # search the dicts.
-            for base in mro:
-                if name in base.__dict__:
-                    homecls = base
-                    break
-
-        # Get the object again, in order to get it from the defining
-        # __dict__ instead of via getattr (if possible).
-        if homecls is not None and name in homecls.__dict__:
-            obj = homecls.__dict__[name]
-
-        # Also get the object via getattr.
-        obj_via_getattr = getattr(cls, name)
+            homecls = getattr(obj, "__objclass__", homecls)
 
         # Classify the object.
         if isinstance(obj, staticmethod):
@@ -320,11 +322,18 @@
             kind = "class method"
         elif isinstance(obj, property):
             kind = "property"
-        elif (ismethod(obj_via_getattr) or
-              ismethoddescriptor(obj_via_getattr)):
+        elif ismethoddescriptor(obj):
             kind = "method"
+        elif isdatadescriptor(obj):
+            kind = "data"
         else:
-            kind = "data"
+            obj_via_getattr = getattr(cls, name)
+            if (ismethod(obj_via_getattr) or
+                ismethoddescriptor(obj_via_getattr)):
+                kind = "method"
+            else:
+                kind = "data"
+            obj = obj_via_getattr
 
         result.append(Attribute(name, kind, homecls, obj))
 
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -740,8 +740,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name), name, mod,
-                                       funcs, classes, mdict, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value, name, mod,
+                                        funcs, classes, mdict, object))
                     push('\n')
             return attrs
 
@@ -781,7 +788,12 @@
         mdict = {}
         for key, kind, homecls, value in attrs:
             mdict[key] = anchor = '#' + name + '-' + key
-            value = getattr(object, key)
+            try:
+                value = getattr(object, name)
+            except Exception:
+                # Some descriptors may meet a failure in their __get__.
+                # (bug #1785)
+                pass
             try:
                 # The value may not be hashable (e.g., a data attr with
                 # a dict or list value).
@@ -1161,8 +1173,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name),
-                                       name, mod, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value,
+                                        name, mod, object))
             return attrs
 
         def spilldescriptors(msg, attrs, predicate):
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -404,10 +404,37 @@
         self.assertEqual(inspect.findsource(co), (lines,0))
         self.assertEqual(inspect.getsource(co), lines[0])
 
+
+class _BrokenDataDescriptor(object):
+    """
+    A broken data descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ data descriptors")
+
+    def __set__(*args):
+        raise RuntimeError
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ data descriptors")
+
+
+class _BrokenMethodDescriptor(object):
+    """
+    A broken method descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ method descriptors")
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ method descriptors")
+
+
 # Helper for testing classify_class_attrs.
 def attrs_wo_objs(cls):
     return [t[:3] for t in inspect.classify_class_attrs(cls)]
 
+
 class TestClassesAndFunctions(unittest.TestCase):
     def test_classic_mro(self):
         # Test classic-class method resolution order.
@@ -494,6 +521,9 @@
 
             datablob = '1'
 
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
         attrs = attrs_wo_objs(A)
         self.assertIn(('s', 'static method', A), attrs, 'missing static method')
         self.assertIn(('c', 'class method', A), attrs, 'missing class method')
@@ -501,6 +531,8 @@
         self.assertIn(('m', 'method', A), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class B(A):
             def m(self): pass
@@ -512,6 +544,8 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
 
         class C(A):
@@ -525,6 +559,8 @@
         self.assertIn(('m', 'method', C), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class D(B, C):
             def m1(self): pass
@@ -539,6 +575,8 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', D), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
 
     def test_classify_oldstyle(self):
@@ -554,6 +592,64 @@
         """
         self._classify_test(True)
 
+    def test_classify_builtin_types(self):
+        # Simple sanity check that all built-in types can have their
+        # attributes classified.
+        for name in dir(__builtin__):
+            builtin = getattr(__builtin__, name)
+            if isinstance(builtin, type):
+                inspect.classify_class_attrs(builtin)
+
+    def test_getmembers_descriptors(self):
+        # Old-style classes
+        class A:
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
+        self.assertEqual(inspect.getmembers(A, inspect.ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(A, inspect.isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        class B(A):
+            pass
+
+        self.assertEqual(inspect.getmembers(B, inspect.ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(B, inspect.isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        # New-style classes
+        class A(object):
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
+        def pred_wrapper(pred):
+            # A quick'n'dirty way to discard standard attributes of new-style
+            # classes.
+            class Empty(object):
+                pass
+            def wrapped(x):
+                if hasattr(x, '__name__') and hasattr(Empty, x.__name__):
+                    return False
+                return pred(x)
+            return wrapped
+
+        ismethoddescriptor = pred_wrapper(inspect.ismethoddescriptor)
+        isdatadescriptor = pred_wrapper(inspect.isdatadescriptor)
+
+        self.assertEqual(inspect.getmembers(A, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(A, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        class B(A):
+            pass
+
+        self.assertEqual(inspect.getmembers(B, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(B, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
 
 
 class TestGetcallargsFunctions(unittest.TestCase):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,8 @@
 Library
 -------
 
+- Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
+
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by
   C?dric Krier.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 11:04:29 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 11:04:29 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2312708=3A_Add_starm?=
 =?utf8?q?ap=28=29_and_starmap=5Fasync=28=29_methods_=28similar_to?=
Message-ID: <E1RdJ2D-0003dc-M9@dinsdale.python.org>

http://hg.python.org/cpython/rev/b07b1e58582d
changeset:   74107:b07b1e58582d
parent:      74105:b08bf8df8eec
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 11:03:24 2011 +0100
summary:
  Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
Patch by Hynek Schlawack.

files:
  Doc/library/multiprocessing.rst  |  18 +++++++++++
  Lib/multiprocessing/managers.py  |   3 +-
  Lib/multiprocessing/pool.py      |  32 ++++++++++++++++++-
  Lib/test/test_multiprocessing.py |  18 +++++++++++
  Misc/ACKS                        |   1 +
  Misc/NEWS                        |   3 +
  6 files changed, 72 insertions(+), 3 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -1669,6 +1669,24 @@
       returned iterator should be considered arbitrary.  (Only when there is
       only one worker process is the order guaranteed to be "correct".)
 
+   .. method:: starmap(func, iterable[, chunksize])
+
+      Like :meth:`map` except that the elements of the `iterable` are expected
+      to be iterables that are unpacked as arguments.
+
+      Hence an `iterable` of `[(1,2), (3, 4)]` results in `[func(1,2),
+      func(3,4)]`.
+
+      .. versionadded:: 3.3
+
+   .. method:: starmap_async(func, iterable[, chunksize[, callback[, error_back]]])
+
+      A combination of :meth:`starmap` and :meth:`map_async` that iterates over
+      `iterable` of iterables and calls `func` with the iterables unpacked.
+      Returns a result object.
+
+      .. versionadded:: 3.3
+
    .. method:: close()
 
       Prevents any more tasks from being submitted to the pool.  Once all the
diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -1066,11 +1066,12 @@
 
 PoolProxy = MakeProxyType('PoolProxy', (
     'apply', 'apply_async', 'close', 'imap', 'imap_unordered', 'join',
-    'map', 'map_async', 'terminate'
+    'map', 'map_async', 'starmap', 'starmap_async', 'terminate'
     ))
 PoolProxy._method_to_typeid_ = {
     'apply_async': 'AsyncResult',
     'map_async': 'AsyncResult',
+    'starmap_async': 'AsyncResult',
     'imap': 'Iterator',
     'imap_unordered': 'Iterator'
     }
diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py
--- a/Lib/multiprocessing/pool.py
+++ b/Lib/multiprocessing/pool.py
@@ -64,6 +64,9 @@
 def mapstar(args):
     return list(map(*args))
 
+def starmapstar(args):
+    return list(itertools.starmap(args[0], args[1]))
+
 #
 # Code run by worker processes
 #
@@ -248,7 +251,25 @@
         in a list that is returned.
         '''
         assert self._state == RUN
-        return self.map_async(func, iterable, chunksize).get()
+        return self._map_async(func, iterable, mapstar, chunksize).get()
+
+    def starmap(self, func, iterable, chunksize=None):
+        '''
+        Like `map()` method but the elements of the `iterable` are expected to
+        be iterables as well and will be unpacked as arguments. Hence
+        `func` and (a, b) becomes func(a, b).
+        '''
+        assert self._state == RUN
+        return self._map_async(func, iterable, starmapstar, chunksize).get()
+
+    def starmap_async(self, func, iterable, chunksize=None, callback=None,
+            error_callback=None):
+        '''
+        Asynchronous version of `starmap()` method.
+        '''
+        assert self._state == RUN
+        return self._map_async(func, iterable, starmapstar, chunksize,
+                               callback, error_callback)
 
     def imap(self, func, iterable, chunksize=1):
         '''
@@ -302,6 +323,13 @@
         Asynchronous version of `map()` method.
         '''
         assert self._state == RUN
+        return self._map_async(func, iterable, mapstar, chunksize)
+
+    def _map_async(self, func, iterable, mapper, chunksize=None, callback=None,
+            error_callback=None):
+        '''
+        Helper function to implement map, starmap and their async counterparts.
+        '''
         if not hasattr(iterable, '__len__'):
             iterable = list(iterable)
 
@@ -315,7 +343,7 @@
         task_batches = Pool._get_tasks(func, iterable, chunksize)
         result = MapResult(self._cache, chunksize, len(iterable), callback,
                            error_callback=error_callback)
-        self._taskqueue.put((((result._job, i, mapstar, (x,), {})
+        self._taskqueue.put((((result._job, i, mapper, (x,), {})
                               for i, x in enumerate(task_batches)), None))
         return result
 
diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py
--- a/Lib/test/test_multiprocessing.py
+++ b/Lib/test/test_multiprocessing.py
@@ -8,6 +8,7 @@
 import queue as pyqueue
 import time
 import io
+import itertools
 import sys
 import os
 import gc
@@ -1125,6 +1126,9 @@
     time.sleep(wait)
     return x*x
 
+def mul(x, y):
+    return x*y
+
 class _TestPool(BaseTestCase):
 
     def test_apply(self):
@@ -1138,6 +1142,20 @@
         self.assertEqual(pmap(sqr, list(range(100)), chunksize=20),
                          list(map(sqr, list(range(100)))))
 
+    def test_starmap(self):
+        psmap = self.pool.starmap
+        tuples = list(zip(range(10), range(9,-1, -1)))
+        self.assertEqual(psmap(mul, tuples),
+                         list(itertools.starmap(mul, tuples)))
+        tuples = list(zip(range(100), range(99,-1, -1)))
+        self.assertEqual(psmap(mul, tuples, chunksize=20),
+                         list(itertools.starmap(mul, tuples)))
+
+    def test_starmap_async(self):
+        tuples = list(zip(range(100), range(99,-1, -1)))
+        self.assertEqual(self.pool.starmap_async(mul, tuples).get(),
+                         list(itertools.starmap(mul, tuples)))
+
     def test_map_chunksize(self):
         try:
             self.pool.map_async(sqr, [], chunksize=1).get(timeout=TIMEOUT1)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -878,6 +878,7 @@
 Andreas Schawo
 Neil Schemenauer
 David Scherer
+Hynek Schlawack
 Bob Schmertz
 Gregor Schmid
 Ralf Schmitt
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #12708: Add starmap() and starmap_async() methods (similar to
+  itertools.starmap()) to multiprocessing.Pool.  Patch by Hynek Schlawack.
+
 - Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
 
 - Issue #13637: "a2b" functions in the binascii module now accept ASCII-only

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 11:25:30 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 11:25:30 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTk3?=
 =?utf8?q?=3A_Fix_the_documentation_of_the_=22-u=22_command-line_option=2C?=
 =?utf8?q?_and?=
Message-ID: <E1RdJMY-0005JM-Bu@dinsdale.python.org>

http://hg.python.org/cpython/rev/19df72a77b39
changeset:   74108:19df72a77b39
branch:      3.2
parent:      74104:902f694a7b0e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 11:24:15 2011 +0100
summary:
  Issue #13597: Fix the documentation of the "-u" command-line option, and wording of "What's new in Python 3.0" about standard streams.

files:
  Doc/using/cmdline.rst |   3 ++-
  Doc/whatsnew/3.0.rst  |  10 +++++++---
  2 files changed, 9 insertions(+), 4 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -247,7 +247,8 @@
 
    Force the binary layer of the stdin, stdout and stderr streams (which is
    available as their ``buffer`` attribute) to be unbuffered.  The text I/O
-   layer will still be line-buffered.
+   layer will still be line-buffered if writing to the console, or
+   block-buffered if redirected to a non-interactive file.
 
    See also :envvar:`PYTHONUNBUFFERED`.
 
diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst
--- a/Doc/whatsnew/3.0.rst
+++ b/Doc/whatsnew/3.0.rst
@@ -301,6 +301,12 @@
   There is no longer any need for using the encoding-aware streams
   in the :mod:`codecs` module.
 
+* The initial values of :data:`sys.stdin`, :data:`sys.stdout` and
+  :data:`sys.stderr` are now unicode-only text files (i.e., they are
+  instances of :class:`io.TextIOBase`).  To read and write bytes data
+  with these streams, you need to use their :data:`io.TextIOBase.buffer`
+  attribute.
+
 * Filenames are passed to and returned from APIs as (Unicode) strings.
   This can present platform-specific problems because on some
   platforms filenames are arbitrary byte strings.  (On the other hand,
@@ -511,9 +517,7 @@
   produces a literal of type :class:`bytes`.
 
 * :ref:`pep-3116`.  The :mod:`io` module is now the standard way of
-  doing file I/O, and the initial values of :data:`sys.stdin`,
-  :data:`sys.stdout` and :data:`sys.stderr` are now instances of
-  :class:`io.TextIOBase`.  The built-in :func:`open` function is now an
+  doing file I/O.  The built-in :func:`open` function is now an
   alias for :func:`io.open` and has additional keyword arguments
   *encoding*, *errors*, *newline* and *closefd*.  Also note that an
   invalid *mode* argument now raises :exc:`ValueError`, not

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 11:25:31 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 11:25:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313597=3A_Fix_the_documentation_of_the_=22-u=22_comm?=
 =?utf8?q?and-line_option=2C_and?=
Message-ID: <E1RdJMZ-0005JQ-9I@dinsdale.python.org>

http://hg.python.org/cpython/rev/1ab124a6f171
changeset:   74109:1ab124a6f171
parent:      74107:b07b1e58582d
parent:      74108:19df72a77b39
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 11:24:47 2011 +0100
summary:
  Issue #13597: Fix the documentation of the "-u" command-line option, and wording of "What's new in Python 3.0" about standard streams.

files:
  Doc/using/cmdline.rst |   3 ++-
  Doc/whatsnew/3.0.rst  |  10 +++++++---
  2 files changed, 9 insertions(+), 4 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -249,7 +249,8 @@
 
    Force the binary layer of the stdin, stdout and stderr streams (which is
    available as their ``buffer`` attribute) to be unbuffered.  The text I/O
-   layer will still be line-buffered.
+   layer will still be line-buffered if writing to the console, or
+   block-buffered if redirected to a non-interactive file.
 
    See also :envvar:`PYTHONUNBUFFERED`.
 
diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst
--- a/Doc/whatsnew/3.0.rst
+++ b/Doc/whatsnew/3.0.rst
@@ -301,6 +301,12 @@
   There is no longer any need for using the encoding-aware streams
   in the :mod:`codecs` module.
 
+* The initial values of :data:`sys.stdin`, :data:`sys.stdout` and
+  :data:`sys.stderr` are now unicode-only text files (i.e., they are
+  instances of :class:`io.TextIOBase`).  To read and write bytes data
+  with these streams, you need to use their :data:`io.TextIOBase.buffer`
+  attribute.
+
 * Filenames are passed to and returned from APIs as (Unicode) strings.
   This can present platform-specific problems because on some
   platforms filenames are arbitrary byte strings.  (On the other hand,
@@ -511,9 +517,7 @@
   produces a literal of type :class:`bytes`.
 
 * :ref:`pep-3116`.  The :mod:`io` module is now the standard way of
-  doing file I/O, and the initial values of :data:`sys.stdin`,
-  :data:`sys.stdout` and :data:`sys.stderr` are now instances of
-  :class:`io.TextIOBase`.  The built-in :func:`open` function is now an
+  doing file I/O.  The built-in :func:`open` function is now an
   alias for :func:`io.open` and has additional keyword arguments
   *encoding*, *errors*, *newline* and *closefd*.  Also note that an
   invalid *mode* argument now raises :exc:`ValueError`, not

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 15:31:53 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Wed, 21 Dec 2011 15:31:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_13620_-_Support_chrom?=
 =?utf8?q?e_browser_in_webbrowser=2Epy_module=2E?=
Message-ID: <E1RdNCz-0001Ks-Hw@dinsdale.python.org>

http://hg.python.org/cpython/rev/7faa90a6324c
changeset:   74110:7faa90a6324c
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Wed Dec 21 22:20:32 2011 +0800
summary:
  Issue 13620 - Support chrome browser in webbrowser.py module.

files:
  Lib/webbrowser.py |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py
--- a/Lib/webbrowser.py
+++ b/Lib/webbrowser.py
@@ -299,6 +299,18 @@
     background = True
 
 
+class Chrome(UnixBrowser):
+    "Launcher class for Google Chrome browser."
+
+    remote_args = ['%action', '%s']
+    remote_action = ""
+    remote_action_newwin = "--new-window"
+    remote_action_newtab = ""
+    background = True
+
+Chromium = Chrome
+
+
 class Opera(UnixBrowser):
     "Launcher class for Opera browser."
 
@@ -466,6 +478,11 @@
     if _iscommand("skipstone"):
         register("skipstone", None, BackgroundBrowser("skipstone"))
 
+    # Google Chrome/Chromium browsers
+    for browser in ("google-chrome", "chrome", "chromium", "chromium-browser"):
+        if _iscommand(browser):
+            register(browser, None, Chrome(browser))
+
     # Opera, quite popular
     if _iscommand("opera"):
         register("opera", None, Opera("opera"))

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 15:31:56 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Wed, 21 Dec 2011 15:31:56 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Docs_and_News_update_for_Is?=
 =?utf8?q?sue13620=2E_Chrome_support_in_webbrowser=2Epy?=
Message-ID: <E1RdND2-0001Kw-Bq@dinsdale.python.org>

http://hg.python.org/cpython/rev/bd3631f9aa5c
changeset:   74111:bd3631f9aa5c
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Wed Dec 21 22:31:36 2011 +0800
summary:
  Docs and News update for Issue13620. Chrome support in webbrowser.py

files:
  Doc/library/webbrowser.rst |  90 ++++++++++++++-----------
  Misc/NEWS                  |   3 +
  2 files changed, 52 insertions(+), 41 deletions(-)


diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst
--- a/Doc/library/webbrowser.rst
+++ b/Doc/library/webbrowser.rst
@@ -96,47 +96,55 @@
 may be passed to the :func:`get` function and the corresponding instantiations
 for the controller classes, all defined in this module.
 
-+-----------------------+-----------------------------------------+-------+
-| Type Name             | Class Name                              | Notes |
-+=======================+=========================================+=======+
-| ``'mozilla'``         | :class:`Mozilla('mozilla')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'firefox'``         | :class:`Mozilla('mozilla')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'netscape'``        | :class:`Mozilla('netscape')`            |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'galeon'``          | :class:`Galeon('galeon')`               |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'epiphany'``        | :class:`Galeon('epiphany')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'skipstone'``       | :class:`BackgroundBrowser('skipstone')` |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'kfmclient'``       | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'konqueror'``       | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'kfm'``             | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'mosaic'``          | :class:`BackgroundBrowser('mosaic')`    |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'opera'``           | :class:`Opera()`                        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'grail'``           | :class:`Grail()`                        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'links'``           | :class:`GenericBrowser('links')`        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'elinks'``          | :class:`Elinks('elinks')`               |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'lynx'``            | :class:`GenericBrowser('lynx')`         |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'w3m'``             | :class:`GenericBrowser('w3m')`          |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'windows-default'`` | :class:`WindowsDefault`                 | \(2)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'internet-config'`` | :class:`InternetConfig`                 | \(3)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'macosx'``          | :class:`MacOSX('default')`              | \(4)  |
-+-----------------------+-----------------------------------------+-------+
++------------------------+-----------------------------------------+-------+
+| Type Name              | Class Name                              | Notes |
++========================+=========================================+=======+
+| ``'mozilla'``          | :class:`Mozilla('mozilla')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'firefox'``          | :class:`Mozilla('mozilla')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'netscape'``         | :class:`Mozilla('netscape')`            |       |
++------------------------+-----------------------------------------+-------+
+| ``'galeon'``           | :class:`Galeon('galeon')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'epiphany'``         | :class:`Galeon('epiphany')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'skipstone'``        | :class:`BackgroundBrowser('skipstone')` |       |
++------------------------+-----------------------------------------+-------+
+| ``'kfmclient'``        | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'konqueror'``        | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'kfm'``              | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'mosaic'``           | :class:`BackgroundBrowser('mosaic')`    |       |
++------------------------+-----------------------------------------+-------+
+| ``'opera'``            | :class:`Opera()`                        |       |
++------------------------+-----------------------------------------+-------+
+| ``'grail'``            | :class:`Grail()`                        |       |
++------------------------+-----------------------------------------+-------+
+| ``'links'``            | :class:`GenericBrowser('links')`        |       |
++------------------------+-----------------------------------------+-------+
+| ``'elinks'``           | :class:`Elinks('elinks')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'lynx'``             | :class:`GenericBrowser('lynx')`         |       |
++------------------------+-----------------------------------------+-------+
+| ``'w3m'``              | :class:`GenericBrowser('w3m')`          |       |
++------------------------+-----------------------------------------+-------+
+| ``'windows-default'``  | :class:`WindowsDefault`                 | \(2)  |
++------------------------+-----------------------------------------+-------+
+| ``'internet-config'``  | :class:`InternetConfig`                 | \(3)  |
++------------------------+-----------------------------------------+-------+
+| ``'macosx'``           | :class:`MacOSX('default')`              | \(4)  |
++------------------------+-----------------------------------------+-------+
+| ``'google-chrome'``    | :class:`Chrome('google-chrome')`        |       |
++------------------------+-----------------------------------------+-------+
+| ``'chrome'``           | :class:`Chrome('chrome')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'chromium'``         | :class:`Chromium('chromium')`           |       |
++------------------------+-----------------------------------------+-------+
+| ``'chromium-browser'`` | :class:`Chromium('chromium-browser')`   |       |
++------------------------+-----------------------------------------+-------+
 
 Notes:
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #13620: Support for Chrome browser in webbrowser.py Patch contributed
+  by Arnaud Calmettes.
+
 - Issue #12708: Add starmap() and starmap_async() methods (similar to
   itertools.starmap()) to multiprocessing.Pool.  Patch by Hynek Schlawack.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 15:54:00 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 15:54:00 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNjQ1?=
 =?utf8?q?=3A_fix_test=5Fimport_failure_when_run_immediately_after_test=5F?=
 =?utf8?q?coding=2E?=
Message-ID: <E1RdNYO-0003bj-B6@dinsdale.python.org>

http://hg.python.org/cpython/rev/f4afc6858ed2
changeset:   74112:f4afc6858ed2
branch:      3.2
parent:      74108:19df72a77b39
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 15:50:42 2011 +0100
summary:
  Issue #13645: fix test_import failure when run immediately after test_coding.

files:
  Lib/test/test_import.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -107,8 +107,9 @@
                 open(fname, 'w').close()
                 os.chmod(fname, (stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH |
                                  stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH))
+                fn = imp.cache_from_source(fname)
+                unlink(fn)
                 __import__(TESTFN)
-                fn = imp.cache_from_source(fname)
                 if not os.path.exists(fn):
                     self.fail("__import__ did not result in creation of "
                               "either a .pyc or .pyo file")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 15:54:22 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 15:54:22 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313645=3A_fix_test=5Fimport_failure_when_run_immedia?=
 =?utf8?q?tely_after_test=5Fcoding=2E?=
Message-ID: <E1RdNYk-0003cM-3t@dinsdale.python.org>

http://hg.python.org/cpython/rev/a6bd166abde5
changeset:   74113:a6bd166abde5
parent:      74111:bd3631f9aa5c
parent:      74112:f4afc6858ed2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 15:53:16 2011 +0100
summary:
  Issue #13645: fix test_import failure when run immediately after test_coding.

files:
  Lib/test/test_import.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -104,8 +104,9 @@
             try:
                 fname = TESTFN + os.extsep + "py"
                 create_empty_file(fname)
+                fn = imp.cache_from_source(fname)
+                unlink(fn)
                 __import__(TESTFN)
-                fn = imp.cache_from_source(fname)
                 if not os.path.exists(fn):
                     self.fail("__import__ did not result in creation of "
                               "either a .pyc or .pyo file")

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 16:02:36 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 16:02:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Fix_test=5Fimpo?=
 =?utf8?q?rt_failure_when_run_multiple_times=2E?=
Message-ID: <E1RdNgi-0004Jc-W4@dinsdale.python.org>

http://hg.python.org/cpython/rev/ea72a1fc4b92
changeset:   74114:ea72a1fc4b92
branch:      2.7
parent:      74106:13f56cd8dec1
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 16:01:53 2011 +0100
summary:
  Fix test_import failure when run multiple times.
(2.7-only)

files:
  Lib/test/test_import.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -263,7 +263,10 @@
                   import imp
             sys.argv.insert(0, C())
             """))
-        script_helper.assert_python_ok(testfn)
+        try:
+            script_helper.assert_python_ok(testfn)
+        finally:
+            unlink(testfn)
 
     def test_bug7732(self):
         source = TESTFN + '.py'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 17:02:17 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 17:02:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Use_context_man?=
 =?utf8?q?agers_in_test=5Fssl_to_simplify_test_writing=2E?=
Message-ID: <E1RdOcT-0001pw-KK@dinsdale.python.org>

http://hg.python.org/cpython/rev/82557279efd2
changeset:   74115:82557279efd2
branch:      3.2
parent:      74112:f4afc6858ed2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 16:52:40 2011 +0100
summary:
  Use context managers in test_ssl to simplify test writing.

files:
  Lib/test/test_ssl.py |  103 ++++++++++--------------------
  1 files changed, 35 insertions(+), 68 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -881,6 +881,14 @@
             threading.Thread.__init__(self)
             self.daemon = True
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            self.stop()
+            self.join()
+
         def start(self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -993,6 +1001,20 @@
         def __str__(self):
             return "<%s %s>" % (self.__class__.__name__, self.server)
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            if support.verbose:
+                sys.stdout.write(" cleanup: stopping server.\n")
+            self.stop()
+            if support.verbose:
+                sys.stdout.write(" cleanup: joining server thread.\n")
+            self.join()
+            if support.verbose:
+                sys.stdout.write(" cleanup: successfully joined.\n")
+
         def start (self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -1020,12 +1042,7 @@
                                     certreqs=ssl.CERT_REQUIRED,
                                     cacerts=CERTFILE, chatty=False,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             try:
                 with socket.socket() as sock:
                     s = ssl.wrap_socket(sock,
@@ -1045,9 +1062,6 @@
                     sys.stdout.write("\IOError is %s\n" % str(x))
             else:
                 raise AssertionError("Use of invalid cert should have failed!")
-        finally:
-            server.stop()
-            server.join()
 
     def server_params_test(client_context, server_context, indata=b"FOO\n",
                            chatty=True, connectionchatty=False):
@@ -1058,12 +1072,7 @@
         server = ThreadedEchoServer(context=server_context,
                                     chatty=chatty,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             s = client_context.wrap_socket(socket.socket())
             s.connect((HOST, server.port))
             for arg in [indata, bytearray(indata), memoryview(indata)]:
@@ -1086,9 +1095,6 @@
                 if support.verbose:
                     sys.stdout.write(" client:  closing connection.\n")
             s.close()
-        finally:
-            server.stop()
-            server.join()
 
     def try_protocol_combo(server_protocol, client_protocol, expect_success,
                            certsreqs=None, server_options=0, client_options=0):
@@ -1157,12 +1163,7 @@
             context.load_verify_locations(CERTFILE)
             context.load_cert_chain(CERTFILE)
             server = ThreadedEchoServer(context=context, chatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = context.wrap_socket(socket.socket())
                 s.connect((HOST, server.port))
                 cert = s.getpeercert()
@@ -1185,9 +1186,6 @@
                 after = ssl.cert_time_to_seconds(cert['notAfter'])
                 self.assertLess(before, after)
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_empty_cert(self):
             """Connecting with an empty cert file"""
@@ -1346,13 +1344,8 @@
                                         starttls_server=True,
                                         chatty=True,
                                         connectionchatty=True)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
             wrapped = False
-            try:
+            with server:
                 s = socket.socket()
                 s.setblocking(1)
                 s.connect((HOST, server.port))
@@ -1399,9 +1392,6 @@
                     conn.close()
                 else:
                     s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_socketserver(self):
             """Using a SocketServer to create and manage SSL connections."""
@@ -1437,12 +1427,7 @@
 
             indata = b"FOO\n"
             server = AsyncoreEchoServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = ssl.wrap_socket(socket.socket())
                 s.connect(('127.0.0.1', server.port))
                 if support.verbose:
@@ -1463,15 +1448,6 @@
                 s.close()
                 if support.verbose:
                     sys.stdout.write(" client:  connection closed.\n")
-            finally:
-                if support.verbose:
-                    sys.stdout.write(" cleanup: stopping server.\n")
-                server.stop()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: joining server thread.\n")
-                server.join()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: successfully joined.\n")
 
         def test_recv_send(self):
             """Test recv(), send() and friends."""
@@ -1484,19 +1460,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # helper methods for standardising recv* method signatures
                 def _recv_into():
                     b = bytearray(b"\0"*100)
@@ -1581,12 +1552,8 @@
                             )
                         # consume data
                         s.read()
-
                 s.write(b"over\n")
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_handshake_timeout(self):
             # Issue #5103: SSL handshake must respect the socket timeout

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 17:02:18 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 17:02:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Use_context_managers_in_test=5Fssl_to_simplify_test_writing?=
 =?utf8?q?=2E?=
Message-ID: <E1RdOcU-0001q6-E1@dinsdale.python.org>

http://hg.python.org/cpython/rev/f3041e7f535d
changeset:   74116:f3041e7f535d
parent:      74113:a6bd166abde5
parent:      74115:82557279efd2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 16:54:45 2011 +0100
summary:
  Use context managers in test_ssl to simplify test writing.

files:
  Lib/test/test_ssl.py |  126 ++++++++++--------------------
  1 files changed, 43 insertions(+), 83 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -986,6 +986,14 @@
             threading.Thread.__init__(self)
             self.daemon = True
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            self.stop()
+            self.join()
+
         def start(self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -1097,6 +1105,20 @@
         def __str__(self):
             return "<%s %s>" % (self.__class__.__name__, self.server)
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            if support.verbose:
+                sys.stdout.write(" cleanup: stopping server.\n")
+            self.stop()
+            if support.verbose:
+                sys.stdout.write(" cleanup: joining server thread.\n")
+            self.join()
+            if support.verbose:
+                sys.stdout.write(" cleanup: successfully joined.\n")
+
         def start (self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -1124,12 +1146,7 @@
                                     certreqs=ssl.CERT_REQUIRED,
                                     cacerts=CERTFILE, chatty=False,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             try:
                 with socket.socket() as sock:
                     s = ssl.wrap_socket(sock,
@@ -1149,9 +1166,6 @@
                     sys.stdout.write("\IOError is %s\n" % str(x))
             else:
                 raise AssertionError("Use of invalid cert should have failed!")
-        finally:
-            server.stop()
-            server.join()
 
     def server_params_test(client_context, server_context, indata=b"FOO\n",
                            chatty=True, connectionchatty=False):
@@ -1162,12 +1176,7 @@
         server = ThreadedEchoServer(context=server_context,
                                     chatty=chatty,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             s = client_context.wrap_socket(socket.socket())
             s.connect((HOST, server.port))
             for arg in [indata, bytearray(indata), memoryview(indata)]:
@@ -1195,9 +1204,6 @@
             }
             s.close()
             return stats
-        finally:
-            server.stop()
-            server.join()
 
     def try_protocol_combo(server_protocol, client_protocol, expect_success,
                            certsreqs=None, server_options=0, client_options=0):
@@ -1266,12 +1272,7 @@
             context.load_verify_locations(CERTFILE)
             context.load_cert_chain(CERTFILE)
             server = ThreadedEchoServer(context=context, chatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = context.wrap_socket(socket.socket())
                 s.connect((HOST, server.port))
                 cert = s.getpeercert()
@@ -1294,9 +1295,6 @@
                 after = ssl.cert_time_to_seconds(cert['notAfter'])
                 self.assertLess(before, after)
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_empty_cert(self):
             """Connecting with an empty cert file"""
@@ -1456,13 +1454,8 @@
                                         starttls_server=True,
                                         chatty=True,
                                         connectionchatty=True)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
             wrapped = False
-            try:
+            with server:
                 s = socket.socket()
                 s.setblocking(1)
                 s.connect((HOST, server.port))
@@ -1509,9 +1502,6 @@
                     conn.close()
                 else:
                     s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_socketserver(self):
             """Using a SocketServer to create and manage SSL connections."""
@@ -1547,12 +1537,7 @@
 
             indata = b"FOO\n"
             server = AsyncoreEchoServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = ssl.wrap_socket(socket.socket())
                 s.connect(('127.0.0.1', server.port))
                 if support.verbose:
@@ -1573,15 +1558,6 @@
                 s.close()
                 if support.verbose:
                     sys.stdout.write(" client:  connection closed.\n")
-            finally:
-                if support.verbose:
-                    sys.stdout.write(" cleanup: stopping server.\n")
-                server.stop()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: joining server thread.\n")
-                server.join()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: successfully joined.\n")
 
         def test_recv_send(self):
             """Test recv(), send() and friends."""
@@ -1594,19 +1570,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # helper methods for standardising recv* method signatures
                 def _recv_into():
                     b = bytearray(b"\0"*100)
@@ -1702,9 +1673,6 @@
 
                 s.write(b"over\n")
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_handshake_timeout(self):
             # Issue #5103: SSL handshake must respect the socket timeout
@@ -1768,19 +1736,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # get the data
                 cb_data = s.get_channel_binding("tls-unique")
                 if support.verbose:
@@ -1819,9 +1782,6 @@
                 self.assertEqual(peer_data_repr,
                                  repr(new_cb_data).encode("us-ascii"))
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_compression(self):
             context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 17:02:19 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Wed, 21 Dec 2011 17:02:19 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Use_context_man?=
 =?utf8?q?agers_in_test=5Fssl_to_simplify_test_writing=2E?=
Message-ID: <E1RdOcV-0001qA-8I@dinsdale.python.org>

http://hg.python.org/cpython/rev/d868208b831d
changeset:   74117:d868208b831d
branch:      2.7
parent:      74114:ea72a1fc4b92
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Wed Dec 21 16:52:40 2011 +0100
summary:
  Use context managers in test_ssl to simplify test writing.

files:
  Lib/test/test_ssl.py |  102 +++++++++++-------------------
  1 files changed, 38 insertions(+), 64 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -532,6 +532,14 @@
             threading.Thread.__init__(self)
             self.daemon = True
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            self.stop()
+            self.join()
+
         def start(self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -638,6 +646,20 @@
         def __str__(self):
             return "<%s %s>" % (self.__class__.__name__, self.server)
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            if test_support.verbose:
+                sys.stdout.write(" cleanup: stopping server.\n")
+            self.stop()
+            if test_support.verbose:
+                sys.stdout.write(" cleanup: joining server thread.\n")
+            self.join()
+            if test_support.verbose:
+                sys.stdout.write(" cleanup: successfully joined.\n")
+
         def start(self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -752,12 +774,7 @@
         server = ThreadedEchoServer(CERTFILE,
                                     certreqs=ssl.CERT_REQUIRED,
                                     cacerts=CERTFILE, chatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             try:
                 s = ssl.wrap_socket(socket.socket(),
                                     certfile=certfile,
@@ -771,9 +788,6 @@
                     sys.stdout.write("\nsocket.error is %s\n" % x[1])
             else:
                 raise AssertionError("Use of invalid cert should have failed!")
-        finally:
-            server.stop()
-            server.join()
 
     def server_params_test(certfile, protocol, certreqs, cacertsfile,
                            client_certfile, client_protocol=None, indata="FOO\n",
@@ -791,14 +805,10 @@
                                     chatty=chatty,
                                     connectionchatty=connectionchatty,
                                     wrap_accepting_socket=wrap_accepting_socket)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        if client_protocol is None:
-            client_protocol = protocol
-        try:
+        with server:
+            # try to connect
+            if client_protocol is None:
+                client_protocol = protocol
             s = ssl.wrap_socket(socket.socket(),
                                 certfile=client_certfile,
                                 ca_certs=cacertsfile,
@@ -826,9 +836,6 @@
                 if test_support.verbose:
                     sys.stdout.write(" client:  closing connection.\n")
             s.close()
-        finally:
-            server.stop()
-            server.join()
 
     def try_protocol_combo(server_protocol,
                            client_protocol,
@@ -930,12 +937,7 @@
                                         ssl_version=ssl.PROTOCOL_SSLv23,
                                         cacerts=CERTFILE,
                                         chatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = ssl.wrap_socket(socket.socket(),
                                     certfile=CERTFILE,
                                     ca_certs=CERTFILE,
@@ -957,9 +959,6 @@
                         "Missing or invalid 'organizationName' field in certificate subject; "
                         "should be 'Python Software Foundation'.")
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_empty_cert(self):
             """Connecting with an empty cert file"""
@@ -1042,13 +1041,8 @@
                                         starttls_server=True,
                                         chatty=True,
                                         connectionchatty=True)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
             wrapped = False
-            try:
+            with server:
                 s = socket.socket()
                 s.setblocking(1)
                 s.connect((HOST, server.port))
@@ -1093,9 +1087,6 @@
                 else:
                     s.send("over\n")
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_socketserver(self):
             """Using a SocketServer to create and manage SSL connections."""
@@ -1145,12 +1136,7 @@
             if test_support.verbose:
                 sys.stdout.write("\n")
             server = AsyncoreEchoServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = ssl.wrap_socket(socket.socket())
                 s.connect(('127.0.0.1', server.port))
                 if test_support.verbose:
@@ -1169,10 +1155,6 @@
                 if test_support.verbose:
                     sys.stdout.write(" client:  closing connection.\n")
                 s.close()
-            finally:
-                server.stop()
-                # wait for server thread to end
-                server.join()
 
         def test_recv_send(self):
             """Test recv(), send() and friends."""
@@ -1185,19 +1167,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # helper methods for standardising recv* method signatures
                 def _recv_into():
                     b = bytearray("\0"*100)
@@ -1285,9 +1262,6 @@
 
                 s.write("over\n".encode("ASCII", "strict"))
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_handshake_timeout(self):
             # Issue #5103: SSL handshake must respect the socket timeout

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 18:22:34 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 21 Dec 2011 18:22:34 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_spacing_around_switch_s?=
	=?utf8?q?tatements?=
Message-ID: <E1RdPsA-00015F-OZ@dinsdale.python.org>

http://hg.python.org/cpython/rev/6cc55bb70714
changeset:   74118:6cc55bb70714
parent:      74102:ac29dc61873c
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 17:23:42 2011 -0600
summary:
  fix spacing around switch statements

files:
  Objects/unicodeobject.c |  45 ++++++++++++++--------------
  1 files changed, 22 insertions(+), 23 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -871,7 +871,7 @@
     {
         if (!PyUnicode_IS_READY(unicode))
             return "wstr";
-        switch(PyUnicode_KIND(unicode))
+        switch (PyUnicode_KIND(unicode))
         {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(unicode))
@@ -887,8 +887,7 @@
         }
     }
     assert(PyUnicode_IS_READY(unicode));
-    switch(PyUnicode_KIND(unicode))
-    {
+    switch (PyUnicode_KIND(unicode)) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(unicode))
             return "ascii";
@@ -1791,7 +1790,7 @@
 static Py_UCS4
 kind_maxchar_limit(unsigned int kind)
 {
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         return 0x80;
     case PyUnicode_2BYTE_KIND:
@@ -1892,7 +1891,7 @@
         PyErr_SetString(PyExc_ValueError, "size must be positive");
         return NULL;
     }
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         return _PyUnicode_FromUCS1(buffer, size);
     case PyUnicode_2BYTE_KIND:
@@ -1994,7 +1993,7 @@
         PyErr_SetString(PyExc_SystemError, "invalid widening attempt");
         return NULL;
     }
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_2BYTE_KIND:
         result = PyMem_Malloc(len * sizeof(Py_UCS2));
         if (!result)
@@ -5007,7 +5006,7 @@
     data = PyUnicode_DATA(unicode);
     size = PyUnicode_GET_LENGTH(unicode);
 
-    switch(kind) {
+    switch (kind) {
     default:
         assert(0);
     case PyUnicode_1BYTE_KIND:
@@ -6026,7 +6025,7 @@
     len = PyUnicode_GET_LENGTH(unicode);
     kind = PyUnicode_KIND(unicode);
     data = PyUnicode_DATA(unicode);
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND: expandsize = 4; break;
     case PyUnicode_2BYTE_KIND: expandsize = 6; break;
     case PyUnicode_4BYTE_KIND: expandsize = 10; break;
@@ -9028,7 +9027,7 @@
     len2 = PyUnicode_GET_LENGTH(s2);
 
     if (direction > 0) {
-        switch(kind) {
+        switch (kind) {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(s1) && PyUnicode_IS_ASCII(s2))
                 result = asciilib_find_slice(buf1, len1, buf2, len2, start, end);
@@ -9046,7 +9045,7 @@
         }
     }
     else {
-        switch(kind) {
+        switch (kind) {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(s1) && PyUnicode_IS_ASCII(s2))
                 result = asciilib_rfind_slice(buf1, len1, buf2, len2, start, end);
@@ -9080,7 +9079,7 @@
                                    const char *grouping,
                                    const char *thousands_sep)
 {
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         if (unicode != NULL && PyUnicode_IS_ASCII(unicode))
             return _PyUnicode_ascii_InsertThousandsGrouping(
@@ -9158,7 +9157,7 @@
     len2 = PyUnicode_GET_LENGTH(sub_obj);
 
     ADJUST_INDICES(start, end, len1);
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(str_obj) && PyUnicode_IS_ASCII(sub_obj))
             result = asciilib_count(
@@ -9860,7 +9859,7 @@
     if (string == NULL || PyUnicode_READY(string) == -1)
         return NULL;
 
-    switch(PyUnicode_KIND(string)) {
+    switch (PyUnicode_KIND(string)) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(string))
             list = asciilib_splitlines(
@@ -9906,7 +9905,7 @@
         return NULL;
 
     if (substring == NULL)
-        switch(PyUnicode_KIND(self)) {
+        switch (PyUnicode_KIND(self)) {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(self))
                 return asciilib_split_whitespace(
@@ -9954,7 +9953,7 @@
     len1 = PyUnicode_GET_LENGTH(self);
     len2 = PyUnicode_GET_LENGTH(substring);
 
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(self) && PyUnicode_IS_ASCII(substring))
             out = asciilib_split(
@@ -9998,7 +9997,7 @@
         return NULL;
 
     if (substring == NULL)
-        switch(PyUnicode_KIND(self)) {
+        switch (PyUnicode_KIND(self)) {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(self))
                 return asciilib_rsplit_whitespace(
@@ -10046,7 +10045,7 @@
     len1 = PyUnicode_GET_LENGTH(self);
     len2 = PyUnicode_GET_LENGTH(substring);
 
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(self) && PyUnicode_IS_ASCII(substring))
             out = asciilib_rsplit(
@@ -10077,7 +10076,7 @@
 anylib_find(int kind, PyObject *str1, void *buf1, Py_ssize_t len1,
             PyObject *str2, void *buf2, Py_ssize_t len2, Py_ssize_t offset)
 {
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(str1) && PyUnicode_IS_ASCII(str2))
             return asciilib_find(buf1, len1, buf2, len2, offset);
@@ -10096,7 +10095,7 @@
 anylib_count(int kind, PyObject *sstr, void* sbuf, Py_ssize_t slen,
              PyObject *str1, void *buf1, Py_ssize_t len1, Py_ssize_t maxcount)
 {
-        switch(kind) {
+        switch (kind) {
         case PyUnicode_1BYTE_KIND:
             if (PyUnicode_IS_ASCII(sstr) && PyUnicode_IS_ASCII(str1))
                 return asciilib_count(sbuf, slen, buf1, len1, maxcount);
@@ -10680,7 +10679,7 @@
     len1 = PyUnicode_GET_LENGTH(str);
     len2 = PyUnicode_GET_LENGTH(sub);
 
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         result = ucs1lib_find(buf1, len1, buf2, len2, 0) != -1;
         break;
@@ -10900,7 +10899,7 @@
     len2 = PyUnicode_GET_LENGTH(substring);
 
     ADJUST_INDICES(start, end, len1);
-    switch(kind) {
+    switch (kind) {
     case PyUnicode_1BYTE_KIND:
         iresult = ucs1lib_count(
             ((Py_UCS1*)buf1) + start, end - start,
@@ -12323,7 +12322,7 @@
     len1 = PyUnicode_GET_LENGTH(str_obj);
     len2 = PyUnicode_GET_LENGTH(sep_obj);
 
-    switch(PyUnicode_KIND(str_obj)) {
+    switch (PyUnicode_KIND(str_obj)) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(str_obj) && PyUnicode_IS_ASCII(sep_obj))
             out = asciilib_partition(str_obj, buf1, len1, sep_obj, buf2, len2);
@@ -12395,7 +12394,7 @@
     len1 = PyUnicode_GET_LENGTH(str_obj);
     len2 = PyUnicode_GET_LENGTH(sep_obj);
 
-    switch(PyUnicode_KIND(str_in)) {
+    switch (PyUnicode_KIND(str_in)) {
     case PyUnicode_1BYTE_KIND:
         if (PyUnicode_IS_ASCII(str_obj) && PyUnicode_IS_ASCII(sep_obj))
             out = asciilib_rpartition(str_obj, buf1, len1, sep_obj, buf2, len2);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 18:22:35 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 21 Dec 2011 18:22:35 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_4_space_indentation?=
Message-ID: <E1RdPsB-00015P-Je@dinsdale.python.org>

http://hg.python.org/cpython/rev/56b70a81eee7
changeset:   74119:56b70a81eee7
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 20 17:24:05 2011 -0600
summary:
  4 space indentation

files:
  Objects/unicodeobject.c |  26 +++++++++++++-------------
  1 files changed, 13 insertions(+), 13 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10095,19 +10095,19 @@
 anylib_count(int kind, PyObject *sstr, void* sbuf, Py_ssize_t slen,
              PyObject *str1, void *buf1, Py_ssize_t len1, Py_ssize_t maxcount)
 {
-        switch (kind) {
-        case PyUnicode_1BYTE_KIND:
-            if (PyUnicode_IS_ASCII(sstr) && PyUnicode_IS_ASCII(str1))
-                return asciilib_count(sbuf, slen, buf1, len1, maxcount);
-            else
-                return ucs1lib_count(sbuf, slen, buf1, len1, maxcount);
-        case PyUnicode_2BYTE_KIND:
-            return ucs2lib_count(sbuf, slen, buf1, len1, maxcount);
-        case PyUnicode_4BYTE_KIND:
-            return ucs4lib_count(sbuf, slen, buf1, len1, maxcount);
-        }
-        assert(0);
-        return 0;
+    switch (kind) {
+    case PyUnicode_1BYTE_KIND:
+        if (PyUnicode_IS_ASCII(sstr) && PyUnicode_IS_ASCII(str1))
+            return asciilib_count(sbuf, slen, buf1, len1, maxcount);
+        else
+            return ucs1lib_count(sbuf, slen, buf1, len1, maxcount);
+    case PyUnicode_2BYTE_KIND:
+        return ucs2lib_count(sbuf, slen, buf1, len1, maxcount);
+    case PyUnicode_4BYTE_KIND:
+        return ucs4lib_count(sbuf, slen, buf1, len1, maxcount);
+    }
+    assert(0);
+    return 0;
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 18:22:36 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 21 Dec 2011 18:22:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RdPsC-00015v-G4@dinsdale.python.org>

http://hg.python.org/cpython/rev/9c897817abde
changeset:   74120:9c897817abde
parent:      74119:56b70a81eee7
parent:      74116:f3041e7f535d
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Dec 21 11:22:02 2011 -0600
summary:
  merge heads

files:
  Doc/library/multiprocessing.rst  |   18 ++
  Doc/library/ssl.rst              |   10 +
  Doc/library/webbrowser.rst       |   90 +++++++------
  Doc/using/cmdline.rst            |    3 +-
  Doc/whatsnew/3.0.rst             |   10 +-
  Lib/inspect.py                   |   79 ++++++----
  Lib/multiprocessing/managers.py  |    3 +-
  Lib/multiprocessing/pool.py      |   32 ++++-
  Lib/pydoc.py                     |   29 +++-
  Lib/ssl.py                       |    2 +-
  Lib/test/test_import.py          |    3 +-
  Lib/test/test_inspect.py         |   79 +++++++++++
  Lib/test/test_multiprocessing.py |   18 ++
  Lib/test/test_ssl.py             |  128 ++++++------------
  Lib/webbrowser.py                |   17 ++
  Misc/ACKS                        |    1 +
  Misc/NEWS                        |    8 +
  Modules/_ssl.c                   |   12 +
  18 files changed, 371 insertions(+), 171 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -1669,6 +1669,24 @@
       returned iterator should be considered arbitrary.  (Only when there is
       only one worker process is the order guaranteed to be "correct".)
 
+   .. method:: starmap(func, iterable[, chunksize])
+
+      Like :meth:`map` except that the elements of the `iterable` are expected
+      to be iterables that are unpacked as arguments.
+
+      Hence an `iterable` of `[(1,2), (3, 4)]` results in `[func(1,2),
+      func(3,4)]`.
+
+      .. versionadded:: 3.3
+
+   .. method:: starmap_async(func, iterable[, chunksize[, callback[, error_back]]])
+
+      A combination of :meth:`starmap` and :meth:`map_async` that iterates over
+      `iterable` of iterables and calls `func` with the iterables unpacked.
+      Returns a result object.
+
+      .. versionadded:: 3.3
+
    .. method:: close()
 
       Prevents any more tasks from being submitted to the pool.  Once all the
diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -445,6 +445,14 @@
 
    .. versionadded:: 3.3
 
+.. data:: HAS_ECDH
+
+   Whether the OpenSSL library has built-in support for Elliptic Curve-based
+   Diffie-Hellman key exchange.  This should be true unless the feature was
+   explicitly disabled by the distributor.
+
+   .. versionadded:: 3.3
+
 .. data:: HAS_SNI
 
    Whether the OpenSSL library has built-in support for the *Server Name
@@ -711,6 +719,8 @@
    This setting doesn't apply to client sockets.  You can also use the
    :data:`OP_SINGLE_ECDH_USE` option to further improve security.
 
+   This method is not available if :data:`HAS_ECDH` is False.
+
    .. versionadded:: 3.3
 
    .. seealso::
diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst
--- a/Doc/library/webbrowser.rst
+++ b/Doc/library/webbrowser.rst
@@ -96,47 +96,55 @@
 may be passed to the :func:`get` function and the corresponding instantiations
 for the controller classes, all defined in this module.
 
-+-----------------------+-----------------------------------------+-------+
-| Type Name             | Class Name                              | Notes |
-+=======================+=========================================+=======+
-| ``'mozilla'``         | :class:`Mozilla('mozilla')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'firefox'``         | :class:`Mozilla('mozilla')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'netscape'``        | :class:`Mozilla('netscape')`            |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'galeon'``          | :class:`Galeon('galeon')`               |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'epiphany'``        | :class:`Galeon('epiphany')`             |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'skipstone'``       | :class:`BackgroundBrowser('skipstone')` |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'kfmclient'``       | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'konqueror'``       | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'kfm'``             | :class:`Konqueror()`                    | \(1)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'mosaic'``          | :class:`BackgroundBrowser('mosaic')`    |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'opera'``           | :class:`Opera()`                        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'grail'``           | :class:`Grail()`                        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'links'``           | :class:`GenericBrowser('links')`        |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'elinks'``          | :class:`Elinks('elinks')`               |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'lynx'``            | :class:`GenericBrowser('lynx')`         |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'w3m'``             | :class:`GenericBrowser('w3m')`          |       |
-+-----------------------+-----------------------------------------+-------+
-| ``'windows-default'`` | :class:`WindowsDefault`                 | \(2)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'internet-config'`` | :class:`InternetConfig`                 | \(3)  |
-+-----------------------+-----------------------------------------+-------+
-| ``'macosx'``          | :class:`MacOSX('default')`              | \(4)  |
-+-----------------------+-----------------------------------------+-------+
++------------------------+-----------------------------------------+-------+
+| Type Name              | Class Name                              | Notes |
++========================+=========================================+=======+
+| ``'mozilla'``          | :class:`Mozilla('mozilla')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'firefox'``          | :class:`Mozilla('mozilla')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'netscape'``         | :class:`Mozilla('netscape')`            |       |
++------------------------+-----------------------------------------+-------+
+| ``'galeon'``           | :class:`Galeon('galeon')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'epiphany'``         | :class:`Galeon('epiphany')`             |       |
++------------------------+-----------------------------------------+-------+
+| ``'skipstone'``        | :class:`BackgroundBrowser('skipstone')` |       |
++------------------------+-----------------------------------------+-------+
+| ``'kfmclient'``        | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'konqueror'``        | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'kfm'``              | :class:`Konqueror()`                    | \(1)  |
++------------------------+-----------------------------------------+-------+
+| ``'mosaic'``           | :class:`BackgroundBrowser('mosaic')`    |       |
++------------------------+-----------------------------------------+-------+
+| ``'opera'``            | :class:`Opera()`                        |       |
++------------------------+-----------------------------------------+-------+
+| ``'grail'``            | :class:`Grail()`                        |       |
++------------------------+-----------------------------------------+-------+
+| ``'links'``            | :class:`GenericBrowser('links')`        |       |
++------------------------+-----------------------------------------+-------+
+| ``'elinks'``           | :class:`Elinks('elinks')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'lynx'``             | :class:`GenericBrowser('lynx')`         |       |
++------------------------+-----------------------------------------+-------+
+| ``'w3m'``              | :class:`GenericBrowser('w3m')`          |       |
++------------------------+-----------------------------------------+-------+
+| ``'windows-default'``  | :class:`WindowsDefault`                 | \(2)  |
++------------------------+-----------------------------------------+-------+
+| ``'internet-config'``  | :class:`InternetConfig`                 | \(3)  |
++------------------------+-----------------------------------------+-------+
+| ``'macosx'``           | :class:`MacOSX('default')`              | \(4)  |
++------------------------+-----------------------------------------+-------+
+| ``'google-chrome'``    | :class:`Chrome('google-chrome')`        |       |
++------------------------+-----------------------------------------+-------+
+| ``'chrome'``           | :class:`Chrome('chrome')`               |       |
++------------------------+-----------------------------------------+-------+
+| ``'chromium'``         | :class:`Chromium('chromium')`           |       |
++------------------------+-----------------------------------------+-------+
+| ``'chromium-browser'`` | :class:`Chromium('chromium-browser')`   |       |
++------------------------+-----------------------------------------+-------+
 
 Notes:
 
diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -249,7 +249,8 @@
 
    Force the binary layer of the stdin, stdout and stderr streams (which is
    available as their ``buffer`` attribute) to be unbuffered.  The text I/O
-   layer will still be line-buffered.
+   layer will still be line-buffered if writing to the console, or
+   block-buffered if redirected to a non-interactive file.
 
    See also :envvar:`PYTHONUNBUFFERED`.
 
diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst
--- a/Doc/whatsnew/3.0.rst
+++ b/Doc/whatsnew/3.0.rst
@@ -301,6 +301,12 @@
   There is no longer any need for using the encoding-aware streams
   in the :mod:`codecs` module.
 
+* The initial values of :data:`sys.stdin`, :data:`sys.stdout` and
+  :data:`sys.stderr` are now unicode-only text files (i.e., they are
+  instances of :class:`io.TextIOBase`).  To read and write bytes data
+  with these streams, you need to use their :data:`io.TextIOBase.buffer`
+  attribute.
+
 * Filenames are passed to and returned from APIs as (Unicode) strings.
   This can present platform-specific problems because on some
   platforms filenames are arbitrary byte strings.  (On the other hand,
@@ -511,9 +517,7 @@
   produces a literal of type :class:`bytes`.
 
 * :ref:`pep-3116`.  The :mod:`io` module is now the standard way of
-  doing file I/O, and the initial values of :data:`sys.stdin`,
-  :data:`sys.stdout` and :data:`sys.stderr` are now instances of
-  :class:`io.TextIOBase`.  The built-in :func:`open` function is now an
+  doing file I/O.  The built-in :func:`open` function is now an
   alias for :func:`io.open` and has additional keyword arguments
   *encoding*, *errors*, *newline* and *closefd*.  Also note that an
   invalid *mode* argument now raises :exc:`ValueError`, not
diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -99,11 +99,11 @@
     tests return false from the ismethoddescriptor() test, simply because
     the other tests promise more -- you can, e.g., count on having the
     __func__ attribute (etc) when an object passes ismethod()."""
-    return (hasattr(object, "__get__")
-            and not hasattr(object, "__set__") # else it's a data descriptor
-            and not ismethod(object)           # mutual exclusion
-            and not isfunction(object)
-            and not isclass(object))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__get__") and not hasattr(tp, "__set__")
 
 def isdatadescriptor(object):
     """Return true if the object is a data descriptor.
@@ -113,7 +113,11 @@
     Typically, data descriptors will also have __name__ and __doc__ attributes
     (properties, getsets, and members have both of these attributes), but this
     is not guaranteed."""
-    return (hasattr(object, "__set__") and hasattr(object, "__get__"))
+    if isclass(object) or ismethod(object) or isfunction(object):
+        # mutual exclusion
+        return False
+    tp = type(object)
+    return hasattr(tp, "__set__") and hasattr(tp, "__get__")
 
 if hasattr(types, 'MemberDescriptorType'):
     # CPython and equivalent
@@ -253,12 +257,23 @@
 def getmembers(object, predicate=None):
     """Return all members of an object as (name, value) pairs sorted by name.
     Optionally, only return members that satisfy a given predicate."""
+    if isclass(object):
+        mro = (object,) + getmro(object)
+    else:
+        mro = ()
     results = []
     for key in dir(object):
-        try:
-            value = getattr(object, key)
-        except AttributeError:
-            continue
+        # First try to get the value via __dict__. Some descriptors don't
+        # like calling their __get__ (see bug #1785).
+        for base in mro:
+            if key in base.__dict__:
+                value = base.__dict__[key]
+                break
+        else:
+            try:
+                value = getattr(object, key)
+            except AttributeError:
+                continue
         if not predicate or predicate(value):
             results.append((key, value))
     results.sort()
@@ -294,30 +309,21 @@
     names = dir(cls)
     result = []
     for name in names:
-        # Get the object associated with the name.
+        # Get the object associated with the name, and where it was defined.
         # Getting an obj from the __dict__ sometimes reveals more than
         # using getattr.  Static and class methods are dramatic examples.
-        if name in cls.__dict__:
-            obj = cls.__dict__[name]
+        # Furthermore, some objects may raise an Exception when fetched with
+        # getattr(). This is the case with some descriptors (bug #1785).
+        # Thus, we only use getattr() as a last resort.
+        homecls = None
+        for base in (cls,) + mro:
+            if name in base.__dict__:
+                obj = base.__dict__[name]
+                homecls = base
+                break
         else:
             obj = getattr(cls, name)
-
-        # Figure out where it was defined.
-        homecls = getattr(obj, "__objclass__", None)
-        if homecls is None:
-            # search the dicts.
-            for base in mro:
-                if name in base.__dict__:
-                    homecls = base
-                    break
-
-        # Get the object again, in order to get it from the defining
-        # __dict__ instead of via getattr (if possible).
-        if homecls is not None and name in homecls.__dict__:
-            obj = homecls.__dict__[name]
-
-        # Also get the object via getattr.
-        obj_via_getattr = getattr(cls, name)
+            homecls = getattr(obj, "__objclass__", homecls)
 
         # Classify the object.
         if isinstance(obj, staticmethod):
@@ -326,11 +332,18 @@
             kind = "class method"
         elif isinstance(obj, property):
             kind = "property"
-        elif (isfunction(obj_via_getattr) or
-              ismethoddescriptor(obj_via_getattr)):
+        elif ismethoddescriptor(obj):
             kind = "method"
+        elif isdatadescriptor(obj):
+            kind = "data"
         else:
-            kind = "data"
+            obj_via_getattr = getattr(cls, name)
+            if (isfunction(obj_via_getattr) or
+                ismethoddescriptor(obj_via_getattr)):
+                kind = "method"
+            else:
+                kind = "data"
+            obj = obj_via_getattr
 
         result.append(Attribute(name, kind, homecls, obj))
 
diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -1066,11 +1066,12 @@
 
 PoolProxy = MakeProxyType('PoolProxy', (
     'apply', 'apply_async', 'close', 'imap', 'imap_unordered', 'join',
-    'map', 'map_async', 'terminate'
+    'map', 'map_async', 'starmap', 'starmap_async', 'terminate'
     ))
 PoolProxy._method_to_typeid_ = {
     'apply_async': 'AsyncResult',
     'map_async': 'AsyncResult',
+    'starmap_async': 'AsyncResult',
     'imap': 'Iterator',
     'imap_unordered': 'Iterator'
     }
diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py
--- a/Lib/multiprocessing/pool.py
+++ b/Lib/multiprocessing/pool.py
@@ -64,6 +64,9 @@
 def mapstar(args):
     return list(map(*args))
 
+def starmapstar(args):
+    return list(itertools.starmap(args[0], args[1]))
+
 #
 # Code run by worker processes
 #
@@ -248,7 +251,25 @@
         in a list that is returned.
         '''
         assert self._state == RUN
-        return self.map_async(func, iterable, chunksize).get()
+        return self._map_async(func, iterable, mapstar, chunksize).get()
+
+    def starmap(self, func, iterable, chunksize=None):
+        '''
+        Like `map()` method but the elements of the `iterable` are expected to
+        be iterables as well and will be unpacked as arguments. Hence
+        `func` and (a, b) becomes func(a, b).
+        '''
+        assert self._state == RUN
+        return self._map_async(func, iterable, starmapstar, chunksize).get()
+
+    def starmap_async(self, func, iterable, chunksize=None, callback=None,
+            error_callback=None):
+        '''
+        Asynchronous version of `starmap()` method.
+        '''
+        assert self._state == RUN
+        return self._map_async(func, iterable, starmapstar, chunksize,
+                               callback, error_callback)
 
     def imap(self, func, iterable, chunksize=1):
         '''
@@ -302,6 +323,13 @@
         Asynchronous version of `map()` method.
         '''
         assert self._state == RUN
+        return self._map_async(func, iterable, mapstar, chunksize)
+
+    def _map_async(self, func, iterable, mapper, chunksize=None, callback=None,
+            error_callback=None):
+        '''
+        Helper function to implement map, starmap and their async counterparts.
+        '''
         if not hasattr(iterable, '__len__'):
             iterable = list(iterable)
 
@@ -315,7 +343,7 @@
         task_batches = Pool._get_tasks(func, iterable, chunksize)
         result = MapResult(self._cache, chunksize, len(iterable), callback,
                            error_callback=error_callback)
-        self._taskqueue.put((((result._job, i, mapstar, (x,), {})
+        self._taskqueue.put((((result._job, i, mapper, (x,), {})
                               for i, x in enumerate(task_batches)), None))
         return result
 
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -748,8 +748,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name), name, mod,
-                                       funcs, classes, mdict, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value, name, mod,
+                                        funcs, classes, mdict, object))
                     push('\n')
             return attrs
 
@@ -790,7 +797,12 @@
         mdict = {}
         for key, kind, homecls, value in attrs:
             mdict[key] = anchor = '#' + name + '-' + key
-            value = getattr(object, key)
+            try:
+                value = getattr(object, name)
+            except Exception:
+                # Some descriptors may meet a failure in their __get__.
+                # (bug #1785)
+                pass
             try:
                 # The value may not be hashable (e.g., a data attr with
                 # a dict or list value).
@@ -1177,8 +1189,15 @@
                 hr.maybe()
                 push(msg)
                 for name, kind, homecls, value in ok:
-                    push(self.document(getattr(object, name),
-                                       name, mod, object))
+                    try:
+                        value = getattr(object, name)
+                    except Exception:
+                        # Some descriptors may meet a failure in their __get__.
+                        # (bug #1785)
+                        push(self._docdescriptor(name, value, mod))
+                    else:
+                        push(self.document(value,
+                                        name, mod, object))
             return attrs
 
         def spilldescriptors(msg, attrs, predicate):
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -86,7 +86,7 @@
     SSL_ERROR_EOF,
     SSL_ERROR_INVALID_ERROR_CODE,
     )
-from _ssl import HAS_SNI
+from _ssl import HAS_SNI, HAS_ECDH
 from _ssl import (PROTOCOL_SSLv3, PROTOCOL_SSLv23,
                   PROTOCOL_TLSv1)
 from _ssl import _OPENSSL_API_VERSION
diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -104,8 +104,9 @@
             try:
                 fname = TESTFN + os.extsep + "py"
                 create_empty_file(fname)
+                fn = imp.cache_from_source(fname)
+                unlink(fn)
                 __import__(TESTFN)
-                fn = imp.cache_from_source(fname)
                 if not os.path.exists(fn):
                     self.fail("__import__ did not result in creation of "
                               "either a .pyc or .pyo file")
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -425,10 +425,37 @@
     def test_class(self):
         self.assertSourceEqual(self.fodderModule.X, 1, 2)
 
+
+class _BrokenDataDescriptor(object):
+    """
+    A broken data descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ data descriptors")
+
+    def __set__(*args):
+        raise RuntimeError
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ data descriptors")
+
+
+class _BrokenMethodDescriptor(object):
+    """
+    A broken method descriptor. See bug #1785.
+    """
+    def __get__(*args):
+        raise AssertionError("should not __get__ method descriptors")
+
+    def __getattr__(*args):
+        raise AssertionError("should not __getattr__ method descriptors")
+
+
 # Helper for testing classify_class_attrs.
 def attrs_wo_objs(cls):
     return [t[:3] for t in inspect.classify_class_attrs(cls)]
 
+
 class TestClassesAndFunctions(unittest.TestCase):
     def test_newstyle_mro(self):
         # The same w/ new-class MRO.
@@ -525,6 +552,9 @@
 
             datablob = '1'
 
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
         attrs = attrs_wo_objs(A)
         self.assertIn(('s', 'static method', A), attrs, 'missing static method')
         self.assertIn(('c', 'class method', A), attrs, 'missing class method')
@@ -533,6 +563,8 @@
                       'missing plain method: %r' % attrs)
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class B(A):
 
@@ -545,6 +577,8 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
 
         class C(A):
@@ -559,6 +593,8 @@
         self.assertIn(('m', 'method', C), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', A), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
 
         class D(B, C):
 
@@ -571,6 +607,49 @@
         self.assertIn(('m', 'method', B), attrs, 'missing plain method')
         self.assertIn(('m1', 'method', D), attrs, 'missing plain method')
         self.assertIn(('datablob', 'data', A), attrs, 'missing data')
+        self.assertIn(('md', 'method', A), attrs, 'missing method descriptor')
+        self.assertIn(('dd', 'data', A), attrs, 'missing data descriptor')
+
+    def test_classify_builtin_types(self):
+        # Simple sanity check that all built-in types can have their
+        # attributes classified.
+        for name in dir(__builtins__):
+            builtin = getattr(__builtins__, name)
+            if isinstance(builtin, type):
+                inspect.classify_class_attrs(builtin)
+
+    def test_getmembers_descriptors(self):
+        class A(object):
+            dd = _BrokenDataDescriptor()
+            md = _BrokenMethodDescriptor()
+
+        def pred_wrapper(pred):
+            # A quick'n'dirty way to discard standard attributes of new-style
+            # classes.
+            class Empty(object):
+                pass
+            def wrapped(x):
+                if '__name__' in dir(x) and hasattr(Empty, x.__name__):
+                    return False
+                return pred(x)
+            return wrapped
+
+        ismethoddescriptor = pred_wrapper(inspect.ismethoddescriptor)
+        isdatadescriptor = pred_wrapper(inspect.isdatadescriptor)
+
+        self.assertEqual(inspect.getmembers(A, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(A, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
+        class B(A):
+            pass
+
+        self.assertEqual(inspect.getmembers(B, ismethoddescriptor),
+            [('md', A.__dict__['md'])])
+        self.assertEqual(inspect.getmembers(B, isdatadescriptor),
+            [('dd', A.__dict__['dd'])])
+
 
 class TestGetcallargsFunctions(unittest.TestCase):
 
diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py
--- a/Lib/test/test_multiprocessing.py
+++ b/Lib/test/test_multiprocessing.py
@@ -8,6 +8,7 @@
 import queue as pyqueue
 import time
 import io
+import itertools
 import sys
 import os
 import gc
@@ -1125,6 +1126,9 @@
     time.sleep(wait)
     return x*x
 
+def mul(x, y):
+    return x*y
+
 class _TestPool(BaseTestCase):
 
     def test_apply(self):
@@ -1138,6 +1142,20 @@
         self.assertEqual(pmap(sqr, list(range(100)), chunksize=20),
                          list(map(sqr, list(range(100)))))
 
+    def test_starmap(self):
+        psmap = self.pool.starmap
+        tuples = list(zip(range(10), range(9,-1, -1)))
+        self.assertEqual(psmap(mul, tuples),
+                         list(itertools.starmap(mul, tuples)))
+        tuples = list(zip(range(100), range(99,-1, -1)))
+        self.assertEqual(psmap(mul, tuples, chunksize=20),
+                         list(itertools.starmap(mul, tuples)))
+
+    def test_starmap_async(self):
+        tuples = list(zip(range(100), range(99,-1, -1)))
+        self.assertEqual(self.pool.starmap_async(mul, tuples).get(),
+                         list(itertools.starmap(mul, tuples)))
+
     def test_map_chunksize(self):
         try:
             self.pool.map_async(sqr, [], chunksize=1).get(timeout=TIMEOUT1)
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -103,6 +103,7 @@
         if ssl.OPENSSL_VERSION_INFO >= (1, 0):
             ssl.OP_NO_COMPRESSION
         self.assertIn(ssl.HAS_SNI, {True, False})
+        self.assertIn(ssl.HAS_ECDH, {True, False})
 
     def test_random(self):
         v = ssl.RAND_status()
@@ -561,6 +562,7 @@
         ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
         ctx.set_default_verify_paths()
 
+    @unittest.skipUnless(ssl.HAS_ECDH, "ECDH disabled on this OpenSSL build")
     def test_set_ecdh_curve(self):
         ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
         ctx.set_ecdh_curve("prime256v1")
@@ -984,6 +986,14 @@
             threading.Thread.__init__(self)
             self.daemon = True
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            self.stop()
+            self.join()
+
         def start(self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -1095,6 +1105,20 @@
         def __str__(self):
             return "<%s %s>" % (self.__class__.__name__, self.server)
 
+        def __enter__(self):
+            self.start(threading.Event())
+            self.flag.wait()
+
+        def __exit__(self, *args):
+            if support.verbose:
+                sys.stdout.write(" cleanup: stopping server.\n")
+            self.stop()
+            if support.verbose:
+                sys.stdout.write(" cleanup: joining server thread.\n")
+            self.join()
+            if support.verbose:
+                sys.stdout.write(" cleanup: successfully joined.\n")
+
         def start (self, flag=None):
             self.flag = flag
             threading.Thread.start(self)
@@ -1122,12 +1146,7 @@
                                     certreqs=ssl.CERT_REQUIRED,
                                     cacerts=CERTFILE, chatty=False,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             try:
                 with socket.socket() as sock:
                     s = ssl.wrap_socket(sock,
@@ -1147,9 +1166,6 @@
                     sys.stdout.write("\IOError is %s\n" % str(x))
             else:
                 raise AssertionError("Use of invalid cert should have failed!")
-        finally:
-            server.stop()
-            server.join()
 
     def server_params_test(client_context, server_context, indata=b"FOO\n",
                            chatty=True, connectionchatty=False):
@@ -1160,12 +1176,7 @@
         server = ThreadedEchoServer(context=server_context,
                                     chatty=chatty,
                                     connectionchatty=False)
-        flag = threading.Event()
-        server.start(flag)
-        # wait for it to start
-        flag.wait()
-        # try to connect
-        try:
+        with server:
             s = client_context.wrap_socket(socket.socket())
             s.connect((HOST, server.port))
             for arg in [indata, bytearray(indata), memoryview(indata)]:
@@ -1193,9 +1204,6 @@
             }
             s.close()
             return stats
-        finally:
-            server.stop()
-            server.join()
 
     def try_protocol_combo(server_protocol, client_protocol, expect_success,
                            certsreqs=None, server_options=0, client_options=0):
@@ -1264,12 +1272,7 @@
             context.load_verify_locations(CERTFILE)
             context.load_cert_chain(CERTFILE)
             server = ThreadedEchoServer(context=context, chatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = context.wrap_socket(socket.socket())
                 s.connect((HOST, server.port))
                 cert = s.getpeercert()
@@ -1292,9 +1295,6 @@
                 after = ssl.cert_time_to_seconds(cert['notAfter'])
                 self.assertLess(before, after)
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_empty_cert(self):
             """Connecting with an empty cert file"""
@@ -1454,13 +1454,8 @@
                                         starttls_server=True,
                                         chatty=True,
                                         connectionchatty=True)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
             wrapped = False
-            try:
+            with server:
                 s = socket.socket()
                 s.setblocking(1)
                 s.connect((HOST, server.port))
@@ -1507,9 +1502,6 @@
                     conn.close()
                 else:
                     s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_socketserver(self):
             """Using a SocketServer to create and manage SSL connections."""
@@ -1545,12 +1537,7 @@
 
             indata = b"FOO\n"
             server = AsyncoreEchoServer(CERTFILE)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            try:
+            with server:
                 s = ssl.wrap_socket(socket.socket())
                 s.connect(('127.0.0.1', server.port))
                 if support.verbose:
@@ -1571,15 +1558,6 @@
                 s.close()
                 if support.verbose:
                     sys.stdout.write(" client:  connection closed.\n")
-            finally:
-                if support.verbose:
-                    sys.stdout.write(" cleanup: stopping server.\n")
-                server.stop()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: joining server thread.\n")
-                server.join()
-                if support.verbose:
-                    sys.stdout.write(" cleanup: successfully joined.\n")
 
         def test_recv_send(self):
             """Test recv(), send() and friends."""
@@ -1592,19 +1570,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # helper methods for standardising recv* method signatures
                 def _recv_into():
                     b = bytearray(b"\0"*100)
@@ -1700,9 +1673,6 @@
 
                 s.write(b"over\n")
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_handshake_timeout(self):
             # Issue #5103: SSL handshake must respect the socket timeout
@@ -1766,19 +1736,14 @@
                                         cacerts=CERTFILE,
                                         chatty=True,
                                         connectionchatty=False)
-            flag = threading.Event()
-            server.start(flag)
-            # wait for it to start
-            flag.wait()
-            # try to connect
-            s = ssl.wrap_socket(socket.socket(),
-                                server_side=False,
-                                certfile=CERTFILE,
-                                ca_certs=CERTFILE,
-                                cert_reqs=ssl.CERT_NONE,
-                                ssl_version=ssl.PROTOCOL_TLSv1)
-            s.connect((HOST, server.port))
-            try:
+            with server:
+                s = ssl.wrap_socket(socket.socket(),
+                                    server_side=False,
+                                    certfile=CERTFILE,
+                                    ca_certs=CERTFILE,
+                                    cert_reqs=ssl.CERT_NONE,
+                                    ssl_version=ssl.PROTOCOL_TLSv1)
+                s.connect((HOST, server.port))
                 # get the data
                 cb_data = s.get_channel_binding("tls-unique")
                 if support.verbose:
@@ -1817,9 +1782,6 @@
                 self.assertEqual(peer_data_repr,
                                  repr(new_cb_data).encode("us-ascii"))
                 s.close()
-            finally:
-                server.stop()
-                server.join()
 
         def test_compression(self):
             context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py
--- a/Lib/webbrowser.py
+++ b/Lib/webbrowser.py
@@ -299,6 +299,18 @@
     background = True
 
 
+class Chrome(UnixBrowser):
+    "Launcher class for Google Chrome browser."
+
+    remote_args = ['%action', '%s']
+    remote_action = ""
+    remote_action_newwin = "--new-window"
+    remote_action_newtab = ""
+    background = True
+
+Chromium = Chrome
+
+
 class Opera(UnixBrowser):
     "Launcher class for Opera browser."
 
@@ -466,6 +478,11 @@
     if _iscommand("skipstone"):
         register("skipstone", None, BackgroundBrowser("skipstone"))
 
+    # Google Chrome/Chromium browsers
+    for browser in ("google-chrome", "chrome", "chromium", "chromium-browser"):
+        if _iscommand(browser):
+            register(browser, None, Chrome(browser))
+
     # Opera, quite popular
     if _iscommand("opera"):
         register("opera", None, Opera("opera"))
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -878,6 +878,7 @@
 Andreas Schawo
 Neil Schemenauer
 David Scherer
+Hynek Schlawack
 Bob Schmertz
 Gregor Schmid
 Ralf Schmitt
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,14 @@
 Library
 -------
 
+- Issue #13620: Support for Chrome browser in webbrowser.py Patch contributed
+  by Arnaud Calmettes.
+
+- Issue #12708: Add starmap() and starmap_async() methods (similar to
+  itertools.starmap()) to multiprocessing.Pool.  Patch by Hynek Schlawack.
+
+- Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
+
 - Issue #13637: "a2b" functions in the binascii module now accept ASCII-only
   unicode strings.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2006,6 +2006,7 @@
     Py_RETURN_NONE;
 }
 
+#ifndef OPENSSL_NO_ECDH
 static PyObject *
 set_ecdh_curve(PySSLContext *self, PyObject *name)
 {
@@ -2032,6 +2033,7 @@
     EC_KEY_free(key);
     Py_RETURN_NONE;
 }
+#endif
 
 static PyGetSetDef context_getsetlist[] = {
     {"options", (getter) get_options,
@@ -2054,8 +2056,10 @@
                       METH_NOARGS, NULL},
     {"set_default_verify_paths", (PyCFunction) set_default_verify_paths,
                                  METH_NOARGS, NULL},
+#ifndef OPENSSL_NO_ECDH
     {"set_ecdh_curve", (PyCFunction) set_ecdh_curve,
                        METH_O, NULL},
+#endif
     {NULL, NULL}        /* sentinel */
 };
 
@@ -2523,6 +2527,14 @@
     Py_INCREF(r);
     PyModule_AddObject(m, "HAS_TLS_UNIQUE", r);
 
+#ifdef OPENSSL_NO_ECDH
+    r = Py_False;
+#else
+    r = Py_True;
+#endif
+    Py_INCREF(r);
+    PyModule_AddObject(m, "HAS_ECDH", r);
+
     /* OpenSSL version */
     /* SSLeay() gives us the version of the library linked against,
        which could be different from the headers version.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 21 19:28:21 2011
From: python-checkins at python.org (lars.gustaebel)
Date: Wed, 21 Dec 2011 19:28:21 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNjM5?=
 =?utf8?q?=3A_Accept_unicode_filenames_in_tarfile=2Eopen=28mode=3D=22w=7Cg?=
 =?utf8?b?eiIpLg==?=
Message-ID: <E1RdQtp-0006zR-R6@dinsdale.python.org>

http://hg.python.org/cpython/rev/a60a3610a97b
changeset:   74121:a60a3610a97b
branch:      2.7
parent:      74117:d868208b831d
user:        Lars Gust?bel <lars at gustaebel.de>
date:        Wed Dec 21 19:27:50 2011 +0100
summary:
  Issue #13639: Accept unicode filenames in tarfile.open(mode="w|gz").

Passing a unicode filename to tarfile.open() along with mode "w|gz" failed
with a UnicodeError because the filename was not encoded properly before being
written to the gzipped stream in the FNAME extra field.

files:
  Lib/tarfile.py           |  2 ++
  Lib/test/test_tarfile.py |  7 +++++++
  Misc/NEWS                |  2 ++
  3 files changed, 11 insertions(+), 0 deletions(-)


diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -454,6 +454,8 @@
                                             0)
         timestamp = struct.pack("<L", long(time.time()))
         self.__write("\037\213\010\010%s\002\377" % timestamp)
+        if type(self.name) is unicode:
+            self.name = self.name.encode("iso-8859-1", "replace")
         if self.name.endswith(".gz"):
             self.name = self.name[:-3]
         self.__write(self.name + NUL)
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -994,6 +994,13 @@
         finally:
             os.umask(original_umask)
 
+    def test_issue13639(self):
+        try:
+            with tarfile.open(unicode(tmpname, sys.getfilesystemencoding()), self.mode):
+                pass
+        except UnicodeDecodeError:
+            self.fail("_Stream failed to write unicode filename")
+
 
 class GNUWriteTest(unittest.TestCase):
     # This testcase checks for correct creation of GNU Longname
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,8 @@
 Library
 -------
 
+- Issue #13639: Accept unicode filenames in tarfile.open(mode="w|gz").
+
 - Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
 
 - Issue #7502: Fix equality comparison for DocTestCase instances.  Patch by

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 02:13:41 2011
From: python-checkins at python.org (michael.foord)
Date: Thu, 22 Dec 2011 02:13:41 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Metaclasses_wit?=
 =?utf8?q?h_metaclasses_with_a_=5F=5Fdict=5F=5F_descriptor_can_no_longer_t?=
 =?utf8?q?rigger?=
Message-ID: <E1RdXE5-0005hN-Vh@dinsdale.python.org>

http://hg.python.org/cpython/rev/8f33758df19a
changeset:   74122:8f33758df19a
branch:      3.2
parent:      74115:82557279efd2
user:        Michael Foord <michael at voidspace.org.uk>
date:        Thu Dec 22 01:13:37 2011 +0000
summary:
  Metaclasses with metaclasses with a __dict__ descriptor can no longer trigger code execution with inspect.getattr_static.
Closes issue 11829.

files:
  Lib/inspect.py           |   9 +++++----
  Lib/test/test_inspect.py |  17 +++++++++++++++++
  Misc/NEWS                |   3 +++
  3 files changed, 25 insertions(+), 4 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -1161,10 +1161,11 @@
     if obj is klass:
         # for types we check the metaclass too
         for entry in _static_getmro(type(klass)):
-            try:
-                return entry.__dict__[attr]
-            except KeyError:
-                pass
+            if _shadowed_dict(type(entry)) is _sentinel:
+                try:
+                    return entry.__dict__[attr]
+                except KeyError:
+                    pass
     if default is not _sentinel:
         return default
     raise AttributeError(attr)
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -1088,6 +1088,23 @@
         self.assertIsNot(inspect.getattr_static(sys, "version", sentinel),
                          sentinel)
 
+    def test_metaclass_with_metaclass_with_dict_as_property(self):
+        class MetaMeta(type):
+            @property
+            def __dict__(self):
+                self.executed = True
+                return dict(spam=42)
+
+        class Meta(type, metaclass=MetaMeta):
+            executed = False
+
+        class Thing(metaclass=Meta):
+            pass
+
+        with self.assertRaises(AttributeError):
+            inspect.getattr_static(Thing, "spam")
+        self.assertFalse(Thing.executed)
+
 class TestGetGeneratorState(unittest.TestCase):
 
     def setUp(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,9 @@
 Library
 -------
 
+- Issue #11829: Fix code execution holes in inspect.getattr_static for
+  metaclasses with metaclasses. Patch by Andreas St?hrk.
+
 - Issue #1785: Fix inspect and pydoc with misbehaving descriptors.
 
 - Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 02:15:56 2011
From: python-checkins at python.org (michael.foord)
Date: Thu, 22 Dec 2011 02:15:56 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge?=
Message-ID: <E1RdXGG-0005t3-Uv@dinsdale.python.org>

http://hg.python.org/cpython/rev/282a2a245278
changeset:   74123:282a2a245278
parent:      74120:9c897817abde
parent:      74122:8f33758df19a
user:        Michael Foord <michael at voidspace.org.uk>
date:        Thu Dec 22 01:15:53 2011 +0000
summary:
  Merge

files:
  Lib/inspect.py           |   9 +++++----
  Lib/test/test_inspect.py |  17 +++++++++++++++++
  Misc/NEWS                |   3 +++
  3 files changed, 25 insertions(+), 4 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -1186,10 +1186,11 @@
     if obj is klass:
         # for types we check the metaclass too
         for entry in _static_getmro(type(klass)):
-            try:
-                return entry.__dict__[attr]
-            except KeyError:
-                pass
+            if _shadowed_dict(type(entry)) is _sentinel:
+                try:
+                    return entry.__dict__[attr]
+                except KeyError:
+                    pass
     if default is not _sentinel:
         return default
     raise AttributeError(attr)
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -1088,6 +1088,23 @@
         self.assertIsNot(inspect.getattr_static(sys, "version", sentinel),
                          sentinel)
 
+    def test_metaclass_with_metaclass_with_dict_as_property(self):
+        class MetaMeta(type):
+            @property
+            def __dict__(self):
+                self.executed = True
+                return dict(spam=42)
+
+        class Meta(type, metaclass=MetaMeta):
+            executed = False
+
+        class Thing(metaclass=Meta):
+            pass
+
+        with self.assertRaises(AttributeError):
+            inspect.getattr_static(Thing, "spam")
+        self.assertFalse(Thing.executed)
+
 class TestGetGeneratorState(unittest.TestCase):
 
     def setUp(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -422,6 +422,9 @@
 - Issue #13620: Support for Chrome browser in webbrowser.py Patch contributed
   by Arnaud Calmettes.
 
+- Issue #11829: Fix code execution holes in inspect.getattr_static for
+  metaclasses with metaclasses. Patch by Andreas St?hrk.
+
 - Issue #12708: Add starmap() and starmap_async() methods (similar to
   itertools.starmap()) to multiprocessing.Pool.  Patch by Hynek Schlawack.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 03:21:07 2011
From: python-checkins at python.org (victor.stinner)
Date: Thu, 22 Dec 2011 03:21:07 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_PyCompactUnicodeObject_?=
	=?utf8?q?doc_=28test=29?=
Message-ID: <E1RdYHL-0001RS-KI@dinsdale.python.org>

http://hg.python.org/cpython/rev/4bbc3ffa1207
changeset:   74124:4bbc3ffa1207
user:        Victor Stinner <victor.stinner at haypocalc.com>
date:        Thu Dec 22 03:23:10 2011 +0100
summary:
  fix PyCompactUnicodeObject doc (test)

files:
  Include/unicodeobject.h |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -232,7 +232,7 @@
        - compact:
 
          * structure = PyCompactUnicodeObject
-         * test: PyUnicode_IS_ASCII(op) && !PyUnicode_IS_COMPACT(op)
+         * test: PyUnicode_IS_COMPACT(op) && !PyUnicode_IS_ASCII(op)
          * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
            PyUnicode_4BYTE_KIND
          * compact = 1

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Dec 22 05:31:33 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 22 Dec 2011 05:31:33 +0100
Subject: [Python-checkins] Daily reference leaks (4bbc3ffa1207): sum=0
Message-ID: <E1RdaJZ-0006l4-6e@ap.vmr.nerim.net>

results for 4bbc3ffa1207 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogeIabKW', '-x']

From python-checkins at python.org  Thu Dec 22 08:21:08 2011
From: python-checkins at python.org (ross.lagerwall)
Date: Thu, 22 Dec 2011 08:21:08 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzExMDA2?=
 =?utf8?q?=3A_Don=27t_issue_low_level_warning_in_subprocess_when_pipe2=28?=
 =?utf8?q?=29_fails=2E?=
Message-ID: <E1Rdcxg-0005D1-Fa@dinsdale.python.org>

http://hg.python.org/cpython/rev/dc913f73a7fb
changeset:   74125:dc913f73a7fb
branch:      3.2
parent:      74122:8f33758df19a
user:        Ross Lagerwall <rosslagerwall at gmail.com>
date:        Thu Dec 22 09:07:30 2011 +0200
summary:
  Issue #11006: Don't issue low level warning in subprocess when pipe2() fails.

files:
  Misc/NEWS                  |  2 ++
  Modules/_posixsubprocess.c |  6 ------
  2 files changed, 2 insertions(+), 6 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,8 @@
 Library
 -------
 
+- Issue #11006: Don't issue low level warning in subprocess when pipe2() fails.
+
 - Issue #11829: Fix code execution holes in inspect.getattr_static for
   metaclasses with metaclasses. Patch by Andreas St?hrk.
 
diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c
--- a/Modules/_posixsubprocess.c
+++ b/Modules/_posixsubprocess.c
@@ -437,12 +437,6 @@
     Py_END_ALLOW_THREADS
     if (res != 0 && errno == ENOSYS)
     {
-        if (PyErr_WarnEx(
-                PyExc_RuntimeWarning,
-                "pipe2 set errno ENOSYS; falling "
-                "back to non-atomic pipe+fcntl.", 1) != 0) {
-            return NULL;
-        }
         {
 #endif
         /* We hold the GIL which offers some protection from other code calling

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 08:21:09 2011
From: python-checkins at python.org (ross.lagerwall)
Date: Thu, 22 Dec 2011 08:21:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_with_3=2E2_for_=2311006=2E?=
Message-ID: <E1Rdcxh-0005D5-8r@dinsdale.python.org>

http://hg.python.org/cpython/rev/b1b35583967a
changeset:   74126:b1b35583967a
parent:      74124:4bbc3ffa1207
parent:      74125:dc913f73a7fb
user:        Ross Lagerwall <rosslagerwall at gmail.com>
date:        Thu Dec 22 09:10:47 2011 +0200
summary:
  Merge with 3.2 for #11006.

files:
  Misc/NEWS                  |  2 ++
  Modules/_posixsubprocess.c |  6 ------
  2 files changed, 2 insertions(+), 6 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,8 @@
 Library
 -------
 
+- Issue #11006: Don't issue low level warning in subprocess when pipe2() fails.
+
 - Issue #13620: Support for Chrome browser in webbrowser.py Patch contributed
   by Arnaud Calmettes.
 
diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c
--- a/Modules/_posixsubprocess.c
+++ b/Modules/_posixsubprocess.c
@@ -442,12 +442,6 @@
     Py_END_ALLOW_THREADS
     if (res != 0 && errno == ENOSYS)
     {
-        if (PyErr_WarnEx(
-                PyExc_RuntimeWarning,
-                "pipe2 set errno ENOSYS; falling "
-                "back to non-atomic pipe+fcntl.", 1) != 0) {
-            return NULL;
-        }
         {
 #endif
         /* We hold the GIL which offers some protection from other code calling

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 08:48:20 2011
From: python-checkins at python.org (ross.lagerwall)
Date: Thu, 22 Dec 2011 08:48:20 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Don=27t_redefin?=
 =?utf8?q?e_=5FGNU=5FSOURCE_if_it=27s_already_defined=2E?=
Message-ID: <E1RddO0-00079x-Cv@dinsdale.python.org>

http://hg.python.org/cpython/rev/69eee9084ba3
changeset:   74127:69eee9084ba3
branch:      3.2
parent:      74125:dc913f73a7fb
user:        Ross Lagerwall <rosslagerwall at gmail.com>
date:        Thu Dec 22 09:45:53 2011 +0200
summary:
  Don't redefine _GNU_SOURCE if it's already defined.

files:
  Modules/_posixsubprocess.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c
--- a/Modules/_posixsubprocess.c
+++ b/Modules/_posixsubprocess.c
@@ -1,7 +1,7 @@
 /* Authors: Gregory P. Smith & Jeffrey Yasskin */
 #include "Python.h"
-#ifdef HAVE_PIPE2
-#define _GNU_SOURCE
+#if defined(HAVE_PIPE2) && !defined(_GNU_SOURCE)
+# define _GNU_SOURCE
 #endif
 #include <unistd.h>
 #include <fcntl.h>

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 08:48:21 2011
From: python-checkins at python.org (ross.lagerwall)
Date: Thu, 22 Dec 2011 08:48:21 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_Merge_with_3=2E2=2E?=
Message-ID: <E1RddO1-0007A7-4n@dinsdale.python.org>

http://hg.python.org/cpython/rev/9682a975bc1e
changeset:   74128:9682a975bc1e
parent:      74126:b1b35583967a
parent:      74127:69eee9084ba3
user:        Ross Lagerwall <rosslagerwall at gmail.com>
date:        Thu Dec 22 09:46:44 2011 +0200
summary:
  Merge with 3.2.

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 10:04:20 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 22 Dec 2011 10:04:20 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313626=3A_Add_suppo?=
 =?utf8?q?rt_for_SSL_Diffie-Hellman_key_exchange=2C_through_the?=
Message-ID: <E1RdeZY-00041r-S8@dinsdale.python.org>

http://hg.python.org/cpython/rev/33dea851f918
changeset:   74129:33dea851f918
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 22 10:03:38 2011 +0100
summary:
  Issue #13626: Add support for SSL Diffie-Hellman key exchange, through the
SSLContext.load_dh_params() method and the ssl.OP_SINGLE_DH_USE option.

files:
  Doc/library/ssl.rst     |  30 ++++++++++++++++++++----
  Lib/ssl.py              |   2 +-
  Lib/test/ssl_servers.py |   4 +++
  Lib/test/test_ssl.py    |  29 ++++++++++++++++++++++++
  Misc/NEWS               |   3 ++
  Modules/_ssl.c          |  35 +++++++++++++++++++++++++++++
  Python/fileutils.c      |   6 ++++
  7 files changed, 103 insertions(+), 6 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -428,9 +428,17 @@
 
    .. versionadded:: 3.3
 
+.. data:: OP_SINGLE_DH_USE
+
+   Prevents re-use of the same DH key for distinct SSL sessions.  This
+   improves forward secrecy but requires more computational resources.
+   This option only applies to server sockets.
+
+   .. versionadded:: 3.3
+
 .. data:: OP_SINGLE_ECDH_USE
 
-   Prevents re-use of the same ECDH key for several SSL sessions.  This
+   Prevents re-use of the same ECDH key for distinct SSL sessions.  This
    improves forward secrecy but requires more computational resources.
    This option only applies to server sockets.
 
@@ -707,12 +715,24 @@
       when connected, the :meth:`SSLSocket.cipher` method of SSL sockets will
       give the currently selected cipher.
 
+.. method:: SSLContext.load_dh_params(dhfile)
+
+   Load the key generation parameters for Diffie-Helman (DH) key exchange.
+   Using DH key exchange improves forward secrecy at the expense of
+   computational resources (both on the server and on the client).
+   The *dhfile* parameter should be the path to a file containing DH
+   parameters in PEM format.
+
+   This setting doesn't apply to client sockets.  You can also use the
+   :data:`OP_SINGLE_DH_USE` option to further improve security.
+
+   .. versionadded:: 3.3
+
 .. method:: SSLContext.set_ecdh_curve(curve_name)
 
-   Set the curve name for Elliptic Curve-based Diffie-Hellman (abbreviated
-   ECDH) key exchange.  Using Diffie-Hellman key exchange improves forward
-   secrecy at the expense of computational resources (both on the server and
-   on the client).  The *curve_name* parameter should be a string describing
+   Set the curve name for Elliptic Curve-based Diffie-Hellman (ECDH) key
+   exchange.  ECDH is significantly faster than regular DH while arguably
+   as secure.  The *curve_name* parameter should be a string describing
    a well-known elliptic curve, for example ``prime256v1`` for a widely
    supported curve.
 
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -68,7 +68,7 @@
 from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED
 from _ssl import (
     OP_ALL, OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1,
-    OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_ECDH_USE,
+    OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE,
     )
 try:
     from _ssl import OP_NO_COMPRESSION
diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py
--- a/Lib/test/ssl_servers.py
+++ b/Lib/test/ssl_servers.py
@@ -180,6 +180,8 @@
     parser.add_argument('--curve-name', dest='curve_name', type=str,
                         action='store',
                         help='curve name for EC-based Diffie-Hellman')
+    parser.add_argument('--dh', dest='dh_file', type=str, action='store',
+                        help='PEM file containing DH parameters')
     args = parser.parse_args()
 
     support.verbose = args.verbose
@@ -192,6 +194,8 @@
     context.load_cert_chain(CERTFILE)
     if args.curve_name:
         context.set_ecdh_curve(args.curve_name)
+    if args.dh_file:
+        context.load_dh_params(args.dh_file)
 
     server = HTTPSServer(("", args.port), handler_class, context)
     if args.verbose:
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -56,6 +56,8 @@
 BADKEY = data_file("badkey.pem")
 NOKIACERT = data_file("nokia.pem")
 
+DHFILE = data_file("dh512.pem")
+BYTES_DHFILE = os.fsencode(DHFILE)
 
 def handle_error(prefix):
     exc_format = ' '.join(traceback.format_exception(*sys.exc_info()))
@@ -99,6 +101,7 @@
         ssl.CERT_OPTIONAL
         ssl.CERT_REQUIRED
         ssl.OP_CIPHER_SERVER_PREFERENCE
+        ssl.OP_SINGLE_DH_USE
         ssl.OP_SINGLE_ECDH_USE
         if ssl.OPENSSL_VERSION_INFO >= (1, 0):
             ssl.OP_NO_COMPRESSION
@@ -538,6 +541,19 @@
         # Issue #10989: crash if the second argument type is invalid
         self.assertRaises(TypeError, ctx.load_verify_locations, None, True)
 
+    def test_load_dh_params(self):
+        ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+        ctx.load_dh_params(DHFILE)
+        if os.name != 'nt':
+            ctx.load_dh_params(BYTES_DHFILE)
+        self.assertRaises(TypeError, ctx.load_dh_params)
+        self.assertRaises(TypeError, ctx.load_dh_params, None)
+        with self.assertRaises(FileNotFoundError) as cm:
+            ctx.load_dh_params(WRONGCERT)
+        self.assertEqual(cm.exception.errno, errno.ENOENT)
+        with self.assertRaisesRegex(ssl.SSLError, "PEM routines"):
+            ctx.load_dh_params(CERTFILE)
+
     @skip_if_broken_ubuntu_ssl
     def test_session_stats(self):
         for proto in PROTOCOLS:
@@ -1802,6 +1818,19 @@
                                        chatty=True, connectionchatty=True)
             self.assertIs(stats['compression'], None)
 
+        def test_dh_params(self):
+            # Check we can get a connection with ephemeral Diffie-Hellman
+            context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            context.load_cert_chain(CERTFILE)
+            context.load_dh_params(DHFILE)
+            context.set_ciphers("kEDH")
+            stats = server_params_test(context, context,
+                                       chatty=True, connectionchatty=True)
+            cipher = stats["cipher"][0]
+            parts = cipher.split("-")
+            if "ADH" not in parts and "EDH" not in parts and "DHE" not in parts:
+                self.fail("Non-DH cipher: " + cipher[0])
+
 
 def test_main(verbose=False):
     if support.verbose:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,9 @@
 Library
 -------
 
+- Issue #13626: Add support for SSL Diffie-Hellman key exchange, through the
+  SSLContext.load_dh_params() method and the ssl.OP_SINGLE_DH_USE option.
+
 - Issue #11006: Don't issue low level warning in subprocess when pipe2() fails.
 
 - Issue #13620: Support for Chrome browser in webbrowser.py Patch contributed
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1922,6 +1922,38 @@
 }
 
 static PyObject *
+load_dh_params(PySSLContext *self, PyObject *filepath)
+{
+    FILE *f;
+    DH *dh;
+
+    f = _Py_fopen(filepath, "rb");
+    if (f == NULL) {
+        if (!PyErr_Occurred())
+            PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, filepath);
+        return NULL;
+    }
+    errno = 0;
+    PySSL_BEGIN_ALLOW_THREADS
+    dh = PEM_read_DHparams(f, NULL, NULL, NULL);
+    PySSL_END_ALLOW_THREADS
+    if (dh == NULL) {
+        if (errno != 0) {
+            ERR_clear_error();
+            PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, filepath);
+        }
+        else {
+            _setSSLError(NULL, 0, __FILE__, __LINE__);
+        }
+        return NULL;
+    }
+    if (SSL_CTX_set_tmp_dh(self->ctx, dh) == 0)
+        _setSSLError(NULL, 0, __FILE__, __LINE__);
+    DH_free(dh);
+    Py_RETURN_NONE;
+}
+
+static PyObject *
 context_wrap_socket(PySSLContext *self, PyObject *args, PyObject *kwds)
 {
     char *kwlist[] = {"sock", "server_side", "server_hostname", NULL};
@@ -2050,6 +2082,8 @@
                     METH_VARARGS, NULL},
     {"load_cert_chain", (PyCFunction) load_cert_chain,
                         METH_VARARGS | METH_KEYWORDS, NULL},
+    {"load_dh_params", (PyCFunction) load_dh_params,
+                       METH_O, NULL},
     {"load_verify_locations", (PyCFunction) load_verify_locations,
                               METH_VARARGS | METH_KEYWORDS, NULL},
     {"session_stats", (PyCFunction) session_stats,
@@ -2505,6 +2539,7 @@
     PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1);
     PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE",
                             SSL_OP_CIPHER_SERVER_PREFERENCE);
+    PyModule_AddIntConstant(m, "OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE);
     PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE);
 #ifdef SSL_OP_NO_COMPRESSION
     PyModule_AddIntConstant(m, "OP_NO_COMPRESSION",
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -310,6 +310,12 @@
     wchar_t wmode[10];
     int usize;
 
+    if (!PyUnicode_Check(path)) {
+        PyErr_Format(PyExc_TypeError,
+                     "str file path expected under Windows, got %R",
+                     Py_TYPE(path));
+        return NULL;
+    }
     wpath = PyUnicode_AsUnicode(path);
     if (wpath == NULL)
         return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 10:06:32 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 22 Dec 2011 10:06:32 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Forgot_the_data_file_for_Di?=
	=?utf8?q?ffie-Hellman_tests=2E?=
Message-ID: <E1Rdebg-0004F1-Mb@dinsdale.python.org>

http://hg.python.org/cpython/rev/475dc8f3cc6a
changeset:   74130:475dc8f3cc6a
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 22 10:05:50 2011 +0100
summary:
  Forgot the data file for Diffie-Hellman tests.

files:
  Lib/test/dh512.pem |  9 +++++++++
  1 files changed, 9 insertions(+), 0 deletions(-)


diff --git a/Lib/test/dh512.pem b/Lib/test/dh512.pem
new file mode 100644
--- /dev/null
+++ b/Lib/test/dh512.pem
@@ -0,0 +1,9 @@
+-----BEGIN DH PARAMETERS-----
+MEYCQQD1Kv884bEpQBgRjXyEpwpy1obEAxnIByl6ypUM2Zafq9AKUJsCRtMIPWak
+XUGfnHy9iUsiGSa6q6Jew1XpKgVfAgEC
+-----END DH PARAMETERS-----
+
+These are the 512 bit DH parameters from "Assigned Number for SKIP Protocols"
+(http://www.skip-vpn.org/spec/numbers.html).
+See there for how they were generated.
+Note that g is not a generator, but this is not a problem since p is a safe prime.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 22 16:45:17 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Thu, 22 Dec 2011 16:45:17 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Issue13443_-_Re?=
 =?utf8?q?move_the_functional_module_examples_from_2=2E7_=28as_module_is?=
Message-ID: <E1RdkpZ-0005O0-2k@dinsdale.python.org>

http://hg.python.org/cpython/rev/c6880edaf6f3
changeset:   74131:c6880edaf6f3
branch:      2.7
parent:      74121:a60a3610a97b
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Thu Dec 22 23:44:53 2011 +0800
summary:
  Issue13443 - Remove the functional module examples from 2.7 (as module is
maintained only till 2.5 and tests had failures with 2.7) and update the links
int the howto document.

files:
  Doc/howto/functional.rst |  149 ++------------------------
  1 files changed, 12 insertions(+), 137 deletions(-)


diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -44,15 +44,14 @@
   functional languages include the ML family (Standard ML, OCaml, and other
   variants) and Haskell.
 
-The designers of some computer languages choose to emphasize one
-particular approach to programming.  This often makes it difficult to
-write programs that use a different approach.  Other languages are
-multi-paradigm languages that support several different approaches.
-Lisp, C++, and Python are multi-paradigm; you can write programs or
-libraries that are largely procedural, object-oriented, or functional
-in all of these languages.  In a large program, different sections
-might be written using different approaches; the GUI might be
-object-oriented while the processing logic is procedural or
+The designers of some computer languages choose to emphasize one particular
+approach to programming.  This often makes it difficult to write programs that
+use a different approach.  Other languages are multi-paradigm languages that
+support several different approaches.  Lisp, C++, and Python are
+multi-paradigm; you can write programs or libraries that are largely
+procedural, object-oriented, or functional in all of these languages.  In a
+large program, different sections might be written using different approaches;
+the GUI might be object-oriented while the processing logic is procedural or
 functional, for example.
 
 In a functional program, input flows through a set of functions. Each function
@@ -1115,132 +1114,6 @@
 Consult the operator module's documentation for a complete list.
 
 
-
-The functional module
----------------------
-
-Collin Winter's `functional module <http://oakwinter.com/code/functional/>`__
-provides a number of more advanced tools for functional programming. It also
-reimplements several Python built-ins, trying to make them more intuitive to
-those used to functional programming in other languages.
-
-This section contains an introduction to some of the most important functions in
-``functional``; full documentation can be found at `the project's website
-<http://oakwinter.com/code/functional/documentation/>`__.
-
-``compose(outer, inner, unpack=False)``
-
-The ``compose()`` function implements function composition.  In other words, it
-returns a wrapper around the ``outer`` and ``inner`` callables, such that the
-return value from ``inner`` is fed directly to ``outer``.  That is, ::
-
-    >>> def add(a, b):
-    ...     return a + b
-    ...
-    >>> def double(a):
-    ...     return 2 * a
-    ...
-    >>> compose(double, add)(5, 6)
-    22
-
-is equivalent to ::
-
-    >>> double(add(5, 6))
-    22
-
-The ``unpack`` keyword is provided to work around the fact that Python functions
-are not always `fully curried <http://en.wikipedia.org/wiki/Currying>`__.  By
-default, it is expected that the ``inner`` function will return a single object
-and that the ``outer`` function will take a single argument. Setting the
-``unpack`` argument causes ``compose`` to expect a tuple from ``inner`` which
-will be expanded before being passed to ``outer``. Put simply, ::
-
-    compose(f, g)(5, 6)
-
-is equivalent to::
-
-    f(g(5, 6))
-
-while ::
-
-    compose(f, g, unpack=True)(5, 6)
-
-is equivalent to::
-
-    f(*g(5, 6))
-
-Even though ``compose()`` only accepts two functions, it's trivial to build up a
-version that will compose any number of functions. We'll use ``reduce()``,
-``compose()`` and ``partial()`` (the last of which is provided by both
-``functional`` and ``functools``). ::
-
-    from functional import compose, partial
-
-    multi_compose = partial(reduce, compose)
-
-
-We can also use ``map()``, ``compose()`` and ``partial()`` to craft a version of
-``"".join(...)`` that converts its arguments to string::
-
-    from functional import compose, partial
-
-    join = compose("".join, partial(map, str))
-
-
-``flip(func)``
-
-``flip()`` wraps the callable in ``func`` and causes it to receive its
-non-keyword arguments in reverse order. ::
-
-    >>> def triple(a, b, c):
-    ...     return (a, b, c)
-    ...
-    >>> triple(5, 6, 7)
-    (5, 6, 7)
-    >>>
-    >>> flipped_triple = flip(triple)
-    >>> flipped_triple(5, 6, 7)
-    (7, 6, 5)
-
-``foldl(func, start, iterable)``
-
-``foldl()`` takes a binary function, a starting value (usually some kind of
-'zero'), and an iterable.  The function is applied to the starting value and the
-first element of the list, then the result of that and the second element of the
-list, then the result of that and the third element of the list, and so on.
-
-This means that a call such as::
-
-    foldl(f, 0, [1, 2, 3])
-
-is equivalent to::
-
-    f(f(f(0, 1), 2), 3)
-
-
-``foldl()`` is roughly equivalent to the following recursive function::
-
-    def foldl(func, start, seq):
-        if len(seq) == 0:
-            return start
-
-        return foldl(func, func(start, seq[0]), seq[1:])
-
-Speaking of equivalence, the above ``foldl`` call can be expressed in terms of
-the built-in ``reduce`` like so::
-
-    reduce(f, [1, 2, 3], 0)
-
-
-We can use ``foldl()``, ``operator.concat()`` and ``partial()`` to write a
-cleaner, more aesthetically-pleasing version of Python's ``"".join(...)``
-idiom::
-
-    from functional import foldl, partial from operator import concat
-
-    join = partial(foldl, concat, "")
-
-
 Revision History and Acknowledgements
 =====================================
 
@@ -1296,9 +1169,10 @@
 
 Mertz also wrote a 3-part series of articles on functional programming
 for IBM's DeveloperWorks site; see
-`part 1 <http://www-128.ibm.com/developerworks/library/l-prog.html>`__,
-`part 2 <http://www-128.ibm.com/developerworks/library/l-prog2.html>`__, and
-`part 3 <http://www-128.ibm.com/developerworks/linux/library/l-prog3.html>`__,
+
+`part 1 <http://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__,
+`part 2 <http://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and
+`part 3 <http://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__,
 
 
 Python documentation

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 03:54:33 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 03:54:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Issue12798_-_Up?=
 =?utf8?q?date_mimetypes_documentation=2E_Correct_the_doc_section_where?=
Message-ID: <E1RdvHF-0005ja-Ds@dinsdale.python.org>

http://hg.python.org/cpython/rev/aef79ff1bc9b
changeset:   74132:aef79ff1bc9b
branch:      3.2
parent:      74127:69eee9084ba3
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 10:52:42 2011 +0800
summary:
  Issue12798 - Update mimetypes documentation. Correct the doc section where
common_types was mentioned under MimeTypes class, move it to helper function
category and other documentation improvements contributed by Sandro Tosi.

files:
  Doc/library/mimetypes.rst |  102 +++++++++++++++----------
  1 files changed, 59 insertions(+), 43 deletions(-)


diff --git a/Doc/library/mimetypes.rst b/Doc/library/mimetypes.rst
--- a/Doc/library/mimetypes.rst
+++ b/Doc/library/mimetypes.rst
@@ -30,23 +30,23 @@
 
    .. index:: pair: MIME; headers
 
-   Guess the type of a file based on its filename or URL, given by *filename*.  The
+   Guess the type of a file based on its filename or URL, given by *url*.  The
    return value is a tuple ``(type, encoding)`` where *type* is ``None`` if the
    type can't be guessed (missing or unknown suffix) or a string of the form
    ``'type/subtype'``, usable for a MIME :mailheader:`content-type` header.
 
    *encoding* is ``None`` for no encoding or the name of the program used to encode
    (e.g. :program:`compress` or :program:`gzip`). The encoding is suitable for use
-   as a :mailheader:`Content-Encoding` header, *not* as a
+   as a :mailheader:`Content-Encoding` header, **not** as a
    :mailheader:`Content-Transfer-Encoding` header. The mappings are table driven.
    Encoding suffixes are case sensitive; type suffixes are first tried case
    sensitively, then case insensitively.
 
-   Optional *strict* is a flag specifying whether the list of known MIME types
+   The optional *strict* argument is a flag specifying whether the list of known MIME types
    is limited to only the official types `registered with IANA
-   <http://www.iana.org/assignments/media-types/>`_ are recognized.
-   When *strict* is true (the default), only the IANA types are supported; when
-   *strict* is false, some additional non-standard but commonly used MIME types
+   <http://www.iana.org/assignments/media-types/>`_.
+   When *strict* is ``True`` (the default), only the IANA types are supported; when
+   *strict* is ``False``, some additional non-standard but commonly used MIME types
    are also recognized.
 
 
@@ -58,7 +58,7 @@
    been associated with any particular data stream, but would be mapped to the MIME
    type *type* by :func:`guess_type`.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 
 .. function:: guess_extension(type, strict=True)
@@ -66,11 +66,11 @@
    Guess the extension for a file based on its MIME type, given by *type*. The
    return value is a string giving a filename extension, including the leading dot
    (``'.'``).  The extension is not guaranteed to have been associated with any
-   particular data stream, but would be mapped to the  MIME type *type* by
+   particular data stream, but would be mapped to the MIME type *type* by
    :func:`guess_type`.  If no extension can be guessed for *type*, ``None`` is
    returned.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 Some additional functions and data items are available for controlling the
 behavior of the module.
@@ -91,7 +91,7 @@
 
 .. function:: read_mime_types(filename)
 
-   Load the type map given in the file *filename*, if it exists.  The  type map is
+   Load the type map given in the file *filename*, if it exists.  The type map is
    returned as a dictionary mapping filename extensions, including the leading dot
    (``'.'``), to strings of the form ``'type/subtype'``.  If the file *filename*
    does not exist or cannot be read, ``None`` is returned.
@@ -99,18 +99,18 @@
 
 .. function:: add_type(type, ext, strict=True)
 
-   Add a mapping from the mimetype *type* to the extension *ext*. When the
+   Add a mapping from the MIME type *type* to the extension *ext*. When the
    extension is already known, the new type will replace the old one. When the type
    is already known the extension will be added to the list of known extensions.
 
-   When *strict* is True (the default), the mapping will added to the official MIME
+   When *strict* is ``True`` (the default), the mapping will added to the official MIME
    types, otherwise to the non-standard ones.
 
 
 .. data:: inited
 
    Flag indicating whether or not the global data structures have been initialized.
-   This is set to true by :func:`init`.
+   This is set to ``True`` by :func:`init`.
 
 
 .. data:: knownfiles
@@ -145,22 +145,6 @@
    Dictionary mapping filename extensions to non-standard, but commonly found MIME
    types.
 
-The :class:`MimeTypes` class may be useful for applications which may want more
-than one MIME-type database:
-
-
-.. class:: MimeTypes(filenames=(), strict=True)
-
-   This class represents a MIME-types database.  By default, it provides access to
-   the same database as the rest of this module. The initial database is a copy of
-   that provided by the module, and may be extended by loading additional
-   :file:`mime.types`\ -style files into the database using the :meth:`read` or
-   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
-   loading additional data if the default data is not desired.
-
-   The optional *filenames* parameter can be used to cause additional files to be
-   loaded "on top" of the default database.
-
 
 An example usage of the module::
 
@@ -181,36 +165,53 @@
 MimeTypes Objects
 -----------------
 
-:class:`MimeTypes` instances provide an interface which is very like that of the
+The :class:`MimeTypes` class may be useful for applications which may want more
+than one MIME-type database; it provides an interface similar to the one of the
 :mod:`mimetypes` module.
 
 
+.. class:: MimeTypes(filenames=(), strict=True)
+
+   This class represents a MIME-types database.  By default, it provides access to
+   the same database as the rest of this module. The initial database is a copy of
+   that provided by the module, and may be extended by loading additional
+   :file:`mime.types`\ -style files into the database using the :meth:`read` or
+   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
+   loading additional data if the default data is not desired.
+
+   The optional *filenames* parameter can be used to cause additional files to be
+   loaded "on top" of the default database.
+
+
 .. attribute:: MimeTypes.suffix_map
 
    Dictionary mapping suffixes to suffixes.  This is used to allow recognition of
    encoded files for which the encoding and the type are indicated by the same
    extension.  For example, the :file:`.tgz` extension is mapped to :file:`.tar.gz`
    to allow the encoding and type to be recognized separately.  This is initially a
-   copy of the global ``suffix_map`` defined in the module.
+   copy of the global :data:`suffix_map` defined in the module.
 
 
 .. attribute:: MimeTypes.encodings_map
 
    Dictionary mapping filename extensions to encoding types.  This is initially a
-   copy of the global ``encodings_map`` defined in the module.
+   copy of the global :data:`encodings_map` defined in the module.
 
 
 .. attribute:: MimeTypes.types_map
 
-   Dictionary mapping filename extensions to MIME types.  This is initially a copy
-   of the global ``types_map`` defined in the module.
+   Tuple containing two dictionaries, mapping filename extensions to MIME types:
+   the first dictionary is for the non-standards types and the second one is for
+   the standard types. They are initialized by :data:`common_types` and
+   :data:`types_map`.
 
 
-.. attribute:: MimeTypes.common_types
+.. attribute:: MimeTypes.types_map_inv
 
-   Dictionary mapping filename extensions to non-standard, but commonly found MIME
-   types.  This is initially a copy of the global ``common_types`` defined in the
-   module.
+   Tuple containing two dictionaries, mapping MIME types to a list of filename
+   extensions: the first dictionary is for the non-standards types and the
+   second one is for the standard types. They are initialized by
+   :data:`common_types` and :data:`types_map`.
 
 
 .. method:: MimeTypes.guess_extension(type, strict=True)
@@ -225,20 +226,35 @@
    the object.
 
 
-.. method:: MimeTypes.read(path)
+.. method:: MimeTypes.guess_all_extensions(type, strict=True)
 
-   Load MIME information from a file named *path*.  This uses :meth:`readfp` to
+   Similar to the :func:`guess_all_extensions` function, using the tables stored
+   as part of the object.
+
+
+.. method:: MimeTypes.read(filename, strict=True)
+
+   Load MIME information from a file named *filename*.  This uses :meth:`readfp` to
    parse the file.
 
+   If *strict* is ``True``, information will be added to list of standard types,
+   else to the list of non-standard types.
 
-.. method:: MimeTypes.readfp(file)
 
-   Load MIME type information from an open file.  The file must have the format of
+.. method:: MimeTypes.readfp(fp, strict=True)
+
+   Load MIME type information from an open file *fp*.  The file must have the format of
    the standard :file:`mime.types` files.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
 
-.. method:: MimeTypes.read_windows_registry()
+
+.. method:: MimeTypes.read_windows_registry(strict=True)
 
    Load MIME type information from the Windows registry.  Availability: Windows.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
+
    .. versionadded:: 3.2

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 03:54:34 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 03:54:34 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Merge_changes_from_3=2E2?=
Message-ID: <E1RdvHG-0005je-Aw@dinsdale.python.org>

http://hg.python.org/cpython/rev/4b306aee21a4
changeset:   74133:4b306aee21a4
parent:      74130:475dc8f3cc6a
parent:      74132:aef79ff1bc9b
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 10:54:11 2011 +0800
summary:
  Merge changes from 3.2

Issue12798 - Update mimetypes documentation. Correct the doc section where
common_types was mentioned under MimeTypes class, move it to helper function
category and other documentation improvements contributed by Sandro Tosi.

files:
  Doc/library/mimetypes.rst |  102 +++++++++++++++----------
  1 files changed, 59 insertions(+), 43 deletions(-)


diff --git a/Doc/library/mimetypes.rst b/Doc/library/mimetypes.rst
--- a/Doc/library/mimetypes.rst
+++ b/Doc/library/mimetypes.rst
@@ -30,23 +30,23 @@
 
    .. index:: pair: MIME; headers
 
-   Guess the type of a file based on its filename or URL, given by *filename*.  The
+   Guess the type of a file based on its filename or URL, given by *url*.  The
    return value is a tuple ``(type, encoding)`` where *type* is ``None`` if the
    type can't be guessed (missing or unknown suffix) or a string of the form
    ``'type/subtype'``, usable for a MIME :mailheader:`content-type` header.
 
    *encoding* is ``None`` for no encoding or the name of the program used to encode
    (e.g. :program:`compress` or :program:`gzip`). The encoding is suitable for use
-   as a :mailheader:`Content-Encoding` header, *not* as a
+   as a :mailheader:`Content-Encoding` header, **not** as a
    :mailheader:`Content-Transfer-Encoding` header. The mappings are table driven.
    Encoding suffixes are case sensitive; type suffixes are first tried case
    sensitively, then case insensitively.
 
-   Optional *strict* is a flag specifying whether the list of known MIME types
+   The optional *strict* argument is a flag specifying whether the list of known MIME types
    is limited to only the official types `registered with IANA
-   <http://www.iana.org/assignments/media-types/>`_ are recognized.
-   When *strict* is true (the default), only the IANA types are supported; when
-   *strict* is false, some additional non-standard but commonly used MIME types
+   <http://www.iana.org/assignments/media-types/>`_.
+   When *strict* is ``True`` (the default), only the IANA types are supported; when
+   *strict* is ``False``, some additional non-standard but commonly used MIME types
    are also recognized.
 
 
@@ -58,7 +58,7 @@
    been associated with any particular data stream, but would be mapped to the MIME
    type *type* by :func:`guess_type`.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 
 .. function:: guess_extension(type, strict=True)
@@ -66,11 +66,11 @@
    Guess the extension for a file based on its MIME type, given by *type*. The
    return value is a string giving a filename extension, including the leading dot
    (``'.'``).  The extension is not guaranteed to have been associated with any
-   particular data stream, but would be mapped to the  MIME type *type* by
+   particular data stream, but would be mapped to the MIME type *type* by
    :func:`guess_type`.  If no extension can be guessed for *type*, ``None`` is
    returned.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 Some additional functions and data items are available for controlling the
 behavior of the module.
@@ -91,7 +91,7 @@
 
 .. function:: read_mime_types(filename)
 
-   Load the type map given in the file *filename*, if it exists.  The  type map is
+   Load the type map given in the file *filename*, if it exists.  The type map is
    returned as a dictionary mapping filename extensions, including the leading dot
    (``'.'``), to strings of the form ``'type/subtype'``.  If the file *filename*
    does not exist or cannot be read, ``None`` is returned.
@@ -99,18 +99,18 @@
 
 .. function:: add_type(type, ext, strict=True)
 
-   Add a mapping from the mimetype *type* to the extension *ext*. When the
+   Add a mapping from the MIME type *type* to the extension *ext*. When the
    extension is already known, the new type will replace the old one. When the type
    is already known the extension will be added to the list of known extensions.
 
-   When *strict* is True (the default), the mapping will added to the official MIME
+   When *strict* is ``True`` (the default), the mapping will added to the official MIME
    types, otherwise to the non-standard ones.
 
 
 .. data:: inited
 
    Flag indicating whether or not the global data structures have been initialized.
-   This is set to true by :func:`init`.
+   This is set to ``True`` by :func:`init`.
 
 
 .. data:: knownfiles
@@ -145,22 +145,6 @@
    Dictionary mapping filename extensions to non-standard, but commonly found MIME
    types.
 
-The :class:`MimeTypes` class may be useful for applications which may want more
-than one MIME-type database:
-
-
-.. class:: MimeTypes(filenames=(), strict=True)
-
-   This class represents a MIME-types database.  By default, it provides access to
-   the same database as the rest of this module. The initial database is a copy of
-   that provided by the module, and may be extended by loading additional
-   :file:`mime.types`\ -style files into the database using the :meth:`read` or
-   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
-   loading additional data if the default data is not desired.
-
-   The optional *filenames* parameter can be used to cause additional files to be
-   loaded "on top" of the default database.
-
 
 An example usage of the module::
 
@@ -181,36 +165,53 @@
 MimeTypes Objects
 -----------------
 
-:class:`MimeTypes` instances provide an interface which is very like that of the
+The :class:`MimeTypes` class may be useful for applications which may want more
+than one MIME-type database; it provides an interface similar to the one of the
 :mod:`mimetypes` module.
 
 
+.. class:: MimeTypes(filenames=(), strict=True)
+
+   This class represents a MIME-types database.  By default, it provides access to
+   the same database as the rest of this module. The initial database is a copy of
+   that provided by the module, and may be extended by loading additional
+   :file:`mime.types`\ -style files into the database using the :meth:`read` or
+   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
+   loading additional data if the default data is not desired.
+
+   The optional *filenames* parameter can be used to cause additional files to be
+   loaded "on top" of the default database.
+
+
 .. attribute:: MimeTypes.suffix_map
 
    Dictionary mapping suffixes to suffixes.  This is used to allow recognition of
    encoded files for which the encoding and the type are indicated by the same
    extension.  For example, the :file:`.tgz` extension is mapped to :file:`.tar.gz`
    to allow the encoding and type to be recognized separately.  This is initially a
-   copy of the global ``suffix_map`` defined in the module.
+   copy of the global :data:`suffix_map` defined in the module.
 
 
 .. attribute:: MimeTypes.encodings_map
 
    Dictionary mapping filename extensions to encoding types.  This is initially a
-   copy of the global ``encodings_map`` defined in the module.
+   copy of the global :data:`encodings_map` defined in the module.
 
 
 .. attribute:: MimeTypes.types_map
 
-   Dictionary mapping filename extensions to MIME types.  This is initially a copy
-   of the global ``types_map`` defined in the module.
+   Tuple containing two dictionaries, mapping filename extensions to MIME types:
+   the first dictionary is for the non-standards types and the second one is for
+   the standard types. They are initialized by :data:`common_types` and
+   :data:`types_map`.
 
 
-.. attribute:: MimeTypes.common_types
+.. attribute:: MimeTypes.types_map_inv
 
-   Dictionary mapping filename extensions to non-standard, but commonly found MIME
-   types.  This is initially a copy of the global ``common_types`` defined in the
-   module.
+   Tuple containing two dictionaries, mapping MIME types to a list of filename
+   extensions: the first dictionary is for the non-standards types and the
+   second one is for the standard types. They are initialized by
+   :data:`common_types` and :data:`types_map`.
 
 
 .. method:: MimeTypes.guess_extension(type, strict=True)
@@ -225,20 +226,35 @@
    the object.
 
 
-.. method:: MimeTypes.read(path)
+.. method:: MimeTypes.guess_all_extensions(type, strict=True)
 
-   Load MIME information from a file named *path*.  This uses :meth:`readfp` to
+   Similar to the :func:`guess_all_extensions` function, using the tables stored
+   as part of the object.
+
+
+.. method:: MimeTypes.read(filename, strict=True)
+
+   Load MIME information from a file named *filename*.  This uses :meth:`readfp` to
    parse the file.
 
+   If *strict* is ``True``, information will be added to list of standard types,
+   else to the list of non-standard types.
 
-.. method:: MimeTypes.readfp(file)
 
-   Load MIME type information from an open file.  The file must have the format of
+.. method:: MimeTypes.readfp(fp, strict=True)
+
+   Load MIME type information from an open file *fp*.  The file must have the format of
    the standard :file:`mime.types` files.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
 
-.. method:: MimeTypes.read_windows_registry()
+
+.. method:: MimeTypes.read_windows_registry(strict=True)
 
    Load MIME type information from the Windows registry.  Availability: Windows.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
+
    .. versionadded:: 3.2

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 04:08:48 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 04:08:48 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_porting_mimetyp?=
 =?utf8?q?e_doc_changes_from_3=2E2=2E?=
Message-ID: <E1RdvV2-0007Ec-H4@dinsdale.python.org>

http://hg.python.org/cpython/rev/cfa3fe9d7b1f
changeset:   74134:cfa3fe9d7b1f
branch:      2.7
parent:      74131:c6880edaf6f3
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 11:08:31 2011 +0800
summary:
  porting mimetype doc changes from 3.2.

Issue12798 - Update mimetypes documentation. Correct the doc section where
common_types was mentioned under MimeTypes class, move it to helper function
category and other documentation improvements contributed by Sandro Tosi.

files:
  Doc/library/mimetypes.rst |  124 +++++++++++++------------
  1 files changed, 66 insertions(+), 58 deletions(-)


diff --git a/Doc/library/mimetypes.rst b/Doc/library/mimetypes.rst
--- a/Doc/library/mimetypes.rst
+++ b/Doc/library/mimetypes.rst
@@ -1,4 +1,3 @@
-
 :mod:`mimetypes` --- Map filenames to MIME types
 ================================================
 
@@ -27,31 +26,31 @@
 the information :func:`init` sets up.
 
 
-.. function:: guess_type(filename[, strict])
+.. function:: guess_type(url, strict=True)
 
    .. index:: pair: MIME; headers
 
-   Guess the type of a file based on its filename or URL, given by *filename*.  The
+   Guess the type of a file based on its filename or URL, given by *url*.  The
    return value is a tuple ``(type, encoding)`` where *type* is ``None`` if the
    type can't be guessed (missing or unknown suffix) or a string of the form
    ``'type/subtype'``, usable for a MIME :mailheader:`content-type` header.
 
    *encoding* is ``None`` for no encoding or the name of the program used to encode
    (e.g. :program:`compress` or :program:`gzip`). The encoding is suitable for use
-   as a :mailheader:`Content-Encoding` header, *not* as a
+   as a :mailheader:`Content-Encoding` header, **not** as a
    :mailheader:`Content-Transfer-Encoding` header. The mappings are table driven.
    Encoding suffixes are case sensitive; type suffixes are first tried case
    sensitively, then case insensitively.
 
-   Optional *strict* is a flag specifying whether the list of known MIME types
+   The optional *strict* argument is a flag specifying whether the list of known MIME types
    is limited to only the official types `registered with IANA
-   <http://www.iana.org/assignments/media-types/>`_ are recognized.
-   When *strict* is true (the default), only the IANA types are supported; when
-   *strict* is false, some additional non-standard but commonly used MIME types
+   <http://www.iana.org/assignments/media-types/>`_.
+   When *strict* is ``True`` (the default), only the IANA types are supported; when
+   *strict* is ``False``, some additional non-standard but commonly used MIME types
    are also recognized.
 
 
-.. function:: guess_all_extensions(type[, strict])
+.. function:: guess_all_extensions(type, strict=True)
 
    Guess the extensions for a file based on its MIME type, given by *type*. The
    return value is a list of strings giving all possible filename extensions,
@@ -59,25 +58,25 @@
    been associated with any particular data stream, but would be mapped to the MIME
    type *type* by :func:`guess_type`.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 
-.. function:: guess_extension(type[, strict])
+.. function:: guess_extension(type, strict=True)
 
    Guess the extension for a file based on its MIME type, given by *type*. The
    return value is a string giving a filename extension, including the leading dot
    (``'.'``).  The extension is not guaranteed to have been associated with any
-   particular data stream, but would be mapped to the  MIME type *type* by
+   particular data stream, but would be mapped to the MIME type *type* by
    :func:`guess_type`.  If no extension can be guessed for *type*, ``None`` is
    returned.
 
-   Optional *strict* has the same meaning as with the :func:`guess_type` function.
+   The optional *strict* argument has the same meaning as with the :func:`guess_type` function.
 
 Some additional functions and data items are available for controlling the
 behavior of the module.
 
 
-.. function:: init([files])
+.. function:: init(files=None)
 
    Initialize the internal data structures.  If given, *files* must be a sequence
    of file names which should be used to augment the default type map.  If omitted,
@@ -92,26 +91,26 @@
 
 .. function:: read_mime_types(filename)
 
-   Load the type map given in the file *filename*, if it exists.  The  type map is
+   Load the type map given in the file *filename*, if it exists.  The type map is
    returned as a dictionary mapping filename extensions, including the leading dot
    (``'.'``), to strings of the form ``'type/subtype'``.  If the file *filename*
    does not exist or cannot be read, ``None`` is returned.
 
 
-.. function:: add_type(type, ext[, strict])
+.. function:: add_type(type, ext, strict=True)
 
-   Add a mapping from the mimetype *type* to the extension *ext*. When the
+   Add a mapping from the MIME type *type* to the extension *ext*. When the
    extension is already known, the new type will replace the old one. When the type
    is already known the extension will be added to the list of known extensions.
 
-   When *strict* is True (the default), the mapping will added to the official MIME
+   When *strict* is ``True`` (the default), the mapping will added to the official MIME
    types, otherwise to the non-standard ones.
 
 
 .. data:: inited
 
    Flag indicating whether or not the global data structures have been initialized.
-   This is set to true by :func:`init`.
+   This is set to ``True`` by :func:`init`.
 
 
 .. data:: knownfiles
@@ -146,23 +145,6 @@
    Dictionary mapping filename extensions to non-standard, but commonly found MIME
    types.
 
-The :class:`MimeTypes` class may be useful for applications which may want more
-than one MIME-type database:
-
-
-.. class:: MimeTypes([filenames])
-
-   This class represents a MIME-types database.  By default, it provides access to
-   the same database as the rest of this module. The initial database is a copy of
-   that provided by the module, and may be extended by loading additional
-   :file:`mime.types`\ -style files into the database using the :meth:`read` or
-   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
-   loading additional data if the default data is not desired.
-
-   The optional *filenames* parameter can be used to cause additional files to be
-   loaded "on top" of the default database.
-
-   .. versionadded:: 2.2
 
 An example usage of the module::
 
@@ -183,70 +165,96 @@
 MimeTypes Objects
 -----------------
 
-:class:`MimeTypes` instances provide an interface which is very like that of the
+The :class:`MimeTypes` class may be useful for applications which may want more
+than one MIME-type database; it provides an interface similar to the one of the
 :mod:`mimetypes` module.
 
 
+.. class:: MimeTypes(filenames=(), strict=True)
+
+   This class represents a MIME-types database.  By default, it provides access to
+   the same database as the rest of this module. The initial database is a copy of
+   that provided by the module, and may be extended by loading additional
+   :file:`mime.types`\ -style files into the database using the :meth:`read` or
+   :meth:`readfp` methods.  The mapping dictionaries may also be cleared before
+   loading additional data if the default data is not desired.
+
+   The optional *filenames* parameter can be used to cause additional files to be
+   loaded "on top" of the default database.
+
+
 .. attribute:: MimeTypes.suffix_map
 
    Dictionary mapping suffixes to suffixes.  This is used to allow recognition of
    encoded files for which the encoding and the type are indicated by the same
    extension.  For example, the :file:`.tgz` extension is mapped to :file:`.tar.gz`
    to allow the encoding and type to be recognized separately.  This is initially a
-   copy of the global ``suffix_map`` defined in the module.
+   copy of the global :data:`suffix_map` defined in the module.
 
 
 .. attribute:: MimeTypes.encodings_map
 
    Dictionary mapping filename extensions to encoding types.  This is initially a
-   copy of the global ``encodings_map`` defined in the module.
+   copy of the global :data:`encodings_map` defined in the module.
 
 
 .. attribute:: MimeTypes.types_map
 
-   Dictionary mapping filename extensions to MIME types.  This is initially a copy
-   of the global ``types_map`` defined in the module.
+   Tuple containing two dictionaries, mapping filename extensions to MIME types:
+   the first dictionary is for the non-standards types and the second one is for
+   the standard types. They are initialized by :data:`common_types` and
+   :data:`types_map`.
 
 
-.. attribute:: MimeTypes.common_types
+.. attribute:: MimeTypes.types_map_inv
 
-   Dictionary mapping filename extensions to non-standard, but commonly found MIME
-   types.  This is initially a copy of the global ``common_types`` defined in the
-   module.
+   Tuple containing two dictionaries, mapping MIME types to a list of filename
+   extensions: the first dictionary is for the non-standards types and the
+   second one is for the standard types. They are initialized by
+   :data:`common_types` and :data:`types_map`.
 
 
-.. method:: MimeTypes.guess_extension(type[, strict])
+.. method:: MimeTypes.guess_extension(type, strict=True)
 
    Similar to the :func:`guess_extension` function, using the tables stored as part
    of the object.
 
 
-.. method:: MimeTypes.guess_all_extensions(type[, strict])
-
-   Similar to the :func:`guess_all_extensions` function, using the tables stored as part
-   of the object.
-
-
-.. method:: MimeTypes.guess_type(url[, strict])
+.. method:: MimeTypes.guess_type(url, strict=True)
 
    Similar to the :func:`guess_type` function, using the tables stored as part of
    the object.
 
 
-.. method:: MimeTypes.read(path)
+.. method:: MimeTypes.guess_all_extensions(type, strict=True)
 
-   Load MIME information from a file named *path*.  This uses :meth:`readfp` to
+   Similar to the :func:`guess_all_extensions` function, using the tables stored
+   as part of the object.
+
+
+.. method:: MimeTypes.read(filename, strict=True)
+
+   Load MIME information from a file named *filename*.  This uses :meth:`readfp` to
    parse the file.
 
+   If *strict* is ``True``, information will be added to list of standard types,
+   else to the list of non-standard types.
 
-.. method:: MimeTypes.readfp(file)
 
-   Load MIME type information from an open file.  The file must have the format of
+.. method:: MimeTypes.readfp(fp, strict=True)
+
+   Load MIME type information from an open file *fp*.  The file must have the format of
    the standard :file:`mime.types` files.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
 
-.. method:: MimeTypes.read_windows_registry()
+
+.. method:: MimeTypes.read_windows_registry(strict=True)
 
    Load MIME type information from the Windows registry.  Availability: Windows.
 
+   If *strict* is ``True``, information will be added to the list of standard
+   types, else to the list of non-standard types.
+
    .. versionadded:: 2.7

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 04:15:24 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 04:15:24 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_News_entry_for_?=
	=?utf8?q?Issue12798?=
Message-ID: <E1RdvbQ-0007kr-IX@dinsdale.python.org>

http://hg.python.org/cpython/rev/9c19df6c8ea0
changeset:   74135:9c19df6c8ea0
branch:      3.2
parent:      74132:aef79ff1bc9b
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 11:11:21 2011 +0800
summary:
  News entry for Issue12798

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,8 @@
 Library
 -------
 
+- Issue #12798: Updated the mimetypes documentation.
+
 - Issue #11006: Don't issue low level warning in subprocess when pipe2() fails.
 
 - Issue #11829: Fix code execution holes in inspect.getattr_static for

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 04:15:25 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 04:15:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_merge_from_3=2E2=2E_News_update=2E?=
Message-ID: <E1RdvbR-0007kw-Az@dinsdale.python.org>

http://hg.python.org/cpython/rev/854e9730da6b
changeset:   74136:854e9730da6b
parent:      74133:4b306aee21a4
parent:      74135:9c19df6c8ea0
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 11:14:17 2011 +0800
summary:
  merge from 3.2. News update.

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -419,6 +419,8 @@
 Library
 -------
 
+- Issue #12798: Updated the mimetypes documentation.
+
 - Issue #13626: Add support for SSL Diffie-Hellman key exchange, through the
   SSLContext.load_dh_params() method and the ssl.OP_SINGLE_DH_USE option.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 04:15:26 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 04:15:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Update_News_ent?=
	=?utf8?b?cnku?=
Message-ID: <E1RdvbS-0007l0-Dj@dinsdale.python.org>

http://hg.python.org/cpython/rev/bdec4cadea8b
changeset:   74137:bdec4cadea8b
branch:      2.7
parent:      74134:cfa3fe9d7b1f
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 11:14:56 2011 +0800
summary:
  Update News entry.

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -86,6 +86,8 @@
 Library
 -------
 
+- Issue #12798: Updated the mimetypes documentation.
+
 - Issue #13639: Accept unicode filenames in tarfile.open(mode="w|gz").
 
 - Issue #1785: Fix inspect and pydoc with misbehaving descriptors.

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Dec 23 05:31:29 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 23 Dec 2011 05:31:29 +0100
Subject: [Python-checkins] Daily reference leaks (475dc8f3cc6a): sum=0
Message-ID: <E1Rdwn3-0007Rn-Sn@ap.vmr.nerim.net>

results for 475dc8f3cc6a on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog_wH2Po', '-x']

From python-checkins at python.org  Fri Dec 23 10:09:57 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 10:09:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Minor_code_styl?=
 =?utf8?q?e_improvements_in_http=2Eserver_suggested_in_Issue13294=2E?=
Message-ID: <E1Re18X-0006PU-Hd@dinsdale.python.org>

http://hg.python.org/cpython/rev/2c1720468dbb
changeset:   74138:2c1720468dbb
branch:      3.2
parent:      74135:9c19df6c8ea0
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 17:03:41 2011 +0800
summary:
  Minor code style improvements in http.server suggested in Issue13294.

files:
  Lib/http/server.py |  9 +++------
  1 files changed, 3 insertions(+), 6 deletions(-)


diff --git a/Lib/http/server.py b/Lib/http/server.py
--- a/Lib/http/server.py
+++ b/Lib/http/server.py
@@ -271,14 +271,11 @@
         self.request_version = version = self.default_request_version
         self.close_connection = 1
         requestline = str(self.raw_requestline, 'iso-8859-1')
-        if requestline[-2:] == '\r\n':
-            requestline = requestline[:-2]
-        elif requestline[-1:] == '\n':
-            requestline = requestline[:-1]
+        requestline = requestline.rstrip('\r\n')
         self.requestline = requestline
         words = requestline.split()
         if len(words) == 3:
-            [command, path, version] = words
+            command, path, version = words
             if version[:5] != 'HTTP/':
                 self.send_error(400, "Bad request version (%r)" % version)
                 return False
@@ -304,7 +301,7 @@
                           "Invalid HTTP Version (%s)" % base_version_number)
                 return False
         elif len(words) == 2:
-            [command, path] = words
+            command, path = words
             self.close_connection = 1
             if command != 'GET':
                 self.send_error(400,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 10:09:58 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 10:09:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_merge_from_3=2E2=2E__Minor_code_style_improvements_in_http?=
 =?utf8?q?=2Eserver_suggested_in?=
Message-ID: <E1Re18Y-0006Pd-C8@dinsdale.python.org>

http://hg.python.org/cpython/rev/0466ee1816b1
changeset:   74139:0466ee1816b1
parent:      74136:854e9730da6b
parent:      74138:2c1720468dbb
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 17:04:23 2011 +0800
summary:
  merge from 3.2.  Minor code style improvements in http.server suggested in Issue13294.

files:
  Lib/http/server.py |  9 +++------
  1 files changed, 3 insertions(+), 6 deletions(-)


diff --git a/Lib/http/server.py b/Lib/http/server.py
--- a/Lib/http/server.py
+++ b/Lib/http/server.py
@@ -272,14 +272,11 @@
         self.request_version = version = self.default_request_version
         self.close_connection = 1
         requestline = str(self.raw_requestline, 'iso-8859-1')
-        if requestline[-2:] == '\r\n':
-            requestline = requestline[:-2]
-        elif requestline[-1:] == '\n':
-            requestline = requestline[:-1]
+        requestline = requestline.rstrip('\r\n')
         self.requestline = requestline
         words = requestline.split()
         if len(words) == 3:
-            [command, path, version] = words
+            command, path, version = words
             if version[:5] != 'HTTP/':
                 self.send_error(400, "Bad request version (%r)" % version)
                 return False
@@ -305,7 +302,7 @@
                           "Invalid HTTP Version (%s)" % base_version_number)
                 return False
         elif len(words) == 2:
-            [command, path] = words
+            command, path = words
             self.close_connection = 1
             if command != 'GET':
                 self.send_error(400,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 10:09:59 2011
From: python-checkins at python.org (senthil.kumaran)
Date: Fri, 23 Dec 2011 10:09:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_port_to_2=2E7_-?=
 =?utf8?q?_Minor_code_style_improvements_in_http=2Eserver_suggested_in?=
Message-ID: <E1Re18Z-0006Pi-7y@dinsdale.python.org>

http://hg.python.org/cpython/rev/1b1818fee351
changeset:   74140:1b1818fee351
branch:      2.7
parent:      74137:bdec4cadea8b
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Fri Dec 23 17:07:13 2011 +0800
summary:
  port to 2.7 - Minor code style improvements in http.server suggested in Issue13294.

files:
  Lib/BaseHTTPServer.py |  9 +++------
  1 files changed, 3 insertions(+), 6 deletions(-)


diff --git a/Lib/BaseHTTPServer.py b/Lib/BaseHTTPServer.py
--- a/Lib/BaseHTTPServer.py
+++ b/Lib/BaseHTTPServer.py
@@ -244,14 +244,11 @@
         self.request_version = version = self.default_request_version
         self.close_connection = 1
         requestline = self.raw_requestline
-        if requestline[-2:] == '\r\n':
-            requestline = requestline[:-2]
-        elif requestline[-1:] == '\n':
-            requestline = requestline[:-1]
+        requestline = requestline.rstrip('\r\n')
         self.requestline = requestline
         words = requestline.split()
         if len(words) == 3:
-            [command, path, version] = words
+            command, path, version = words
             if version[:5] != 'HTTP/':
                 self.send_error(400, "Bad request version (%r)" % version)
                 return False
@@ -277,7 +274,7 @@
                           "Invalid HTTP Version (%s)" % base_version_number)
                 return False
         elif len(words) == 2:
-            [command, path] = words
+            command, path = words
             self.close_connection = 1
             if command != 'GET':
                 self.send_error(400,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 12:41:06 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 23 Dec 2011 12:41:06 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2313577=3A_Built-in_?=
 =?utf8?q?methods_and_functions_now_have_a_=5F=5Fqualname=5F=5F=2E?=
Message-ID: <E1Re3Uo-0000SY-7l@dinsdale.python.org>

http://hg.python.org/cpython/rev/7a7b698f6f21
changeset:   74141:7a7b698f6f21
parent:      74139:0466ee1816b1
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 23 12:40:16 2011 +0100
summary:
  Issue #13577: Built-in methods and functions now have a __qualname__.
Patch by sbt.

files:
  Include/methodobject.h     |   3 +-
  Lib/test/test_funcattrs.py |  28 +++++++++++++++-
  Misc/NEWS                  |   3 +
  Objects/methodobject.c     |  44 +++++++++++++++++++++++--
  Objects/typeobject.c       |   2 +-
  5 files changed, 73 insertions(+), 7 deletions(-)


diff --git a/Include/methodobject.h b/Include/methodobject.h
--- a/Include/methodobject.h
+++ b/Include/methodobject.h
@@ -30,7 +30,8 @@
 #define PyCFunction_GET_FUNCTION(func) \
         (((PyCFunctionObject *)func) -> m_ml -> ml_meth)
 #define PyCFunction_GET_SELF(func) \
-	(((PyCFunctionObject *)func) -> m_self)
+        (((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_STATIC ? \
+         NULL : ((PyCFunctionObject *)func) -> m_self)
 #define PyCFunction_GET_FLAGS(func) \
 	(((PyCFunctionObject *)func) -> m_ml -> ml_flags)
 #endif
diff --git a/Lib/test/test_funcattrs.py b/Lib/test/test_funcattrs.py
--- a/Lib/test/test_funcattrs.py
+++ b/Lib/test/test_funcattrs.py
@@ -342,11 +342,37 @@
         self.assertTrue(s.__func__ is f)
 
 
+class BuiltinFunctionPropertiesTest(unittest.TestCase):
+    # XXX Not sure where this should really go since I can't find a
+    # test module specifically for builtin_function_or_method.
+
+    def test_builtin__qualname__(self):
+        import time
+
+        # builtin function:
+        self.assertEqual(len.__qualname__, 'len')
+        self.assertEqual(time.time.__qualname__, 'time')
+
+        # builtin classmethod:
+        self.assertEqual(dict.fromkeys.__qualname__, 'dict.fromkeys')
+        self.assertEqual(float.__getformat__.__qualname__,
+                         'float.__getformat__')
+
+        # builtin staticmethod:
+        self.assertEqual(str.maketrans.__qualname__, 'str.maketrans')
+        self.assertEqual(bytes.maketrans.__qualname__, 'bytes.maketrans')
+
+        # builtin bound instance method:
+        self.assertEqual([1, 2, 3].append.__qualname__, 'list.append')
+        self.assertEqual({'foo': 'bar'}.pop.__qualname__, 'dict.pop')
+
+
 def test_main():
     support.run_unittest(FunctionPropertiesTest, InstancemethodAttrTest,
                               ArbitraryFunctionAttrTest, FunctionDictsTest,
                               FunctionDocstringTest, CellTest,
-                              StaticMethodAttrsTest)
+                              StaticMethodAttrsTest,
+                              BuiltinFunctionPropertiesTest)
 
 if __name__ == "__main__":
     test_main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #13577: Built-in methods and functions now have a __qualname__.
+  Patch by sbt.
+
 - Issue #6695: Full garbage collection runs now clear the freelist of set
   objects.  Initial patch by Matthias Troffaes.
 
diff --git a/Objects/methodobject.c b/Objects/methodobject.c
--- a/Objects/methodobject.c
+++ b/Objects/methodobject.c
@@ -44,7 +44,7 @@
         PyErr_BadInternalCall();
         return NULL;
     }
-    return ((PyCFunctionObject *)op) -> m_ml -> ml_meth;
+    return PyCFunction_GET_FUNCTION(op);
 }
 
 PyObject *
@@ -54,7 +54,7 @@
         PyErr_BadInternalCall();
         return NULL;
     }
-    return ((PyCFunctionObject *)op) -> m_self;
+    return PyCFunction_GET_SELF(op);
 }
 
 int
@@ -64,7 +64,7 @@
         PyErr_BadInternalCall();
         return -1;
     }
-    return ((PyCFunctionObject *)op) -> m_ml -> ml_flags;
+    return PyCFunction_GET_FLAGS(op);
 }
 
 PyObject *
@@ -151,6 +151,41 @@
     return PyUnicode_FromString(m->m_ml->ml_name);
 }
 
+static PyObject *
+meth_get__qualname__(PyCFunctionObject *m, void *closure)
+{
+    /* If __self__ is a module or NULL, return m.__name__
+       (e.g. len.__qualname__ == 'len')
+
+       If __self__ is a type, return m.__self__.__qualname__ + '.' + m.__name__
+       (e.g. dict.fromkeys.__qualname__ == 'dict.fromkeys')
+
+       Otherwise return type(m.__self__).__qualname__ + '.' + m.__name__
+       (e.g. [].append.__qualname__ == 'list.append') */
+    PyObject *type, *type_qualname, *res;
+    _Py_IDENTIFIER(__qualname__);
+
+    if (m->m_self == NULL || PyModule_Check(m->m_self))
+        return PyUnicode_FromString(m->m_ml->ml_name);
+
+    type = PyType_Check(m->m_self) ? m->m_self : (PyObject*)Py_TYPE(m->m_self);
+
+    type_qualname = _PyObject_GetAttrId(type, &PyId___qualname__);
+    if (type_qualname == NULL)
+        return NULL;
+
+    if (!PyUnicode_Check(type_qualname)) {
+        PyErr_SetString(PyExc_TypeError, "<method>.__class__."
+                        "__qualname__ is not a unicode object");
+        Py_XDECREF(type_qualname);
+        return NULL;
+    }
+
+    res = PyUnicode_FromFormat("%S.%s", type_qualname, m->m_ml->ml_name);
+    Py_DECREF(type_qualname);
+    return res;
+}
+
 static int
 meth_traverse(PyCFunctionObject *m, visitproc visit, void *arg)
 {
@@ -164,7 +199,7 @@
 {
     PyObject *self;
 
-    self = m->m_self;
+    self = PyCFunction_GET_SELF(m);
     if (self == NULL)
         self = Py_None;
     Py_INCREF(self);
@@ -174,6 +209,7 @@
 static PyGetSetDef meth_getsets [] = {
     {"__doc__",  (getter)meth_get__doc__,  NULL, NULL},
     {"__name__", (getter)meth_get__name__, NULL, NULL},
+    {"__qualname__", (getter)meth_get__qualname__, NULL, NULL},
     {"__self__", (getter)meth_get__self__, NULL, NULL},
     {0}
 };
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -3726,7 +3726,7 @@
             descr = PyDescr_NewClassMethod(type, meth);
         }
         else if (meth->ml_flags & METH_STATIC) {
-            PyObject *cfunc = PyCFunction_New(meth, NULL);
+            PyObject *cfunc = PyCFunction_New(meth, (PyObject*)type);
             if (cfunc == NULL)
                 return -1;
             descr = PyStaticMethod_New(cfunc);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 16:45:11 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Fri, 23 Dec 2011 16:45:11 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=238623=3A_Fix_some_s?=
 =?utf8?q?trict-aliasing_warnings=2E_Patch_by_David_Watson=2E?=
Message-ID: <E1Re7J1-0004JV-SX@dinsdale.python.org>

http://hg.python.org/cpython/rev/683a1b1ff15d
changeset:   74142:683a1b1ff15d
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Fri Dec 23 16:44:51 2011 +0100
summary:
  Issue #8623: Fix some strict-aliasing warnings. Patch by David Watson.

files:
  Modules/socketmodule.c |  25 +++++++++----------------
  Modules/socketmodule.h |   1 +
  2 files changed, 10 insertions(+), 16 deletions(-)


diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -436,7 +436,8 @@
 #define SEGMENT_SIZE (32 * 1024 -1)
 #endif
 
-#define SAS2SA(x)       ((struct sockaddr *)(x))
+/* Convert "sock_addr_t *" to "struct sockaddr *". */
+#define SAS2SA(x)       (&((x)->sa))
 
 /*
  * Constants for getnameinfo()
@@ -4148,11 +4149,7 @@
 {
     char *name;
     struct hostent *h;
-#ifdef ENABLE_IPV6
-    struct sockaddr_storage addr;
-#else
-    struct sockaddr_in addr;
-#endif
+    sock_addr_t addr;
     struct sockaddr *sa;
     PyObject *ret = NULL;
 #ifdef HAVE_GETHOSTBYNAME_R
@@ -4171,7 +4168,7 @@
 
     if (!PyArg_ParseTuple(args, "et:gethostbyname_ex", "idna", &name))
         return NULL;
-    if (setipaddr(name, (struct sockaddr *)&addr, sizeof(addr), AF_INET) < 0)
+    if (setipaddr(name, SAS2SA(&addr), sizeof(addr), AF_INET) < 0)
         goto finally;
     Py_BEGIN_ALLOW_THREADS
 #ifdef HAVE_GETHOSTBYNAME_R
@@ -4196,8 +4193,8 @@
        addr.ss_family.
        Therefore, we cast the sockaddr_storage into sockaddr to
        access sa_family. */
-    sa = (struct sockaddr*)&addr;
-    ret = gethost_common(h, (struct sockaddr *)&addr, sizeof(addr),
+    sa = SAS2SA(&addr);
+    ret = gethost_common(h, SAS2SA(&addr), sizeof(addr),
                          sa->sa_family);
 #ifdef USE_GETHOSTBYNAME_LOCK
     PyThread_release_lock(netdb_lock);
@@ -4220,12 +4217,8 @@
 static PyObject *
 socket_gethostbyaddr(PyObject *self, PyObject *args)
 {
-#ifdef ENABLE_IPV6
-    struct sockaddr_storage addr;
-#else
-    struct sockaddr_in addr;
-#endif
-    struct sockaddr *sa = (struct sockaddr *)&addr;
+    sock_addr_t addr;
+    struct sockaddr *sa = SAS2SA(&addr);
     char *ip_num;
     struct hostent *h;
     PyObject *ret = NULL;
@@ -4294,7 +4287,7 @@
     h = gethostbyaddr(ap, al, af);
 #endif /* HAVE_GETHOSTBYNAME_R */
     Py_END_ALLOW_THREADS
-    ret = gethost_common(h, (struct sockaddr *)&addr, sizeof(addr), af);
+    ret = gethost_common(h, SAS2SA(&addr), sizeof(addr), af);
 #ifdef USE_GETHOSTBYNAME_LOCK
     PyThread_release_lock(netdb_lock);
 #endif
diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h
--- a/Modules/socketmodule.h
+++ b/Modules/socketmodule.h
@@ -115,6 +115,7 @@
 /* Socket address */
 typedef union sock_addr {
     struct sockaddr_in in;
+    struct sockaddr sa;
 #ifdef AF_UNIX
     struct sockaddr_un un;
 #endif

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 19:10:52 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Fri, 23 Dec 2011 19:10:52 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEzNTY1?=
 =?utf8?q?=3A_Increase_multiprocessing=27s_server_socket_backlog=2C_to_avo?=
 =?utf8?q?id?=
Message-ID: <E1Re9a0-0007it-Fw@dinsdale.python.org>

http://hg.python.org/cpython/rev/94494a779c20
changeset:   74143:94494a779c20
branch:      2.7
parent:      74140:1b1818fee351
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Fri Dec 23 19:05:45 2011 +0100
summary:
  Issue #13565: Increase multiprocessing's server socket backlog, to avoid
dropped connections in case of simultaneous connection requests.

files:
  Lib/multiprocessing/managers.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -159,7 +159,7 @@
         Listener, Client = listener_client[serializer]
 
         # do authentication later
-        self.listener = Listener(address=address, backlog=5)
+        self.listener = Listener(address=address, backlog=16)
         self.address = self.listener.address
 
         self.id_to_obj = {'0': (None, ())}

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 19:10:53 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Fri, 23 Dec 2011 19:10:53 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNTY1?=
 =?utf8?q?=3A_Increase_multiprocessing=27s_server_socket_backlog=2C_to_avo?=
 =?utf8?q?id?=
Message-ID: <E1Re9a1-0007ix-9i@dinsdale.python.org>

http://hg.python.org/cpython/rev/9b99adef3c78
changeset:   74144:9b99adef3c78
branch:      3.2
parent:      74138:2c1720468dbb
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Fri Dec 23 19:06:48 2011 +0100
summary:
  Issue #13565: Increase multiprocessing's server socket backlog, to avoid
dropped connections in case of simultaneous connection requests.

files:
  Lib/multiprocessing/managers.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -162,7 +162,7 @@
         Listener, Client = listener_client[serializer]
 
         # do authentication later
-        self.listener = Listener(address=address, backlog=5)
+        self.listener = Listener(address=address, backlog=16)
         self.address = self.listener.address
 
         self.id_to_obj = {'0': (None, ())}

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 19:10:54 2011
From: python-checkins at python.org (charles-francois.natali)
Date: Fri, 23 Dec 2011 19:10:54 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313565=3A_Increase_multiprocessing=27s_server_socket?=
 =?utf8?q?_backlog=2C_to_avoid?=
Message-ID: <E1Re9a2-0007j1-PL@dinsdale.python.org>

http://hg.python.org/cpython/rev/29cad1ac828c
changeset:   74145:29cad1ac828c
parent:      74142:683a1b1ff15d
parent:      74144:9b99adef3c78
user:        Charles-Fran?ois Natali <neologix at free.fr>
date:        Fri Dec 23 19:07:58 2011 +0100
summary:
  Issue #13565: Increase multiprocessing's server socket backlog, to avoid
dropped connections in case of simultaneous connection requests.

files:
  Lib/multiprocessing/managers.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py
--- a/Lib/multiprocessing/managers.py
+++ b/Lib/multiprocessing/managers.py
@@ -158,7 +158,7 @@
         Listener, Client = listener_client[serializer]
 
         # do authentication later
-        self.listener = Listener(address=address, backlog=5)
+        self.listener = Listener(address=address, backlog=16)
         self.address = self.listener.address
 
         self.id_to_obj = {'0': (None, ())}

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 23 20:59:24 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 23 Dec 2011 20:59:24 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Add_whatsnew_items?=
Message-ID: <E1ReBH2-0006VZ-8z@dinsdale.python.org>

http://hg.python.org/cpython/rev/38df3e657ded
changeset:   74146:38df3e657ded
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 23 20:58:36 2011 +0100
summary:
  Add whatsnew items

files:
  Doc/whatsnew/3.3.rst |  12 ++++++++++++
  1 files changed, 12 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -639,12 +639,24 @@
 
   (Contributed by Adam Simpkins in :issue:`12803`)
 
+* Diffie-Hellman key exchange, both regular and Elliptic Curve-based, is
+  now supported through the :meth:`~ssl.SSLContext.load_dh_params` and
+  :meth:`~ssl.SSLContext.set_ecdh_curve` methods.
+
+  (Contributed by Antoine Pitrou in :issue:`13626` and :issue:`13627`)
+
 * SSL sockets have a new :meth:`~ssl.SSLSocket.get_channel_binding` method
   allowing the implementation of certain authentication mechanisms such as
   SCRAM-SHA-1-PLUS.
 
   (Contributed by Jacek Konieczny in :issue:`12551`)
 
+* You can query the SSL compression algorithm used by an SSL socket, thanks
+  to its new :meth:`~ssl.SSLSocket.compression` method.
+
+  (Contributed by Antoine Pitrou in :issue:`13634`)
+
+
 shutil
 ------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 03:02:24 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 24 Dec 2011 03:02:24 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_kill_superfluou?=
 =?utf8?q?s_=27comprehension=27_case_=28closes_=2313658=29?=
Message-ID: <E1ReGwK-0002on-Mz@dinsdale.python.org>

http://hg.python.org/cpython/rev/b65007ef59c0
changeset:   74147:b65007ef59c0
branch:      3.2
parent:      74144:9b99adef3c78
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 23 20:00:56 2011 -0600
summary:
  kill superfluous 'comprehension' case (closes #13658)

files:
  Doc/reference/compound_stmts.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -554,7 +554,7 @@
 
 .. productionlist::
    classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite`
-   inheritance: "(" [`argument_list` [","] | `comprehension`] ")"
+   inheritance: "(" [`parameter_list`] ")"
    classname: `identifier`
 
 A class definition is an executable statement.  The inheritance list usually

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 03:02:25 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 24 Dec 2011 03:02:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_parameter=5Flis?=
	=?utf8?q?t_was_meant_here?=
Message-ID: <E1ReGwL-0002or-EL@dinsdale.python.org>

http://hg.python.org/cpython/rev/71ff2235bb4c
changeset:   74148:71ff2235bb4c
branch:      3.2
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 23 20:01:43 2011 -0600
summary:
  parameter_list was meant here

files:
  Doc/reference/compound_stmts.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -427,7 +427,7 @@
 .. productionlist::
    funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" ["->" `expression`] ":" `suite`
    decorators: `decorator`+
-   decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE
+   decorator: "@" `dotted_name` ["(" [`parameter_list` [","]] ")"] NEWLINE
    dotted_name: `identifier` ("." `identifier`)*
    parameter_list: (`defparameter` ",")*
                  : (  "*" [`parameter`] ("," `defparameter`)*

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 03:02:26 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 24 Dec 2011 03:02:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1ReGwM-0002ov-69@dinsdale.python.org>

http://hg.python.org/cpython/rev/cd459def1446
changeset:   74149:cd459def1446
parent:      74146:38df3e657ded
parent:      74148:71ff2235bb4c
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 23 20:02:12 2011 -0600
summary:
  merge 3.2

files:
  Doc/reference/compound_stmts.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -427,7 +427,7 @@
 .. productionlist::
    funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" ["->" `expression`] ":" `suite`
    decorators: `decorator`+
-   decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE
+   decorator: "@" `dotted_name` ["(" [`parameter_list` [","]] ")"] NEWLINE
    dotted_name: `identifier` ("." `identifier`)*
    parameter_list: (`defparameter` ",")*
                  : (  "*" [`parameter`] ("," `defparameter`)*
@@ -554,7 +554,7 @@
 
 .. productionlist::
    classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite`
-   inheritance: "(" [`argument_list` [","] | `comprehension`] ")"
+   inheritance: "(" [`parameter_list`] ")"
    classname: `identifier`
 
 A class definition is an executable statement.  The inheritance list usually

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Dec 24 05:30:33 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 24 Dec 2011 05:30:33 +0100
Subject: [Python-checkins] Daily reference leaks (cd459def1446): sum=0
Message-ID: <E1ReJFh-0007fM-EK@ap.vmr.nerim.net>

results for cd459def1446 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogkPwXNv', '-x']

From python-checkins at python.org  Sat Dec 24 06:14:30 2011
From: python-checkins at python.org (meador.inge)
Date: Sat, 24 Dec 2011 06:14:30 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzEzNjMy?=
 =?utf8?q?=3A_Update_token_documentation_to_reflect_actual_token_types?=
Message-ID: <E1ReJwE-000535-2M@dinsdale.python.org>

http://hg.python.org/cpython/rev/b7d099e8c136
changeset:   74150:b7d099e8c136
branch:      3.2
parent:      74148:71ff2235bb4c
user:        Meador Inge <meadori at gmail.com>
date:        Fri Dec 23 22:30:16 2011 -0600
summary:
  Issue #13632: Update token documentation to reflect actual token types

files:
  Doc/library/token.rst |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Doc/library/token.rst b/Doc/library/token.rst
--- a/Doc/library/token.rst
+++ b/Doc/library/token.rst
@@ -68,7 +68,6 @@
           EQUAL
           DOT
           PERCENT
-          BACKQUOTE
           LBRACE
           RBRACE
           EQEQUAL
@@ -94,6 +93,8 @@
           DOUBLESLASH
           DOUBLESLASHEQUAL
           AT
+          RARROW
+          ELLIPSIS
           OP
           ERRORTOKEN
           N_TOKENS

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 06:14:31 2011
From: python-checkins at python.org (meador.inge)
Date: Sat, 24 Dec 2011 06:14:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Issue_=2313632=3A_Update_token_documentation_to_reflect_actu?=
 =?utf8?q?al_token_types?=
Message-ID: <E1ReJwF-000539-46@dinsdale.python.org>

http://hg.python.org/cpython/rev/1461327e63b5
changeset:   74151:1461327e63b5
parent:      74149:cd459def1446
parent:      74150:b7d099e8c136
user:        Meador Inge <meadori at gmail.com>
date:        Fri Dec 23 23:08:50 2011 -0600
summary:
  Issue #13632: Update token documentation to reflect actual token types

files:
  Doc/library/token.rst |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Doc/library/token.rst b/Doc/library/token.rst
--- a/Doc/library/token.rst
+++ b/Doc/library/token.rst
@@ -68,7 +68,6 @@
           EQUAL
           DOT
           PERCENT
-          BACKQUOTE
           LBRACE
           RBRACE
           EQEQUAL
@@ -94,6 +93,8 @@
           DOUBLESLASH
           DOUBLESLASHEQUAL
           AT
+          RARROW
+          ELLIPSIS
           OP
           ERRORTOKEN
           N_TOKENS

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 15:01:25 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 15:01:25 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_correct_syslog?=
 =?utf8?q?=2Eopenlog=28=29_argument_name_to_logoption=3B_thanks_to_Arnaud?=
Message-ID: <E1ReSA9-0000WN-8T@dinsdale.python.org>

http://hg.python.org/cpython/rev/71f7175e2b34
changeset:   74152:71f7175e2b34
branch:      2.7
parent:      74143:94494a779c20
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 14:51:26 2011 +0100
summary:
  correct syslog.openlog() argument name to logoption; thanks to Arnaud Gomes-do-Vale from docs@

files:
  Doc/library/syslog.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst
--- a/Doc/library/syslog.rst
+++ b/Doc/library/syslog.rst
@@ -30,7 +30,7 @@
    ``openlog()`` will be called with no arguments.
 
 
-.. function:: openlog([ident[, logopt[, facility]]])
+.. function:: openlog([ident[, logoption[, facility]]])
 
    Logging options of subsequent :func:`syslog` calls can be set by calling
    :func:`openlog`.  :func:`syslog` will call :func:`openlog` with no arguments
@@ -38,7 +38,7 @@
 
    The optional *ident* keyword argument is a string which is prepended to every
    message, and defaults to ``sys.argv[0]`` with leading path components
-   stripped.  The optional *logopt* keyword argument (default is 0) is a bit
+   stripped.  The optional *logoption* keyword argument (default is 0) is a bit
    field -- see below for possible values to combine.  The optional *facility*
    keyword argument (default is :const:`LOG_USER`) sets the default facility for
    messages which do not have a facility explicitly encoded.
@@ -98,5 +98,5 @@
 logged messages, and write the messages to the destination facility used for
 mail logging::
 
-   syslog.openlog(logopt=syslog.LOG_PID, facility=syslog.LOG_MAIL)
+   syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)
    syslog.syslog('E-mail processing initiated...')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 15:01:26 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 15:01:26 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_correct_syslog?=
 =?utf8?q?=2Eopenlog=28=29_argument_name_to_logoption=3B_thanks_to_Arnaud?=
Message-ID: <E1ReSAA-0000Wz-0l@dinsdale.python.org>

http://hg.python.org/cpython/rev/ba2f6978502e
changeset:   74153:ba2f6978502e
branch:      3.2
parent:      74150:b7d099e8c136
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 14:51:49 2011 +0100
summary:
  correct syslog.openlog() argument name to logoption; thanks to Arnaud Gomes-do-Vale from docs@

files:
  Doc/library/syslog.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst
--- a/Doc/library/syslog.rst
+++ b/Doc/library/syslog.rst
@@ -30,7 +30,7 @@
    ``openlog()`` will be called with no arguments.
 
 
-.. function:: openlog([ident[, logopt[, facility]]])
+.. function:: openlog([ident[, logoption[, facility]]])
 
    Logging options of subsequent :func:`syslog` calls can be set by calling
    :func:`openlog`.  :func:`syslog` will call :func:`openlog` with no arguments
@@ -38,7 +38,7 @@
 
    The optional *ident* keyword argument is a string which is prepended to every
    message, and defaults to ``sys.argv[0]`` with leading path components
-   stripped.  The optional *logopt* keyword argument (default is 0) is a bit
+   stripped.  The optional *logoption* keyword argument (default is 0) is a bit
    field -- see below for possible values to combine.  The optional *facility*
    keyword argument (default is :const:`LOG_USER`) sets the default facility for
    messages which do not have a facility explicitly encoded.
@@ -103,5 +103,5 @@
 logged messages, and write the messages to the destination facility used for
 mail logging::
 
-   syslog.openlog(logopt=syslog.LOG_PID, facility=syslog.LOG_MAIL)
+   syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)
    syslog.syslog('E-mail processing initiated...')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 15:01:27 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 15:01:27 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ReSAB-0000X7-Nf@dinsdale.python.org>

http://hg.python.org/cpython/rev/ea4ff5528e40
changeset:   74154:ea4ff5528e40
parent:      74151:1461327e63b5
parent:      74153:ba2f6978502e
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 14:52:19 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/syslog.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst
--- a/Doc/library/syslog.rst
+++ b/Doc/library/syslog.rst
@@ -30,7 +30,7 @@
    ``openlog()`` will be called with no arguments.
 
 
-.. function:: openlog([ident[, logopt[, facility]]])
+.. function:: openlog([ident[, logoption[, facility]]])
 
    Logging options of subsequent :func:`syslog` calls can be set by calling
    :func:`openlog`.  :func:`syslog` will call :func:`openlog` with no arguments
@@ -38,7 +38,7 @@
 
    The optional *ident* keyword argument is a string which is prepended to every
    message, and defaults to ``sys.argv[0]`` with leading path components
-   stripped.  The optional *logopt* keyword argument (default is 0) is a bit
+   stripped.  The optional *logoption* keyword argument (default is 0) is a bit
    field -- see below for possible values to combine.  The optional *facility*
    keyword argument (default is :const:`LOG_USER`) sets the default facility for
    messages which do not have a facility explicitly encoded.
@@ -103,5 +103,5 @@
 logged messages, and write the messages to the destination facility used for
 mail logging::
 
-   syslog.openlog(logopt=syslog.LOG_PID, facility=syslog.LOG_MAIL)
+   syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)
    syslog.syslog('E-mail processing initiated...')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 16:02:58 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 16:02:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_remove_spurious?=
 =?utf8?q?_dot_from_string_formatting_example=3B_thanks_to_Anthon_van_der?=
Message-ID: <E1ReT7i-0004tW-B9@dinsdale.python.org>

http://hg.python.org/cpython/rev/a1edc6e4b49c
changeset:   74155:a1edc6e4b49c
branch:      2.7
parent:      74152:71f7175e2b34
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 15:52:36 2011 +0100
summary:
  remove spurious dot from string formatting example; thanks to Anthon van der Neut from docs@

files:
  Doc/library/string.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/string.rst b/Doc/library/string.rst
--- a/Doc/library/string.rst
+++ b/Doc/library/string.rst
@@ -602,7 +602,7 @@
 
    >>> points = 19.5
    >>> total = 22
-   >>> 'Correct answers: {:.2%}.'.format(points/total)
+   >>> 'Correct answers: {:.2%}'.format(points/total)
    'Correct answers: 88.64%'
 
 Using type-specific formatting::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 16:02:59 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 16:02:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_remove_spurious?=
 =?utf8?q?_dot_from_string_formatting_example=3B_thanks_to_Anthon_van_der?=
Message-ID: <E1ReT7j-0004th-3f@dinsdale.python.org>

http://hg.python.org/cpython/rev/6be3cbbe5013
changeset:   74156:6be3cbbe5013
branch:      3.2
parent:      74153:ba2f6978502e
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 15:53:35 2011 +0100
summary:
  remove spurious dot from string formatting example; thanks to Anthon van der Neut from docs@

files:
  Doc/library/string.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/string.rst b/Doc/library/string.rst
--- a/Doc/library/string.rst
+++ b/Doc/library/string.rst
@@ -583,7 +583,7 @@
 
    >>> points = 19
    >>> total = 22
-   >>> 'Correct answers: {:.2%}.'.format(points/total)
+   >>> 'Correct answers: {:.2%}'.format(points/total)
    'Correct answers: 86.36%'
 
 Using type-specific formatting::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 16:02:59 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 16:02:59 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ReT7j-0004uE-Md@dinsdale.python.org>

http://hg.python.org/cpython/rev/b2f1518660a8
changeset:   74157:b2f1518660a8
parent:      74154:ea4ff5528e40
parent:      74156:6be3cbbe5013
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 15:53:51 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/string.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/string.rst b/Doc/library/string.rst
--- a/Doc/library/string.rst
+++ b/Doc/library/string.rst
@@ -583,7 +583,7 @@
 
    >>> points = 19
    >>> total = 22
-   >>> 'Correct answers: {:.2%}.'.format(points/total)
+   >>> 'Correct answers: {:.2%}'.format(points/total)
    'Correct answers: 86.36%'
 
 Using type-specific formatting::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 20:05:42 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 20:05:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_fix_Rather_that?=
 =?utf8?q?/Rather_than=3B_thanks_to_Reuben_Thomas_from_docs=40?=
Message-ID: <E1ReWuc-0004Wr-IT@dinsdale.python.org>

http://hg.python.org/cpython/rev/c560302d1a5e
changeset:   74158:c560302d1a5e
branch:      2.7
parent:      74155:a1edc6e4b49c
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 19:55:49 2011 +0100
summary:
  fix Rather that/Rather than; thanks to Reuben Thomas from docs@

files:
  Doc/reference/simple_stmts.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -352,7 +352,7 @@
    del_stmt: "del" `target_list`
 
 Deletion is recursively defined very similar to the way assignment is defined.
-Rather that spelling it out in full details, here are some hints.
+Rather than spelling it out in full details, here are some hints.
 
 Deletion of a target list recursively deletes each target, from left to right.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 20:05:46 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 20:05:46 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_Rather_that?=
 =?utf8?q?/Rather_than=3B_thanks_to_Reuben_Thomas_from_docs=40?=
Message-ID: <E1ReWug-0004X1-Be@dinsdale.python.org>

http://hg.python.org/cpython/rev/b1d665781abf
changeset:   74159:b1d665781abf
branch:      3.2
parent:      74156:6be3cbbe5013
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 19:56:04 2011 +0100
summary:
  fix Rather that/Rather than; thanks to Reuben Thomas from docs@

files:
  Doc/reference/simple_stmts.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -372,7 +372,7 @@
    del_stmt: "del" `target_list`
 
 Deletion is recursively defined very similar to the way assignment is defined.
-Rather that spelling it out in full details, here are some hints.
+Rather than spelling it out in full details, here are some hints.
 
 Deletion of a target list recursively deletes each target, from left to right.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 20:05:49 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 20:05:49 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ReWuj-0004X6-1U@dinsdale.python.org>

http://hg.python.org/cpython/rev/1596eb147736
changeset:   74160:1596eb147736
parent:      74157:b2f1518660a8
parent:      74159:b1d665781abf
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 19:56:35 2011 +0100
summary:
  merge with 3.2

files:
  Doc/reference/simple_stmts.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -372,7 +372,7 @@
    del_stmt: "del" `target_list`
 
 Deletion is recursively defined very similar to the way assignment is defined.
-Rather that spelling it out in full details, here are some hints.
+Rather than spelling it out in full details, here are some hints.
 
 Deletion of a target list recursively deletes each target, from left to right.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 23:22:45 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 23:22:45 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_point_IronPytho?=
 =?utf8?q?n_to_ironpython=2Enet=3B_thanks_to_Lazar_Pancic_from_docs=40?=
Message-ID: <E1ReZzJ-0007Cr-G8@dinsdale.python.org>

http://hg.python.org/cpython/rev/ae5a0aa39e77
changeset:   74161:ae5a0aa39e77
branch:      2.7
parent:      74158:c560302d1a5e
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 23:13:08 2011 +0100
summary:
  point IronPython to ironpython.net; thanks to Lazar Pancic from docs@

files:
  Doc/reference/introduction.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst
--- a/Doc/reference/introduction.rst
+++ b/Doc/reference/introduction.rst
@@ -65,7 +65,7 @@
    An alternate Python for .NET.  Unlike Python.NET, this is a complete Python
    implementation that generates IL, and compiles Python code directly to .NET
    assemblies.  It was created by Jim Hugunin, the original creator of Jython.  For
-   more information, see `the IronPython website <http://www.ironpython.com/>`_.
+   more information, see `the IronPython website <http://www.ironpython.net/>`_.
 
 PyPy
    An implementation of Python written completely in Python. It supports several

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 23:22:46 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 23:22:46 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_point_IronPytho?=
 =?utf8?q?n_to_ironpython=2Enet=3B_thanks_to_Lazar_Pancic_from_docs=40?=
Message-ID: <E1ReZzK-0007D1-9L@dinsdale.python.org>

http://hg.python.org/cpython/rev/2f1f2b0c9d63
changeset:   74162:2f1f2b0c9d63
branch:      3.2
parent:      74159:b1d665781abf
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 23:13:19 2011 +0100
summary:
  point IronPython to ironpython.net; thanks to Lazar Pancic from docs@

files:
  Doc/reference/introduction.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst
--- a/Doc/reference/introduction.rst
+++ b/Doc/reference/introduction.rst
@@ -66,7 +66,7 @@
    An alternate Python for .NET.  Unlike Python.NET, this is a complete Python
    implementation that generates IL, and compiles Python code directly to .NET
    assemblies.  It was created by Jim Hugunin, the original creator of Jython.  For
-   more information, see `the IronPython website <http://www.ironpython.com/>`_.
+   more information, see `the IronPython website <http://www.ironpython.net/>`_.
 
 PyPy
    An implementation of Python written completely in Python. It supports several

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 24 23:22:47 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 24 Dec 2011 23:22:47 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ReZzL-0007D8-Bk@dinsdale.python.org>

http://hg.python.org/cpython/rev/c65790aff86d
changeset:   74163:c65790aff86d
parent:      74160:1596eb147736
parent:      74162:2f1f2b0c9d63
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 24 23:13:39 2011 +0100
summary:
  merge with 3.2

files:
  Doc/reference/introduction.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst
--- a/Doc/reference/introduction.rst
+++ b/Doc/reference/introduction.rst
@@ -66,7 +66,7 @@
    An alternate Python for .NET.  Unlike Python.NET, this is a complete Python
    implementation that generates IL, and compiles Python code directly to .NET
    assemblies.  It was created by Jim Hugunin, the original creator of Jython.  For
-   more information, see `the IronPython website <http://www.ironpython.com/>`_.
+   more information, see `the IronPython website <http://www.ironpython.net/>`_.
 
 PyPy
    An implementation of Python written completely in Python. It supports several

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 03:59:30 2011
From: python-checkins at python.org (ezio.melotti)
Date: Sun, 25 Dec 2011 03:59:30 +0100
Subject: [Python-checkins] =?utf8?q?devguide=3A_Update_documentation_about?=
 =?utf8?q?_a_couple_of_tracker_fields=2E?=
Message-ID: <E1ReeJ8-0004tq-72@dinsdale.python.org>

http://hg.python.org/devguide/rev/2f53e7d75186
changeset:   465:2f53e7d75186
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Dec 25 03:59:24 2011 +0100
summary:
  Update documentation about a couple of tracker fields.

files:
  triaging.rst |  8 +++-----
  1 files changed, 3 insertions(+), 5 deletions(-)


diff --git a/triaging.rst b/triaging.rst
--- a/triaging.rst
+++ b/triaging.rst
@@ -36,9 +36,11 @@
     should not be made public, please report it to security at python.org instead.
 performance
     Situations where too much time is necessary to complete the task.
-feature requests
+enhancement
     Issues that propose the addition of new functionality, such as new
     functions, classes, modules, or even new arguments for existing functions.
+    Also used for improvements in the documentation and test suite and for
+    other refactorings.
 
 Stage
 '''''
@@ -157,9 +159,6 @@
 ''''''''
 Various flags about the issue. Multiple values are possible.
 
-after moratorium
-    The issue is in regards to a language change which was not allowed during
-    the `language moratorium`_ in effect during the 3.2 development cycle.
 buildbot
     A buildbot triggered the issue being reported.
 easy
@@ -327,5 +326,4 @@
 .. _Developer's guide: http://hg.python.org/devguide/
 .. _GSoC: http://code.google.com/soc/
 .. _issue tracker: http://bugs.python.org
-.. _language moratorium: http://www.python.org/dev/peps/pep-3003/
 .. _meta tracker: http://psf.upfronthosting.co.za/roundup/meta/

-- 
Repository URL: http://hg.python.org/devguide

From python-checkins at python.org  Sun Dec 25 05:03:19 2011
From: python-checkins at python.org (ezio.melotti)
Date: Sun, 25 Dec 2011 05:03:19 +0100 (CET)
Subject: [Python-checkins] r88929 - in tracker/instances/python-dev/html:
	issue.item.js style.css
Message-ID: <3T9qYz2WwfzPqM@mail.python.org>

Author: ezio.melotti
Date: Sun Dec 25 05:03:18 2011
New Revision: 88929

Log:
#422: show an help popup for the keyboard shortcuts when ? is pressed.

Modified:
   tracker/instances/python-dev/html/issue.item.js
   tracker/instances/python-dev/html/style.css

Modified: tracker/instances/python-dev/html/issue.item.js
==============================================================================
--- tracker/instances/python-dev/html/issue.item.js	(original)
+++ tracker/instances/python-dev/html/issue.item.js	Sun Dec 25 05:03:18 2011
@@ -56,31 +56,53 @@
         // scroll the page to the given node
         window.scrollTo(0, node.offset().top)
     }
-    function add_help() {
-        // add some help to the sidebar
-        $(['<li><strong>Keyboard shortcuts</strong>',
-           '<ul id="help" class="level-three">',
-           '  <li><a>f/h: first msg</a></li>',
-           '  <li><a>p/k: previous msg</a></li>',
-           '  <li><a>n/j: next msg</a></li>',
-           '  <li><a>l: last msg</a></li>',
-           '  <li><a>r/i: focus textarea</a></li>',
-           '  <li><a>ESC: unfocus textarea</a></li>',
-           '</ul></div>'].join('\n')).appendTo('div#menu ul.level-two');
-        // the empty <a> are just an hack to get the style right,
-        // this help will anyway be moved to the devguide soon
+    function create_help_popup() {
+        // create a popup that lists the available shortcuts
+        var div = $([
+           '<div id="shortcuts-help"><table>',
+           '  <caption>Keyboard shortcuts</caption>',
+           '  <tr><th></th><th>mnemonic</th><th>vim-style</th></tr>',
+           '  <tr><th>first message</th><td>f</td><td>h</td></tr>',
+           '  <tr><th>previous message</th><td>p</td><td>k</td></tr>',
+           '  <tr><th>next message</th><td>n</td><td>j</td></tr>',
+           '  <tr><th>last message</th><td>l</td><td>l</td></tr>',
+           '  <tr><th>focus textarea</th><td>r</td><td>i</td></tr>',
+           '  <tr><th>unfocus textarea</th><td>Esc</td><td>Esc</td></tr>',
+           '  <tr><th>shortcuts help</th><td>?</td><td>?</td></tr>',
+           '</table>',
+           '<p>All these shortcuts can be used at any time to',
+           'navigate through the messages in the issue page.</p>',
+           '<span id="close-help">Close shortcuts help</span></div>'].join('\n'))
+        // this should point to the devguide once a section with the shortcuts
+        // is added
+        var menu = $('div#menu ul.level-three:last');
+        $('<li id="menu-shortcuts-help"><span>Keyb. shortcuts ' +
+          '(?)</span></li>').appendTo(menu);
+        menu.find('li#menu-shortcuts-help span').click(
+            function() { div.toggle(); return false; })
+        div.find('span').click(function() { div.hide() })
+        div.hide()
+        div.appendTo('body');
     }
-    add_help()
+    create_help_popup()
+
+    var help_popup = $('#shortcuts-help');
     var textarea = $('textarea').first();
     var messages = $('table.messages tr th a:first-child');
     var last_index = messages.length - 1;
     // start from -1 so 'n' sends to the first message at the beginning
     var current = -1;
     $(document).keydown(function (event) {
+        // '?' shows the help if the user is not editing the form.
+        // this is here because usually '?' requires 'shift' too, and the
+        // switch below is not reached when shift is pressed
+        if (event.shiftKey && (event.which == 191) && !is_editing(event.target)) {
+            help_popup.toggle()
+            return false;
+        }
         // do nothing if ctrl/alt/shift/meta are pressed
         if (event.ctrlKey || event.altKey || event.shiftKey || event.metaKey)
             return true;
-
         // disable the shortcuts while editing form elements (except ESC)
         if (is_editing(event.target)) {
             // ESC - unfocus the form
@@ -90,6 +112,11 @@
             }
             return true;
         }
+        // ESC hides the help if the user is not editing
+        if (event.keyCode == 27) {
+            help_popup.hide();
+            return false;
+        }
 
         // support two groups of shortcuts for first/prev/next/last/reply:
         //    mnemonics: f/p/n/l/r

Modified: tracker/instances/python-dev/html/style.css
==============================================================================
--- tracker/instances/python-dev/html/style.css	(original)
+++ tracker/instances/python-dev/html/style.css	Sun Dec 25 05:03:18 2011
@@ -531,3 +531,49 @@
     max-width: 20em;
     overflow: auto;
 }
+
+/* styles for the keyboard shortcut help popup */
+div#shortcuts-help {
+    background-color: #efefef;
+    border: 1px solid #afafaf;
+    position: fixed;
+    top: 6em;
+    left: 50%;
+    z-index: 42;
+    width: 400px;
+    margin-left: -200px;
+    padding: 1em;
+    text-align: center;
+    opacity: 0.95;
+}
+div#shortcuts-help table {
+    margin: 0 auto;
+    border: 1px solid #afafaf;
+}
+div#shortcuts-help caption {
+    font-weight: bold;
+    padding: .2em;
+}
+div#shortcuts-help th, div#shortcuts-help td {
+    text-align: center;
+    padding: .2em;
+    background-color: #dddddd;
+}
+div#shortcuts-help tr th:first-child {
+    text-align: right;
+}
+div#shortcuts-help p {
+    margin: 1em 2.5em 1.5em;
+    font-size: 80%;
+}
+div#shortcuts-help span {
+    cursor: pointer;
+    border: 1px solid #afafaf;
+    padding: .2em;
+    background-color: #dddddd;
+}
+li#menu-shortcuts-help span {
+    margin-left: .9em;
+    padding-left: 1em;
+    cursor: help;
+}

From solipsis at pitrou.net  Sun Dec 25 05:30:54 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 25 Dec 2011 05:30:54 +0100
Subject: [Python-checkins] Daily reference leaks (c65790aff86d): sum=0
Message-ID: <E1Refja-0007tN-DP@ap.vmr.nerim.net>

results for c65790aff86d on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogjTz5A_', '-x']

From python-checkins at python.org  Sun Dec 25 11:37:00 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:37:00 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_fix_Constuctor/?=
 =?utf8?q?Constructor_typo=3B_thanks_to_Anthon_van_der_Neut_from_docs=40?=
Message-ID: <E1RelRs-0004JB-1z@dinsdale.python.org>

http://hg.python.org/cpython/rev/eea6b76eb2fc
changeset:   74164:eea6b76eb2fc
branch:      2.7
parent:      74161:ae5a0aa39e77
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:27:22 2011 +0100
summary:
  fix Constuctor/Constructor typo; thanks to Anthon van der Neut from docs@

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -233,7 +233,7 @@
 detail in the :class:`Popen` constructor documentation.
 
 
-Popen Constuctor
+Popen Constructor
 ^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 11:37:01 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:37:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_Constuctor/?=
 =?utf8?q?Constructor_typo=3B_thanks_to_Anthon_van_der_Neut_from_docs=40?=
Message-ID: <E1RelRt-0004Jo-03@dinsdale.python.org>

http://hg.python.org/cpython/rev/e50af83296fd
changeset:   74165:e50af83296fd
branch:      3.2
parent:      74162:2f1f2b0c9d63
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:27:37 2011 +0100
summary:
  fix Constuctor/Constructor typo; thanks to Anthon van der Neut from docs@

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -239,7 +239,7 @@
 detail in the :class:`Popen` constructor documentation.
 
 
-Popen Constuctor
+Popen Constructor
 ^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 11:37:01 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:37:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1RelRt-0004Jv-Nu@dinsdale.python.org>

http://hg.python.org/cpython/rev/db2f80515807
changeset:   74166:db2f80515807
parent:      74163:c65790aff86d
parent:      74165:e50af83296fd
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:27:51 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -272,7 +272,7 @@
 detail in the :class:`Popen` constructor documentation.
 
 
-Popen Constuctor
+Popen Constructor
 ^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 11:54:10 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:54:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_fix_to_sent/sen?=
	=?utf8?q?d_typo?=
Message-ID: <E1ReliU-0005GR-8v@dinsdale.python.org>

http://hg.python.org/cpython/rev/46896e66de07
changeset:   74167:46896e66de07
branch:      2.7
parent:      74164:eea6b76eb2fc
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:43:37 2011 +0100
summary:
  fix to sent/send typo

files:
  Doc/library/httplib.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/httplib.rst b/Doc/library/httplib.rst
--- a/Doc/library/httplib.rst
+++ b/Doc/library/httplib.rst
@@ -452,7 +452,7 @@
    Set the host and the port for HTTP Connect Tunnelling. Normally used when
    it is required to do HTTPS Conection through a proxy server.
 
-   The headers argument should be a mapping of extra HTTP headers to sent
+   The headers argument should be a mapping of extra HTTP headers to send
    with the CONNECT request.
 
    .. versionadded:: 2.7

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 11:54:11 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:54:11 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_to_sent/sen?=
	=?utf8?q?d_typo?=
Message-ID: <E1ReliV-0005Gd-1F@dinsdale.python.org>

http://hg.python.org/cpython/rev/cadf43c67f00
changeset:   74168:cadf43c67f00
branch:      3.2
parent:      74165:e50af83296fd
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:44:38 2011 +0100
summary:
  fix to sent/send typo

files:
  Doc/library/http.client.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -435,7 +435,7 @@
    Set the host and the port for HTTP Connect Tunnelling. Normally used when it
    is required to a HTTPS Connection through a proxy server.
 
-   The headers argument should be a mapping of extra HTTP headers to sent
+   The headers argument should be a mapping of extra HTTP headers to send
    with the CONNECT request.
 
    .. versionadded:: 3.2

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 11:54:11 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 11:54:11 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ReliV-0005Gh-Ub@dinsdale.python.org>

http://hg.python.org/cpython/rev/745f9fd9856d
changeset:   74169:745f9fd9856d
parent:      74166:db2f80515807
parent:      74168:cadf43c67f00
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 11:44:55 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/http.client.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -435,7 +435,7 @@
    Set the host and the port for HTTP Connect Tunnelling. Normally used when it
    is required to a HTTPS Connection through a proxy server.
 
-   The headers argument should be a mapping of extra HTTP headers to sent
+   The headers argument should be a mapping of extra HTTP headers to send
    with the CONNECT request.
 
    .. versionadded:: 3.2

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:07:50 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:07:50 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_use_double-quot?=
 =?utf8?q?e_char_instead_of_unicode_ones=3B_thanks_to_tomo_cocoa_from_docs?=
 =?utf8?q?=40?=
Message-ID: <E1Reqc2-0007NF-5F@dinsdale.python.org>

http://hg.python.org/cpython/rev/79ca3246dac2
changeset:   74170:79ca3246dac2
branch:      2.7
parent:      74167:46896e66de07
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:05:30 2011 +0100
summary:
  use double-quote char instead of unicode ones; thanks to tomo cocoa from docs@

files:
  Doc/library/ttk.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ttk.rst b/Doc/library/ttk.rst
--- a/Doc/library/ttk.rst
+++ b/Doc/library/ttk.rst
@@ -1294,7 +1294,7 @@
 
        * sticky=spec
           Specifies how the image is placed within the final parcel. spec
-          contains zero or more characters ?n?, ?s?, ?w?, or ?e?.
+          contains zero or more characters "n", "s", "w", or "e".
 
        * width=width
           Specifies a minimum width for the element. If less than zero, the

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:07:50 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:07:50 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_use_double-quot?=
 =?utf8?q?e_char_instead_of_unicode_ones=3B_thanks_to_tomo_cocoa_from_docs?=
 =?utf8?q?=40?=
Message-ID: <E1Reqc2-0007NP-U6@dinsdale.python.org>

http://hg.python.org/cpython/rev/15713292e79c
changeset:   74171:15713292e79c
branch:      3.2
parent:      74168:cadf43c67f00
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:07:22 2011 +0100
summary:
  use double-quote char instead of unicode ones; thanks to tomo cocoa from docs@

files:
  Doc/library/tkinter.ttk.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst
--- a/Doc/library/tkinter.ttk.rst
+++ b/Doc/library/tkinter.ttk.rst
@@ -1291,7 +1291,7 @@
 
        * sticky=spec
           Specifies how the image is placed within the final parcel. spec
-          contains zero or more characters ?n?, ?s?, ?w?, or ?e?.
+          contains zero or more characters "n", "s", "w", or "e".
 
        * width=width
           Specifies a minimum width for the element. If less than zero, the

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:07:51 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:07:51 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1Reqc3-0007NT-Ll@dinsdale.python.org>

http://hg.python.org/cpython/rev/35fca63d7b3e
changeset:   74172:35fca63d7b3e
parent:      74169:745f9fd9856d
parent:      74171:15713292e79c
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:07:38 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/tkinter.ttk.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst
--- a/Doc/library/tkinter.ttk.rst
+++ b/Doc/library/tkinter.ttk.rst
@@ -1291,7 +1291,7 @@
 
        * sticky=spec
           Specifies how the image is placed within the final parcel. spec
-          contains zero or more characters ?n?, ?s?, ?w?, or ?e?.
+          contains zero or more characters "n", "s", "w", or "e".
 
        * width=width
           Specifies a minimum width for the element. If less than zero, the

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:14:41 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:14:41 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_fix_=22Title_un?=
 =?utf8?q?derline_too_short=22_sphinx_warning?=
Message-ID: <E1Reqif-0007oF-A1@dinsdale.python.org>

http://hg.python.org/cpython/rev/43808e022444
changeset:   74173:43808e022444
branch:      2.7
parent:      74170:79ca3246dac2
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:13:10 2011 +0100
summary:
  fix "Title underline too short" sphinx warning

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -234,7 +234,7 @@
 
 
 Popen Constructor
-^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by
 the :class:`Popen` class. It offers a lot of flexibility so that developers

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:14:42 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:14:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_=22Title_un?=
 =?utf8?q?derline_too_short=22_sphinx_warning?=
Message-ID: <E1Reqig-0007oP-3a@dinsdale.python.org>

http://hg.python.org/cpython/rev/ed9a8a5dcf7a
changeset:   74174:ed9a8a5dcf7a
branch:      3.2
parent:      74171:15713292e79c
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:14:11 2011 +0100
summary:
  fix "Title underline too short" sphinx warning

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -240,7 +240,7 @@
 
 
 Popen Constructor
-^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by
 the :class:`Popen` class. It offers a lot of flexibility so that developers

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:14:42 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:14:42 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1Reqig-0007oT-RX@dinsdale.python.org>

http://hg.python.org/cpython/rev/9df400dc6559
changeset:   74175:9df400dc6559
parent:      74172:35fca63d7b3e
parent:      74174:ed9a8a5dcf7a
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:14:28 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/subprocess.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -273,7 +273,7 @@
 
 
 Popen Constructor
-^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^
 
 The underlying process creation and management in this module is handled by
 the :class:`Popen` class. It offers a lot of flexibility so that developers

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:26:17 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:26:17 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMy4yKTogdXNlIGxpc3QoKSB0?=
 =?utf8?q?o_wrap_range=28=29_in_itertools=2Epermutations=28=29_example=3B_?=
 =?utf8?q?thanks_to?=
Message-ID: <E1Reqtt-0000Dk-Tt@dinsdale.python.org>

http://hg.python.org/cpython/rev/6924b9548962
changeset:   74176:6924b9548962
branch:      3.2
parent:      74174:ed9a8a5dcf7a
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:25:45 2011 +0100
summary:
  use list() to wrap range() in itertools.permutations() example; thanks to Romain MORLEVAT from docs@

files:
  Doc/library/itertools.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -419,7 +419,7 @@
             if r > n:
                 return
             indices = list(range(n))
-            cycles = range(n, n-r, -1)
+            cycles = list(range(n, n-r, -1))
             yield tuple(pool[i] for i in indices[:r])
             while n:
                 for i in reversed(range(r)):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 17:26:18 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sun, 25 Dec 2011 17:26:18 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1Reqtu-0000Dw-LR@dinsdale.python.org>

http://hg.python.org/cpython/rev/e58c67d58d14
changeset:   74177:e58c67d58d14
parent:      74175:9df400dc6559
parent:      74176:6924b9548962
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sun Dec 25 17:26:09 2011 +0100
summary:
  merge with 3.2

files:
  Doc/library/itertools.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -457,7 +457,7 @@
             if r > n:
                 return
             indices = list(range(n))
-            cycles = range(n, n-r, -1)
+            cycles = list(range(n, n-r, -1))
             yield tuple(pool[i] for i in indices[:r])
             while n:
                 for i in reversed(range(r)):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 19:04:00 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 25 Dec 2011 19:04:00 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_I_do_not_think_?=
 =?utf8?q?we_will_ever_have_auto-indent_at_the_prompt=2C_that_is_for_IPyth?=
 =?utf8?q?on?=
Message-ID: <E1ResQS-0005cG-8A@dinsdale.python.org>

http://hg.python.org/cpython/rev/f0197f1d158e
changeset:   74178:f0197f1d158e
branch:      3.2
parent:      74176:6924b9548962
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 25 19:03:07 2011 +0100
summary:
  I do not think we will ever have auto-indent at the prompt, that is for IPython and the like.

files:
  Doc/tutorial/introduction.rst |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst
--- a/Doc/tutorial/introduction.rst
+++ b/Doc/tutorial/introduction.rst
@@ -593,13 +593,13 @@
   and ``!=`` (not equal to).
 
 * The *body* of the loop is *indented*: indentation is Python's way of grouping
-  statements.  Python does not (yet!) provide an intelligent input line editing
-  facility, so you have to type a tab or space(s) for each indented line.  In
-  practice you will prepare more complicated input for Python with a text editor;
-  most text editors have an auto-indent facility.  When a compound statement is
-  entered interactively, it must be followed by a blank line to indicate
-  completion (since the parser cannot guess when you have typed the last line).
-  Note that each line within a basic block must be indented by the same amount.
+  statements.  At the interactive prompt, you have to type a tab or space(s) for
+  each indented line.  In practice you will prepare more complicated input
+  for Python with a text editor; all decent text editors have an auto-indent
+  facility.  When a compound statement is entered interactively, it must be
+  followed by a blank line to indicate completion (since the parser cannot
+  guess when you have typed the last line).  Note that each line within a basic
+  block must be indented by the same amount.
 
 * The :func:`print` function writes the value of the expression(s) it is
   given.  It differs from just writing the expression you want to write (as we did

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 19:04:01 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 25 Dec 2011 19:04:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1ResQT-0005cQ-0f@dinsdale.python.org>

http://hg.python.org/cpython/rev/47f48553dd6b
changeset:   74179:47f48553dd6b
parent:      74177:e58c67d58d14
parent:      74178:f0197f1d158e
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 25 19:03:22 2011 +0100
summary:
  merge with 3.2

files:
  Doc/tutorial/introduction.rst |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst
--- a/Doc/tutorial/introduction.rst
+++ b/Doc/tutorial/introduction.rst
@@ -593,13 +593,13 @@
   and ``!=`` (not equal to).
 
 * The *body* of the loop is *indented*: indentation is Python's way of grouping
-  statements.  Python does not (yet!) provide an intelligent input line editing
-  facility, so you have to type a tab or space(s) for each indented line.  In
-  practice you will prepare more complicated input for Python with a text editor;
-  most text editors have an auto-indent facility.  When a compound statement is
-  entered interactively, it must be followed by a blank line to indicate
-  completion (since the parser cannot guess when you have typed the last line).
-  Note that each line within a basic block must be indented by the same amount.
+  statements.  At the interactive prompt, you have to type a tab or space(s) for
+  each indented line.  In practice you will prepare more complicated input
+  for Python with a text editor; all decent text editors have an auto-indent
+  facility.  When a compound statement is entered interactively, it must be
+  followed by a blank line to indicate completion (since the parser cannot
+  guess when you have typed the last line).  Note that each line within a basic
+  block must be indented by the same amount.
 
 * The :func:`print` function writes the value of the expression(s) it is
   given.  It differs from just writing the expression you want to write (as we did

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Dec 25 19:04:01 2011
From: python-checkins at python.org (georg.brandl)
Date: Sun, 25 Dec 2011 19:04:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_I_do_not_think_?=
 =?utf8?q?we_will_ever_have_auto-indent_at_the_prompt=2C_that_is_for_IPyth?=
 =?utf8?q?on?=
Message-ID: <E1ResQT-0005cX-Pm@dinsdale.python.org>

http://hg.python.org/cpython/rev/b48991f426a7
changeset:   74180:b48991f426a7
branch:      2.7
parent:      74173:43808e022444
user:        Georg Brandl <georg at python.org>
date:        Sun Dec 25 19:03:07 2011 +0100
summary:
  I do not think we will ever have auto-indent at the prompt, that is for IPython and the like.

files:
  Doc/tutorial/introduction.rst |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst
--- a/Doc/tutorial/introduction.rst
+++ b/Doc/tutorial/introduction.rst
@@ -630,13 +630,13 @@
   and ``!=`` (not equal to).
 
 * The *body* of the loop is *indented*: indentation is Python's way of grouping
-  statements.  Python does not (yet!) provide an intelligent input line editing
-  facility, so you have to type a tab or space(s) for each indented line.  In
-  practice you will prepare more complicated input for Python with a text editor;
-  most text editors have an auto-indent facility.  When a compound statement is
-  entered interactively, it must be followed by a blank line to indicate
-  completion (since the parser cannot guess when you have typed the last line).
-  Note that each line within a basic block must be indented by the same amount.
+  statements.  At the interactive prompt, you have to type a tab or space(s) for
+  each indented line.  In practice you will prepare more complicated input
+  for Python with a text editor; all decent text editors have an auto-indent
+  facility.  When a compound statement is entered interactively, it must be
+  followed by a blank line to indicate completion (since the parser cannot
+  guess when you have typed the last line).  Note that each line within a basic
+  block must be indented by the same amount.
 
 * The :keyword:`print` statement writes the value of the expression(s) it is
   given.  It differs from just writing the expression you want to write (as we did

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Dec 26 05:30:23 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 26 Dec 2011 05:30:23 +0100
Subject: [Python-checkins] Daily reference leaks (47f48553dd6b): sum=0
Message-ID: <E1Rf2Cd-0007kw-L8@ap.vmr.nerim.net>

results for 47f48553dd6b on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogLaPv41', '-x']

From python-checkins at python.org  Mon Dec 26 18:22:35 2011
From: python-checkins at python.org (jason.coombs)
Date: Mon, 26 Dec 2011 18:22:35 +0100
Subject: [Python-checkins] =?utf8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzExNjM4?=
 =?utf8?q?=3A_Adding_test_to_ensure_=2Etar=2Egz_files_can_be_generated_by_?=
 =?utf8?q?sdist?=
Message-ID: <E1RfEFv-0007VW-4T@dinsdale.python.org>

http://hg.python.org/cpython/rev/dc1045d08bd8
changeset:   74181:dc1045d08bd8
branch:      2.7
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Mon Dec 26 10:15:15 2011 -0500
summary:
  Issue #11638: Adding test to ensure .tar.gz files can be generated by sdist command with unicode metadata, based on David Barnett's patch.
Issue #11638: Added tests to capture failures in make_tarball with various unicode strings.
Following fix for Issue #13639, these tests now pass.

files:
  Lib/distutils/tests/test_archive_util.py |  31 +++++++++++-
  Lib/distutils/tests/test_sdist.py        |  22 ++++++++
  Misc/NEWS                                |   3 +
  3 files changed, 54 insertions(+), 2 deletions(-)


diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py
--- a/Lib/distutils/tests/test_archive_util.py
+++ b/Lib/distutils/tests/test_archive_util.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 """Tests for distutils.archive_util."""
 __revision__ = "$Id$"
 
@@ -40,6 +41,9 @@
 
     @unittest.skipUnless(zlib, "requires zlib")
     def test_make_tarball(self):
+        self._make_tarball('archive')
+
+    def _make_tarball(self, target_name):
         # creating something to tar
         tmpdir = self.mkdtemp()
         self.write_file([tmpdir, 'file1'], 'xxx')
@@ -51,7 +55,7 @@
         unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0],
                             "source and target should be on same drive")
 
-        base_name = os.path.join(tmpdir2, 'archive')
+        base_name = os.path.join(tmpdir2, target_name)
 
         # working with relative paths to avoid tar warnings
         old_dir = os.getcwd()
@@ -66,7 +70,7 @@
         self.assertTrue(os.path.exists(tarball))
 
         # trying an uncompressed one
-        base_name = os.path.join(tmpdir2, 'archive')
+        base_name = os.path.join(tmpdir2, target_name)
         old_dir = os.getcwd()
         os.chdir(tmpdir)
         try:
@@ -277,6 +281,29 @@
         finally:
             del ARCHIVE_FORMATS['xxx']
 
+    @unittest.skipUnless(zlib, "requires zlib")
+    def test_make_tarball_unicode(self):
+        """
+        Mirror test_make_tarball, except filename is unicode.
+        """
+        self._make_tarball(u'archive')
+
+    @unittest.skipUnless(zlib, "requires zlib")
+    def test_make_tarball_unicode_latin1(self):
+        """
+        Mirror test_make_tarball, except filename is unicode and contains
+        latin characters.
+        """
+        self._make_tarball(u'?rchiv') # note this isn't a real word
+
+    @unittest.skipUnless(zlib, "requires zlib")
+    def test_make_tarball_unicode_extended(self):
+        """
+        Mirror test_make_tarball, except filename is unicode and contains
+        characters outside the latin charset.
+        """
+        self._make_tarball(u'??????') # japanese for archive
+
 def test_suite():
     return unittest.makeSuite(ArchiveUtilTestCase)
 
diff --git a/Lib/distutils/tests/test_sdist.py b/Lib/distutils/tests/test_sdist.py
--- a/Lib/distutils/tests/test_sdist.py
+++ b/Lib/distutils/tests/test_sdist.py
@@ -166,6 +166,28 @@
         self.assertEqual(result, ['fake-1.0.tar', 'fake-1.0.tar.gz'])
 
     @unittest.skipUnless(zlib, "requires zlib")
+    def test_unicode_metadata_tgz(self):
+        """
+        Unicode name or version should not break building to tar.gz format.
+        Reference issue #11638.
+        """
+
+        # create the sdist command with unicode parameters
+        dist, cmd = self.get_cmd({'name': u'fake', 'version': u'1.0'})
+
+        # create the sdist as gztar and run the command
+        cmd.formats = ['gztar']
+        cmd.ensure_finalized()
+        cmd.run()
+
+        # The command should have created the .tar.gz file
+        dist_folder = join(self.tmp_dir, 'dist')
+        result = os.listdir(dist_folder)
+        self.assertEqual(result, ['fake-1.0.tar.gz'])
+
+        os.remove(join(dist_folder, 'fake-1.0.tar.gz'))
+
+    @unittest.skipUnless(zlib, "requires zlib")
     def test_add_defaults(self):
 
         # http://bugs.python.org/issue2279
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,9 @@
 Core and Builtins
 -----------------
 
+- Issue #11638: Unicode strings in 'name' and 'version' no longer cause
+  UnicodeDecodeErrors.
+
 - Fix the fix for issue #12149: it was incorrect, although it had the side
   effect of appearing to resolve the issue.  Thanks to Mark Shannon for
   noticing.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Dec 26 18:22:49 2011
From: python-checkins at python.org (jason.coombs)
Date: Mon, 26 Dec 2011 18:22:49 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Ported_some_test_cases_from?=
	=?utf8?q?_2=2E7_for_=2311638?=
Message-ID: <E1RfEG9-0007Vn-N2@dinsdale.python.org>

http://hg.python.org/cpython/rev/f0fcb82a88e9
changeset:   74182:f0fcb82a88e9
parent:      74179:47f48553dd6b
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Mon Dec 26 12:17:01 2011 -0500
summary:
  Ported some test cases from 2.7 for #11638

files:
  Lib/distutils/tests/test_archive_util.py |  23 ++++++++++-
  1 files changed, 21 insertions(+), 2 deletions(-)


diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py
--- a/Lib/distutils/tests/test_archive_util.py
+++ b/Lib/distutils/tests/test_archive_util.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 """Tests for distutils.archive_util."""
 import unittest
 import os
@@ -32,6 +33,24 @@
 
     @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
     def test_make_tarball(self):
+        self._make_tarball('archive')
+
+    @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    def test_make_tarball_latin1(self):
+        """
+        Mirror test_make_tarball, except filename contains latin characters.
+        """
+        self._make_tarball('?rchiv') # note this isn't a real word
+
+    @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    def test_make_tarball_extended(self):
+        """
+        Mirror test_make_tarball, except filename contains extended
+        characters outside the latin charset.
+        """
+        self._make_tarball('??????') # japanese for archive
+
+    def _make_tarball(self, target_name):
         # creating something to tar
         tmpdir = self.mkdtemp()
         self.write_file([tmpdir, 'file1'], 'xxx')
@@ -43,7 +62,7 @@
         unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0],
                             "Source and target should be on same drive")
 
-        base_name = os.path.join(tmpdir2, 'archive')
+        base_name = os.path.join(tmpdir2, target_name)
 
         # working with relative paths to avoid tar warnings
         old_dir = os.getcwd()
@@ -58,7 +77,7 @@
         self.assertTrue(os.path.exists(tarball))
 
         # trying an uncompressed one
-        base_name = os.path.join(tmpdir2, 'archive')
+        base_name = os.path.join(tmpdir2, target_name)
         old_dir = os.getcwd()
         os.chdir(tmpdir)
         try:

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Dec 27 05:32:23 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 27 Dec 2011 05:32:23 +0100
Subject: [Python-checkins] Daily reference leaks (f0fcb82a88e9): sum=0
Message-ID: <E1RfOi7-00083D-Rm@ap.vmr.nerim.net>

results for f0fcb82a88e9 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogmoZHdh', '-x']

From python-checkins at python.org  Tue Dec 27 22:16:01 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 22:16:01 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_add_a_flags_parameter_to_se?=
	=?utf8?q?lect=2Eepoll?=
Message-ID: <E1RfeNN-0003Vi-1K@dinsdale.python.org>

http://hg.python.org/cpython/rev/b30a94f7e053
changeset:   74183:b30a94f7e053
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 15:15:41 2011 -0600
summary:
  add a flags parameter to select.epoll

files:
  Doc/library/select.rst |  17 ++++++++++----
  Lib/test/test_epoll.py |   2 +
  Misc/NEWS              |   2 +
  Modules/selectmodule.c |  34 +++++++++++++----------------
  4 files changed, 31 insertions(+), 24 deletions(-)


diff --git a/Doc/library/select.rst b/Doc/library/select.rst
--- a/Doc/library/select.rst
+++ b/Doc/library/select.rst
@@ -39,12 +39,19 @@
 
    .. versionadded:: 3.3
 
-.. function:: epoll(sizehint=-1)
+.. function:: epoll(sizehint=-1, flags=0)
 
-   (Only supported on Linux 2.5.44 and newer.)  Returns an edge polling object,
-   which can be used as Edge or Level Triggered interface for I/O events; see
-   section :ref:`epoll-objects` below for the methods supported by epolling
-   objects.
+   (Only supported on Linux 2.5.44 and newer.) Return an edge polling object,
+   which can be used as Edge or Level Triggered interface for I/O
+   events. *sizehint* is deprecated and completely ignored. *flags* can be set
+   to :const:`EPOLL_CLOEXEC`, which causes the epoll descriptor to be closed
+   automatically when :func:`os.execve` is called. See section
+   :ref:`epoll-objects` below for the methods supported by epolling objects.
+
+
+   .. versionchanged:: 3.3
+
+      Added the *flags* parameter.
 
 
 .. function:: poll()
diff --git a/Lib/test/test_epoll.py b/Lib/test/test_epoll.py
--- a/Lib/test/test_epoll.py
+++ b/Lib/test/test_epoll.py
@@ -74,6 +74,8 @@
         ep.close()
         self.assertTrue(ep.closed)
         self.assertRaises(ValueError, ep.fileno)
+        select.epoll(select.EPOLL_CLOEXEC).close()
+        self.assertRaises(OSError, select.epoll, flags=12356)
 
     def test_badcreate(self):
         self.assertRaises(TypeError, select.epoll, 1, 2, 3)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -422,6 +422,8 @@
 Library
 -------
 
+- Add a flags parameter to select.epoll.
+
 - Issue #12798: Updated the mimetypes documentation.
 
 - Issue #13626: Add support for SSL Diffie-Hellman key exchange, through the
diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c
--- a/Modules/selectmodule.c
+++ b/Modules/selectmodule.c
@@ -1087,20 +1087,10 @@
 }
 
 static PyObject *
-newPyEpoll_Object(PyTypeObject *type, int sizehint, SOCKET fd)
+newPyEpoll_Object(PyTypeObject *type, int flags, SOCKET fd)
 {
     pyEpoll_Object *self;
 
-    if (sizehint == -1) {
-        sizehint = FD_SETSIZE-1;
-    }
-    else if (sizehint < 1) {
-        PyErr_Format(PyExc_ValueError,
-                     "sizehint must be greater zero, got %d",
-                     sizehint);
-        return NULL;
-    }
-
     assert(type != NULL && type->tp_alloc != NULL);
     self = (pyEpoll_Object *) type->tp_alloc(type, 0);
     if (self == NULL)
@@ -1108,7 +1098,7 @@
 
     if (fd == -1) {
         Py_BEGIN_ALLOW_THREADS
-        self->epfd = epoll_create(sizehint);
+        self->epfd = epoll_create1(flags);
         Py_END_ALLOW_THREADS
     }
     else {
@@ -1126,14 +1116,18 @@
 static PyObject *
 pyepoll_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
 {
-    int sizehint = -1;
-    static char *kwlist[] = {"sizehint", NULL};
+    int flags = 0, sizehint = 0;
+    static char *kwlist[] = {"sizehint", "flags", NULL};
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwds, "|i:epoll", kwlist,
-                                     &sizehint))
+    if (!PyArg_ParseTupleAndKeywords(args, kwds, "|ii:epoll", kwlist,
+                                     &sizehint, &flags))
         return NULL;
+    if (sizehint < 0) {
+        PyErr_SetString(PyExc_ValueError, "negative sizehint");
+        return NULL;
+    }
 
-    return newPyEpoll_Object(type, sizehint, -1);
+    return newPyEpoll_Object(type, flags, -1);
 }
 
 
@@ -1191,7 +1185,7 @@
     if (!PyArg_ParseTuple(args, "i:fromfd", &fd))
         return NULL;
 
-    return newPyEpoll_Object((PyTypeObject*)cls, -1, fd);
+    return newPyEpoll_Object((PyTypeObject*)cls, 0, fd);
 }
 
 PyDoc_STRVAR(pyepoll_fromfd_doc,
@@ -1420,7 +1414,7 @@
 };
 
 PyDoc_STRVAR(pyepoll_doc,
-"select.epoll([sizehint=-1])\n\
+"select.epoll(sizehint=-1, flags=0)\n\
 \n\
 Returns an epolling object\n\
 \n\
@@ -2218,6 +2212,8 @@
     PyModule_AddIntConstant(m, "EPOLLWRNORM", EPOLLWRNORM);
     PyModule_AddIntConstant(m, "EPOLLWRBAND", EPOLLWRBAND);
     PyModule_AddIntConstant(m, "EPOLLMSG", EPOLLMSG);
+
+    PyModule_AddIntConstant(m, "EPOLL_CLOEXEC", EPOLL_CLOEXEC);
 #endif /* HAVE_EPOLL */
 
 #ifdef HAVE_KQUEUE

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 22:17:30 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 22:17:30 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_propgate_error_?=
	=?utf8?q?in_other_conditions?=
Message-ID: <E1RfeOo-0003d4-Dd@dinsdale.python.org>

http://hg.python.org/cpython/rev/4964712a7269
changeset:   74184:4964712a7269
branch:      3.2
parent:      74178:f0197f1d158e
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 15:16:34 2011 -0600
summary:
  propgate error in other conditions

files:
  Lib/test/test_epoll.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_epoll.py b/Lib/test/test_epoll.py
--- a/Lib/test/test_epoll.py
+++ b/Lib/test/test_epoll.py
@@ -36,6 +36,7 @@
 except IOError as e:
     if e.errno == errno.ENOSYS:
         raise unittest.SkipTest("kernel doesn't support epoll()")
+    raise
 
 class TestEPoll(unittest.TestCase):
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 22:17:33 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 22:17:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_propgate_error_?=
	=?utf8?q?in_other_conditions?=
Message-ID: <E1RfeOr-0003dE-3x@dinsdale.python.org>

http://hg.python.org/cpython/rev/e71e4bd45c89
changeset:   74185:e71e4bd45c89
branch:      2.7
parent:      74181:dc1045d08bd8
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 15:16:34 2011 -0600
summary:
  propgate error in other conditions

files:
  Lib/test/test_epoll.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_epoll.py b/Lib/test/test_epoll.py
--- a/Lib/test/test_epoll.py
+++ b/Lib/test/test_epoll.py
@@ -36,6 +36,7 @@
 except IOError, e:
     if e.errno == errno.ENOSYS:
         raise unittest.SkipTest("kernel doesn't support epoll()")
+    raise
 
 class TestEPoll(unittest.TestCase):
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 22:17:33 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 22:17:33 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1RfeOr-0003dn-Tc@dinsdale.python.org>

http://hg.python.org/cpython/rev/0cae103a42d2
changeset:   74186:0cae103a42d2
parent:      74183:b30a94f7e053
parent:      74184:4964712a7269
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 15:17:15 2011 -0600
summary:
  merge 3.2

files:
  Lib/test/test_epoll.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_epoll.py b/Lib/test/test_epoll.py
--- a/Lib/test/test_epoll.py
+++ b/Lib/test/test_epoll.py
@@ -36,6 +36,7 @@
 except IOError as e:
     if e.errno == errno.ENOSYS:
         raise unittest.SkipTest("kernel doesn't support epoll()")
+    raise
 
 class TestEPoll(unittest.TestCase):
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 22:36:43 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 22:36:43 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_for_old_kernels_which_d?=
 =?utf8?q?on=27t_have_epoll=5Fcreate1?=
Message-ID: <E1RfehP-0005kt-FI@dinsdale.python.org>

http://hg.python.org/cpython/rev/0f9276411dce
changeset:   74187:0f9276411dce
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 15:36:32 2011 -0600
summary:
  fix for old kernels which don't have epoll_create1

files:
  Modules/selectmodule.c |  12 +++++++++---
  configure              |  29 +++++++++++++++++++++++++++--
  configure.in           |   6 ++++++
  pyconfig.h.in          |   3 +++
  4 files changed, 45 insertions(+), 5 deletions(-)


diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c
--- a/Modules/selectmodule.c
+++ b/Modules/selectmodule.c
@@ -1087,7 +1087,7 @@
 }
 
 static PyObject *
-newPyEpoll_Object(PyTypeObject *type, int flags, SOCKET fd)
+newPyEpoll_Object(PyTypeObject *type, int sizehint, int flags, SOCKET fd)
 {
     pyEpoll_Object *self;
 
@@ -1098,7 +1098,11 @@
 
     if (fd == -1) {
         Py_BEGIN_ALLOW_THREADS
+#ifdef HAVE_EPOLL_CREATE1
         self->epfd = epoll_create1(flags);
+#else
+        self->epfd = epoll_create(sizehint);
+#endif
         Py_END_ALLOW_THREADS
     }
     else {
@@ -1127,7 +1131,7 @@
         return NULL;
     }
 
-    return newPyEpoll_Object(type, flags, -1);
+    return newPyEpoll_Object(type, sizehint, flags, -1);
 }
 
 
@@ -1185,7 +1189,7 @@
     if (!PyArg_ParseTuple(args, "i:fromfd", &fd))
         return NULL;
 
-    return newPyEpoll_Object((PyTypeObject*)cls, 0, fd);
+    return newPyEpoll_Object((PyTypeObject*)cls, FD_SETSIZE - 1, 0, fd);
 }
 
 PyDoc_STRVAR(pyepoll_fromfd_doc,
@@ -2213,7 +2217,9 @@
     PyModule_AddIntConstant(m, "EPOLLWRBAND", EPOLLWRBAND);
     PyModule_AddIntConstant(m, "EPOLLMSG", EPOLLMSG);
 
+#ifdef EPOLL_CLOEXEC
     PyModule_AddIntConstant(m, "EPOLL_CLOEXEC", EPOLL_CLOEXEC);
+#endif
 #endif /* HAVE_EPOLL */
 
 #ifdef HAVE_KQUEUE
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9606,6 +9606,31 @@
 
 fi
 rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for epoll_create1" >&5
+$as_echo_n "checking for epoll_create1... " >&6; }
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+#include <sys/epoll.h>
+int
+main ()
+{
+void *x=epoll_create1
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_compile "$LINENO"; then :
+
+$as_echo "#define HAVE_EPOLL_CREATE1 1" >>confdefs.h
+
+   { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for kqueue" >&5
 $as_echo_n "checking for kqueue... " >&6; }
 cat confdefs.h - <<_ACEOF >conftest.$ac_ext
@@ -14564,8 +14589,8 @@
 
 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
 # Files that config.status was made for.
-config_files="`echo $ac_config_files`"
-config_headers="`echo $ac_config_headers`"
+config_files="$ac_config_files"
+config_headers="$ac_config_headers"
 
 _ACEOF
 
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -2628,6 +2628,12 @@
    AC_MSG_RESULT(yes)],
   [AC_MSG_RESULT(no)
 ])
+AC_MSG_CHECKING(for epoll_create1)
+AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/epoll.h>]], [[void *x=epoll_create1]])],
+  [AC_DEFINE(HAVE_EPOLL_CREATE1, 1, Define if you have the 'epoll_create1' function.)
+   AC_MSG_RESULT(yes)],
+  [AC_MSG_RESULT(no)
+])
 AC_MSG_CHECKING(for kqueue)
 AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
 #include <sys/types.h>
diff --git a/pyconfig.h.in b/pyconfig.h.in
--- a/pyconfig.h.in
+++ b/pyconfig.h.in
@@ -196,6 +196,9 @@
 /* Define if you have the 'epoll' functions. */
 #undef HAVE_EPOLL
 
+/* Define if you have the 'epoll_create1' function. */
+#undef HAVE_EPOLL_CREATE1
+
 /* Define to 1 if you have the `erf' function. */
 #undef HAVE_ERF
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 23:01:32 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 23:01:32 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_try_to_always_use_the_old_A?=
	=?utf8?q?PI?=
Message-ID: <E1Rff5Q-0008Ty-EU@dinsdale.python.org>

http://hg.python.org/cpython/rev/4aa7b25f9509
changeset:   74188:4aa7b25f9509
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 16:01:21 2011 -0600
summary:
  try to always use the old API

files:
  Modules/selectmodule.c |  9 +++++----
  1 files changed, 5 insertions(+), 4 deletions(-)


diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c
--- a/Modules/selectmodule.c
+++ b/Modules/selectmodule.c
@@ -1099,10 +1099,11 @@
     if (fd == -1) {
         Py_BEGIN_ALLOW_THREADS
 #ifdef HAVE_EPOLL_CREATE1
-        self->epfd = epoll_create1(flags);
-#else
+        if (flags)
+            self->epfd = epoll_create1(flags);
+        else
+#endif
         self->epfd = epoll_create(sizehint);
-#endif
         Py_END_ALLOW_THREADS
     }
     else {
@@ -1120,7 +1121,7 @@
 static PyObject *
 pyepoll_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
 {
-    int flags = 0, sizehint = 0;
+    int flags = 0, sizehint = FD_SETSIZE - 1;
     static char *kwlist[] = {"sizehint", "flags", NULL};
 
     if (!PyArg_ParseTupleAndKeywords(args, kwds, "|ii:epoll", kwlist,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Dec 27 23:52:31 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 27 Dec 2011 23:52:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_don=27t_fail_if_CLOEXEC_doe?=
	=?utf8?q?sn=27t_exist?=
Message-ID: <E1Rffsl-0005Vc-Qf@dinsdale.python.org>

http://hg.python.org/cpython/rev/456b02030100
changeset:   74189:456b02030100
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Dec 27 16:52:20 2011 -0600
summary:
  don't fail if CLOEXEC doesn't exist

files:
  Lib/test/test_epoll.py |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_epoll.py b/Lib/test/test_epoll.py
--- a/Lib/test/test_epoll.py
+++ b/Lib/test/test_epoll.py
@@ -75,8 +75,9 @@
         ep.close()
         self.assertTrue(ep.closed)
         self.assertRaises(ValueError, ep.fileno)
-        select.epoll(select.EPOLL_CLOEXEC).close()
-        self.assertRaises(OSError, select.epoll, flags=12356)
+        if hasattr(select, "EPOLL_CLOEXEC"):
+            select.epoll(select.EPOLL_CLOEXEC).close()
+            self.assertRaises(OSError, select.epoll, flags=12356)
 
     def test_badcreate(self):
         self.assertRaises(TypeError, select.epoll, 1, 2, 3)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 28 00:52:30 2011
From: python-checkins at python.org (frank.wierzbicki)
Date: Wed, 28 Dec 2011 00:52:30 +0100
Subject: [Python-checkins] =?utf8?q?devguide=3A_Fix_minor_grammar_issue=2E?=
Message-ID: <E1Rfgoo-00038M-AN@dinsdale.python.org>

http://hg.python.org/devguide/rev/d6da06dc69c2
changeset:   466:d6da06dc69c2
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Tue Dec 27 15:51:40 2011 -0800
summary:
  Fix minor grammar issue.

files:
  index.rst |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/index.rst b/index.rst
--- a/index.rst
+++ b/index.rst
@@ -133,7 +133,8 @@
 
 There are other Python implementations, each with a different focus.
 Like CPython, they always have more things they would like to do than they
-have developers to work on them. Some major example that may of interest are:
+have developers to work on them. Some major example that may be of interest
+are:
 
 * PyPy_: A Python interpreter focused on high speed (JIT-compiled) operation
   on major platforms

-- 
Repository URL: http://hg.python.org/devguide

From python-checkins at python.org  Wed Dec 28 02:15:08 2011
From: python-checkins at python.org (frank.wierzbicki)
Date: Wed, 28 Dec 2011 02:15:08 +0100
Subject: [Python-checkins] =?utf8?q?devguide=3A_Fix_minor_grammar_issue=2E?=
Message-ID: <E1Rfi6m-0001xO-Dq@dinsdale.python.org>

http://hg.python.org/devguide/rev/f3843e932e3b
changeset:   467:f3843e932e3b
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Tue Dec 27 17:14:32 2011 -0800
summary:
  Fix minor grammar issue.

files:
  setup.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/setup.rst b/setup.rst
--- a/setup.rst
+++ b/setup.rst
@@ -49,7 +49,7 @@
 You will need to re-compile CPython when you do such an update.
 
 Do note that CPython will notice that it is being run from a working copy.
-This means that it if you edit CPython's source code in your working copy,
+This means that if you edit CPython's source code in your working copy,
 changes to Python code will be picked up by the interpreter for immediate
 use and testing.  (If you change C code, you will need to recompile the
 affected files as described below.)

-- 
Repository URL: http://hg.python.org/devguide

From solipsis at pitrou.net  Wed Dec 28 05:31:45 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 28 Dec 2011 05:31:45 +0100
Subject: [Python-checkins] Daily reference leaks (456b02030100): sum=0
Message-ID: <E1RflB3-0008Q7-MJ@ap.vmr.nerim.net>

results for 456b02030100 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogH3C921', '-x']

From python-checkins at python.org  Wed Dec 28 16:45:53 2011
From: python-checkins at python.org (jason.coombs)
Date: Wed, 28 Dec 2011 16:45:53 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Limit_test_scope_to_those_p?=
 =?utf8?q?latforms_that_can_save_the_target_filenames=2E?=
Message-ID: <E1RfvhR-0006Ma-5D@dinsdale.python.org>

http://hg.python.org/cpython/rev/a7744f778646
changeset:   74190:a7744f778646
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Wed Dec 28 10:45:08 2011 -0500
summary:
  Limit test scope to those platforms that can save the target filenames. Reference #11638.

files:
  Lib/distutils/tests/test_archive_util.py |  17 ++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py
--- a/Lib/distutils/tests/test_archive_util.py
+++ b/Lib/distutils/tests/test_archive_util.py
@@ -2,6 +2,7 @@
 """Tests for distutils.archive_util."""
 import unittest
 import os
+import sys
 import tarfile
 from os.path import splitdrive
 import warnings
@@ -26,6 +27,18 @@
 except ImportError:
     ZLIB_SUPPORT = False
 
+def can_fs_encode(filename):
+    """
+    Return True if the filename can be saved in the file system.
+    """
+    if os.path.supports_unicode_filenames:
+        return True
+    try:
+        filename.encode(sys.getfilesystemencoding())
+    except UnicodeEncodeError:
+        return False
+    return True
+
 
 class ArchiveUtilTestCase(support.TempdirManager,
                           support.LoggingSilencer,
@@ -36,6 +49,8 @@
         self._make_tarball('archive')
 
     @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    @unittest.skipUnless(can_fs_encode('?rchiv'),
+        'File system cannot handle this filename')
     def test_make_tarball_latin1(self):
         """
         Mirror test_make_tarball, except filename contains latin characters.
@@ -43,6 +58,8 @@
         self._make_tarball('?rchiv') # note this isn't a real word
 
     @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    @unittest.skipUnless(can_fs_encode('??????'),
+        'File system cannot handle this filename')
     def test_make_tarball_extended(self):
         """
         Mirror test_make_tarball, except filename contains extended

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 28 17:42:38 2011
From: python-checkins at python.org (jason.coombs)
Date: Wed, 28 Dec 2011 17:42:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Limit_test_scop?=
 =?utf8?q?e_to_those_platforms_that_can_save_the_target_filenames=2E?=
Message-ID: <E1RfwaM-0003Vj-JN@dinsdale.python.org>

http://hg.python.org/cpython/rev/9b681e0c04ed
changeset:   74191:9b681e0c04ed
branch:      2.7
parent:      74185:e71e4bd45c89
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Wed Dec 28 11:42:22 2011 -0500
summary:
  Limit test scope to those platforms that can save the target filenames. Reference #11638.

files:
  Lib/distutils/tests/test_archive_util.py |  17 ++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py
--- a/Lib/distutils/tests/test_archive_util.py
+++ b/Lib/distutils/tests/test_archive_util.py
@@ -4,6 +4,7 @@
 
 import unittest
 import os
+import sys
 import tarfile
 from os.path import splitdrive
 import warnings
@@ -34,6 +35,18 @@
 except ImportError:
     zlib = None
 
+def can_fs_encode(filename):
+    """
+    Return True if the filename can be saved in the file system.
+    """
+    if os.path.supports_unicode_filenames:
+        return True
+    try:
+        filename.encode(sys.getfilesystemencoding())
+    except UnicodeEncodeError:
+        return False
+    return True
+
 
 class ArchiveUtilTestCase(support.TempdirManager,
                           support.LoggingSilencer,
@@ -289,6 +302,8 @@
         self._make_tarball(u'archive')
 
     @unittest.skipUnless(zlib, "requires zlib")
+    @unittest.skipUnless(can_fs_encode(u'?rchiv'),
+        'File system cannot handle this filename')
     def test_make_tarball_unicode_latin1(self):
         """
         Mirror test_make_tarball, except filename is unicode and contains
@@ -297,6 +312,8 @@
         self._make_tarball(u'?rchiv') # note this isn't a real word
 
     @unittest.skipUnless(zlib, "requires zlib")
+    @unittest.skipUnless(can_fs_encode(u'??????'),
+        'File system cannot handle this filename')
     def test_make_tarball_unicode_extended(self):
         """
         Mirror test_make_tarball, except filename is unicode and contains

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 28 19:02:03 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 28 Dec 2011 19:02:03 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_weird_indentation?=
Message-ID: <E1RfxpD-0001j9-7E@dinsdale.python.org>

http://hg.python.org/cpython/rev/0558a03b1bec
changeset:   74192:0558a03b1bec
parent:      74189:456b02030100
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Dec 28 12:01:31 2011 -0600
summary:
  fix weird indentation

files:
  Objects/abstract.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -1984,7 +1984,7 @@
 int
 PyMapping_Check(PyObject *o)
 {
-    return      o && o->ob_type->tp_as_mapping &&
+    return o && o->ob_type->tp_as_mapping &&
         o->ob_type->tp_as_mapping->mp_subscript;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Dec 28 19:02:04 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 28 Dec 2011 19:02:04 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RfxpE-0001jJ-Gr@dinsdale.python.org>

http://hg.python.org/cpython/rev/e3ba7e1dd406
changeset:   74193:e3ba7e1dd406
parent:      74192:0558a03b1bec
parent:      74190:a7744f778646
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Dec 28 12:01:51 2011 -0600
summary:
  merge heads

files:
  Lib/distutils/tests/test_archive_util.py |  17 ++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Lib/distutils/tests/test_archive_util.py b/Lib/distutils/tests/test_archive_util.py
--- a/Lib/distutils/tests/test_archive_util.py
+++ b/Lib/distutils/tests/test_archive_util.py
@@ -2,6 +2,7 @@
 """Tests for distutils.archive_util."""
 import unittest
 import os
+import sys
 import tarfile
 from os.path import splitdrive
 import warnings
@@ -26,6 +27,18 @@
 except ImportError:
     ZLIB_SUPPORT = False
 
+def can_fs_encode(filename):
+    """
+    Return True if the filename can be saved in the file system.
+    """
+    if os.path.supports_unicode_filenames:
+        return True
+    try:
+        filename.encode(sys.getfilesystemencoding())
+    except UnicodeEncodeError:
+        return False
+    return True
+
 
 class ArchiveUtilTestCase(support.TempdirManager,
                           support.LoggingSilencer,
@@ -36,6 +49,8 @@
         self._make_tarball('archive')
 
     @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    @unittest.skipUnless(can_fs_encode('?rchiv'),
+        'File system cannot handle this filename')
     def test_make_tarball_latin1(self):
         """
         Mirror test_make_tarball, except filename contains latin characters.
@@ -43,6 +58,8 @@
         self._make_tarball('?rchiv') # note this isn't a real word
 
     @unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+    @unittest.skipUnless(can_fs_encode('??????'),
+        'File system cannot handle this filename')
     def test_make_tarball_extended(self):
         """
         Mirror test_make_tarball, except filename contains extended

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Dec 29 05:31:02 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 29 Dec 2011 05:31:02 +0100
Subject: [Python-checkins] Daily reference leaks (e3ba7e1dd406): sum=0
Message-ID: <E1Rg7du-0000FA-Mh@ap.vmr.nerim.net>

results for e3ba7e1dd406 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog0TeEK8', '-x']

From python-checkins at python.org  Thu Dec 29 18:55:12 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Thu, 29 Dec 2011 18:55:12 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_Issue_=2312715=3A_Add_an_op?=
 =?utf8?q?tional_symlinks_argument_to_shutil_functions_=28copyfile=2C?=
Message-ID: <E1RgKC8-0006BJ-9N@dinsdale.python.org>

http://hg.python.org/cpython/rev/cf57ef65bcd0
changeset:   74194:cf57ef65bcd0
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Thu Dec 29 18:54:15 2011 +0100
summary:
  Issue #12715: Add an optional symlinks argument to shutil functions (copyfile, copymode, copystat, copy, copy2).
When that parameter is true, symlinks aren't dereferenced and the operation
instead acts on the symlink itself (or creates one, if relevant).

Patch by Hynek Schlawack.

files:
  Doc/library/shutil.rst  |   46 ++++-
  Lib/shutil.py           |  101 +++++++++---
  Lib/test/test_shutil.py |  219 ++++++++++++++++++++++++++++
  Misc/NEWS               |    5 +
  4 files changed, 333 insertions(+), 38 deletions(-)


diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -45,7 +45,7 @@
    be copied.
 
 
-.. function:: copyfile(src, dst)
+.. function:: copyfile(src, dst[, symlinks=False])
 
    Copy the contents (no metadata) of the file named *src* to a file named *dst*.
    *dst* must be the complete target file name; look at :func:`copy` for a copy that
@@ -56,37 +56,56 @@
    such as character or block devices and pipes cannot be copied with this
    function.  *src* and *dst* are path names given as strings.
 
+   If *symlinks* is true and *src* is a symbolic link, a new symbolic link will
+   be created instead of copying the file *src* points to.
+
    .. versionchanged:: 3.3
       :exc:`IOError` used to be raised instead of :exc:`OSError`.
+      Added *symlinks* argument.
 
 
-.. function:: copymode(src, dst)
+.. function:: copymode(src, dst[, symlinks=False])
 
    Copy the permission bits from *src* to *dst*.  The file contents, owner, and
-   group are unaffected.  *src* and *dst* are path names given as strings.
+   group are unaffected.  *src* and *dst* are path names given as strings.  If
+   *symlinks* is true, *src* a symbolic link and the operating system supports
+   modes for symbolic links (for example BSD-based ones), the mode of the link
+   will be copied.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copystat(src, dst)
+.. function:: copystat(src, dst[, symlinks=False])
 
    Copy the permission bits, last access time, last modification time, and flags
    from *src* to *dst*.  The file contents, owner, and group are unaffected.  *src*
-   and *dst* are path names given as strings.
+   and *dst* are path names given as strings.  If *src* and *dst* are both
+   symbolic links and *symlinks* true, the stats of the link will be copied as
+   far as the platform allows.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copy(src, dst)
+.. function:: copy(src, dst[, symlinks=False]))
 
    Copy the file *src* to the file or directory *dst*.  If *dst* is a directory, a
    file with the same basename as *src*  is created (or overwritten) in the
    directory specified.  Permission bits are copied.  *src* and *dst* are path
-   names given as strings.
+   names given as strings.  If *symlinks* is true, symbolic links won't be
+   followed but recreated instead -- this resembles GNU's :program:`cp -P`.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copy2(src, dst)
+.. function:: copy2(src, dst[, symlinks=False])
 
    Similar to :func:`copy`, but metadata is copied as well -- in fact, this is just
    :func:`copy` followed by :func:`copystat`.  This is similar to the
-   Unix command :program:`cp -p`.
+   Unix command :program:`cp -p`.  If *symlinks* is true, symbolic links won't
+   be followed but recreated instead -- this resembles GNU's :program:`cp -P`.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
 .. function:: ignore_patterns(\*patterns)
 
@@ -104,9 +123,9 @@
    :func:`copy2`.
 
    If *symlinks* is true, symbolic links in the source tree are represented as
-   symbolic links in the new tree, but the metadata of the original links is NOT
-   copied; if false or omitted, the contents and metadata of the linked files
-   are copied to the new tree.
+   symbolic links in the new tree and the metadata of the original links will
+   be copied as far as the platform allows; if false or omitted, the contents
+   and metadata of the linked files are copied to the new tree.
 
    When *symlinks* is false, if the file pointed by the symlink doesn't
    exist, a exception will be added in the list of errors raised in
@@ -140,6 +159,9 @@
       Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
       errors when *symlinks* is false.
 
+   .. versionchanged:: 3.3
+      Copy metadata when *symlinks* is false.
+
 
 .. function:: rmtree(path, ignore_errors=False, onerror=None)
 
diff --git a/Lib/shutil.py b/Lib/shutil.py
--- a/Lib/shutil.py
+++ b/Lib/shutil.py
@@ -82,8 +82,13 @@
     return (os.path.normcase(os.path.abspath(src)) ==
             os.path.normcase(os.path.abspath(dst)))
 
-def copyfile(src, dst):
-    """Copy data from src to dst"""
+def copyfile(src, dst, symlinks=False):
+    """Copy data from src to dst.
+
+    If optional flag `symlinks` is set and `src` is a symbolic link, a new
+    symlink will be created instead of copying the file it points to.
+
+    """
     if _samefile(src, dst):
         raise Error("`%s` and `%s` are the same file" % (src, dst))
 
@@ -98,54 +103,94 @@
             if stat.S_ISFIFO(st.st_mode):
                 raise SpecialFileError("`%s` is a named pipe" % fn)
 
-    with open(src, 'rb') as fsrc:
-        with open(dst, 'wb') as fdst:
-            copyfileobj(fsrc, fdst)
+    if symlinks and os.path.islink(src):
+        os.symlink(os.readlink(src), dst)
+    else:
+        with open(src, 'rb') as fsrc:
+            with open(dst, 'wb') as fdst:
+                copyfileobj(fsrc, fdst)
 
-def copymode(src, dst):
-    """Copy mode bits from src to dst"""
-    if hasattr(os, 'chmod'):
-        st = os.stat(src)
-        mode = stat.S_IMODE(st.st_mode)
-        os.chmod(dst, mode)
+def copymode(src, dst, symlinks=False):
+    """Copy mode bits from src to dst.
 
-def copystat(src, dst):
-    """Copy all stat info (mode bits, atime, mtime, flags) from src to dst"""
-    st = os.stat(src)
+    If the optional flag `symlinks` is set, symlinks aren't followed if and
+    only if both `src` and `dst` are symlinks. If `lchmod` isn't available (eg.
+    Linux), in these cases, this method does nothing.
+
+    """
+    if symlinks and os.path.islink(src) and os.path.islink(dst):
+        if hasattr(os, 'lchmod'):
+            stat_func, chmod_func = os.lstat, os.lchmod
+        else:
+            return
+    elif hasattr(os, 'chmod'):
+        stat_func, chmod_func = os.stat, os.chmod
+    else:
+        return
+
+    st = stat_func(src)
+    chmod_func(dst, stat.S_IMODE(st.st_mode))
+
+def copystat(src, dst, symlinks=False):
+    """Copy all stat info (mode bits, atime, mtime, flags) from src to dst.
+
+    If the optional flag `symlinks` is set, symlinks aren't followed if and
+    only if both `src` and `dst` are symlinks.
+
+    """
+    def _nop(*args):
+        pass
+
+    if symlinks and os.path.islink(src) and os.path.islink(dst):
+        stat_func = os.lstat
+        utime_func = os.lutimes if hasattr(os, 'lutimes') else _nop
+        chmod_func = os.lchmod if hasattr(os, 'lchmod') else _nop
+        chflags_func = os.lchflags if hasattr(os, 'lchflags') else _nop
+    else:
+        stat_func = os.stat
+        utime_func = os.utime if hasattr(os, 'utime') else _nop
+        chmod_func = os.chmod if hasattr(os, 'chmod') else _nop
+        chflags_func = os.chflags if hasattr(os, 'chflags') else _nop
+
+    st = stat_func(src)
     mode = stat.S_IMODE(st.st_mode)
-    if hasattr(os, 'utime'):
-        os.utime(dst, (st.st_atime, st.st_mtime))
-    if hasattr(os, 'chmod'):
-        os.chmod(dst, mode)
-    if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):
+    utime_func(dst, (st.st_atime, st.st_mtime))
+    chmod_func(dst, mode)
+    if hasattr(st, 'st_flags'):
         try:
-            os.chflags(dst, st.st_flags)
+            chflags_func(dst, st.st_flags)
         except OSError as why:
             if (not hasattr(errno, 'EOPNOTSUPP') or
                 why.errno != errno.EOPNOTSUPP):
                 raise
 
-def copy(src, dst):
+def copy(src, dst, symlinks=False):
     """Copy data and mode bits ("cp src dst").
 
     The destination may be a directory.
 
+    If the optional flag `symlinks` is set, symlinks won't be followed. This
+    resembles GNU's "cp -P src dst".
+
     """
     if os.path.isdir(dst):
         dst = os.path.join(dst, os.path.basename(src))
-    copyfile(src, dst)
-    copymode(src, dst)
+    copyfile(src, dst, symlinks=symlinks)
+    copymode(src, dst, symlinks=symlinks)
 
-def copy2(src, dst):
+def copy2(src, dst, symlinks=False):
     """Copy data and all stat info ("cp -p src dst").
 
     The destination may be a directory.
 
+    If the optional flag `symlinks` is set, symlinks won't be followed. This
+    resembles GNU's "cp -P src dst".
+
     """
     if os.path.isdir(dst):
         dst = os.path.join(dst, os.path.basename(src))
-    copyfile(src, dst)
-    copystat(src, dst)
+    copyfile(src, dst, symlinks=symlinks)
+    copystat(src, dst, symlinks=symlinks)
 
 def ignore_patterns(*patterns):
     """Function that can be used as copytree() ignore parameter.
@@ -212,7 +257,11 @@
             if os.path.islink(srcname):
                 linkto = os.readlink(srcname)
                 if symlinks:
+                    # We can't just leave it to `copy_function` because legacy
+                    # code with a custom `copy_function` may rely on copytree
+                    # doing the right thing.
                     os.symlink(linkto, dstname)
+                    copystat(srcname, dstname, symlinks=symlinks)
                 else:
                     # ignore dangling symlink if the flag is on
                     if not os.path.exists(linkto) and ignore_dangling_symlinks:
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -164,6 +164,197 @@
             self.assertTrue(issubclass(exc[0], OSError))
             self.errorState = 2
 
+    @unittest.skipUnless(hasattr(os, 'chmod'), 'requires os.chmod')
+    @support.skip_unless_symlink
+    def test_copymode_follow_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        os.chmod(src, stat.S_IRWXU|stat.S_IRWXG)
+        # file to file
+        os.chmod(dst, stat.S_IRWXO)
+        self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        shutil.copymode(src, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow src link
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src_link, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow dst link
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src, dst_link)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow both links
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src_link, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+
+    @unittest.skipUnless(hasattr(os, 'lchmod'), 'requires os.lchmod')
+    @support.skip_unless_symlink
+    def test_copymode_symlink_to_symlink(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        os.chmod(src, stat.S_IRWXU|stat.S_IRWXG)
+        os.chmod(dst, stat.S_IRWXU)
+        os.lchmod(src_link, stat.S_IRWXO|stat.S_IRWXG)
+        # link to link
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src_link, dst_link, symlinks=True)
+        self.assertEqual(os.lstat(src_link).st_mode,
+                         os.lstat(dst_link).st_mode)
+        self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # src link - use chmod
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src_link, dst, symlinks=True)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # dst link - use chmod
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src, dst_link, symlinks=True)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+
+    @unittest.skipIf(hasattr(os, 'lchmod'), 'requires os.lchmod to be missing')
+    @support.skip_unless_symlink
+    def test_copymode_symlink_to_symlink_wo_lchmod(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        shutil.copymode(src_link, dst_link, symlinks=True)  # silent fail
+
+    @support.skip_unless_symlink
+    def test_copystat_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'qux')
+        write_file(src, 'foo')
+        src_stat = os.stat(src)
+        os.utime(src, (src_stat.st_atime,
+                       src_stat.st_mtime - 42.0))  # ensure different mtimes
+        write_file(dst, 'bar')
+        self.assertNotEqual(os.stat(src).st_mtime, os.stat(dst).st_mtime)
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_link_stat = os.lstat(src_link)
+        # follow
+        if hasattr(os, 'lchmod'):
+            shutil.copystat(src_link, dst_link, symlinks=False)
+            self.assertNotEqual(src_link_stat.st_mode, os.stat(dst).st_mode)
+        # don't follow
+        shutil.copystat(src_link, dst_link, symlinks=True)
+        dst_link_stat = os.lstat(dst_link)
+        if hasattr(os, 'lutimes'):
+            for attr in 'st_atime', 'st_mtime':
+                # The modification times may be truncated in the new file.
+                self.assertLessEqual(getattr(src_link_stat, attr),
+                                     getattr(dst_link_stat, attr) + 1)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(src_link_stat.st_mode, dst_link_stat.st_mode)
+        if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'):
+            self.assertEqual(src_link_stat.st_flags, dst_link_stat.st_flags)
+        # tell to follow but dst is not a link
+        shutil.copystat(src_link, dst, symlinks=True)
+        self.assertTrue(abs(os.stat(src).st_mtime - os.stat(dst).st_mtime) <
+                        00000.1)
+
+    @support.skip_unless_symlink
+    def test_copy_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        write_file(src, 'foo')
+        os.symlink(src, src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        # don't follow
+        shutil.copy(src_link, dst, symlinks=False)
+        self.assertFalse(os.path.islink(dst))
+        self.assertEqual(read_file(src), read_file(dst))
+        os.remove(dst)
+        # follow
+        shutil.copy(src_link, dst, symlinks=True)
+        self.assertTrue(os.path.islink(dst))
+        self.assertEqual(os.readlink(dst), os.readlink(src_link))
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(os.lstat(src_link).st_mode,
+                             os.lstat(dst).st_mode)
+
+    @support.skip_unless_symlink
+    def test_copy2_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        write_file(src, 'foo')
+        os.symlink(src, src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_stat = os.stat(src)
+        src_link_stat = os.lstat(src_link)
+        # follow
+        shutil.copy2(src_link, dst, symlinks=False)
+        self.assertFalse(os.path.islink(dst))
+        self.assertEqual(read_file(src), read_file(dst))
+        os.remove(dst)
+        # don't follow
+        shutil.copy2(src_link, dst, symlinks=True)
+        self.assertTrue(os.path.islink(dst))
+        self.assertEqual(os.readlink(dst), os.readlink(src_link))
+        dst_stat = os.lstat(dst)
+        if hasattr(os, 'lutimes'):
+            for attr in 'st_atime', 'st_mtime':
+                # The modification times may be truncated in the new file.
+                self.assertLessEqual(getattr(src_link_stat, attr),
+                                     getattr(dst_stat, attr) + 1)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(src_link_stat.st_mode, dst_stat.st_mode)
+            self.assertNotEqual(src_stat.st_mode, dst_stat.st_mode)
+        if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'):
+            self.assertEqual(src_link_stat.st_flags, dst_stat.st_flags)
+
+    @support.skip_unless_symlink
+    def test_copyfile_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'src')
+        dst = os.path.join(tmp_dir, 'dst')
+        dst_link = os.path.join(tmp_dir, 'dst_link')
+        link = os.path.join(tmp_dir, 'link')
+        write_file(src, 'foo')
+        os.symlink(src, link)
+        # don't follow
+        shutil.copyfile(link, dst_link, symlinks=True)
+        self.assertTrue(os.path.islink(dst_link))
+        self.assertEqual(os.readlink(link), os.readlink(dst_link))
+        # follow
+        shutil.copyfile(link, dst)
+        self.assertFalse(os.path.islink(dst))
+
     def test_rmtree_dont_delete_file(self):
         # When called on a file instead of a directory, don't delete it.
         handle, path = tempfile.mkstemp()
@@ -190,6 +381,34 @@
         actual = read_file((dst_dir, 'test_dir', 'test.txt'))
         self.assertEqual(actual, '456')
 
+    @support.skip_unless_symlink
+    def test_copytree_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src_dir = os.path.join(tmp_dir, 'src')
+        dst_dir = os.path.join(tmp_dir, 'dst')
+        sub_dir = os.path.join(src_dir, 'sub')
+        os.mkdir(src_dir)
+        os.mkdir(sub_dir)
+        write_file((src_dir, 'file.txt'), 'foo')
+        src_link = os.path.join(sub_dir, 'link')
+        dst_link = os.path.join(dst_dir, 'sub/link')
+        os.symlink(os.path.join(src_dir, 'file.txt'),
+                   src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_stat = os.lstat(src_link)
+        shutil.copytree(src_dir, dst_dir, symlinks=True)
+        self.assertTrue(os.path.islink(os.path.join(dst_dir, 'sub', 'link')))
+        self.assertEqual(os.readlink(os.path.join(dst_dir, 'sub', 'link')),
+                         os.path.join(src_dir, 'file.txt'))
+        dst_stat = os.lstat(dst_link)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(dst_stat.st_mode, src_stat.st_mode)
+        if hasattr(os, 'lchflags'):
+            self.assertEqual(dst_stat.st_flags, src_stat.st_flags)
+
     def test_copytree_with_exclude(self):
         # creating data
         join = os.path.join
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -422,6 +422,11 @@
 Library
 -------
 
+- Issue #12715: Add an optional symlinks argument to shutil functions
+  (copyfile, copymode, copystat, copy, copy2).  When that parameter is
+  true, symlinks aren't dereferenced and the operation instead acts on the
+  symlink itself (or creates one, if relevant).  Patch by Hynek Schlawack.
+
 - Add a flags parameter to select.epoll.
 
 - Issue #12798: Updated the mimetypes documentation.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 29 19:04:38 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 29 Dec 2011 19:04:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_run_AST_toolchain_on_python?=
	=?utf8?q?3?=
Message-ID: <E1RgKLG-0006wK-Rf@dinsdale.python.org>

http://hg.python.org/cpython/rev/ac100a4e18b8
changeset:   74195:ac100a4e18b8
parent:      74193:e3ba7e1dd406
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 29 12:03:55 2011 -0600
summary:
  run AST toolchain on python3

files:
  Parser/asdl_c.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py
--- a/Parser/asdl_c.py
+++ b/Parser/asdl_c.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 """Generate C code from an ASDL description."""
 
 # TO DO

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 29 19:04:39 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 29 Dec 2011 19:04:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_default_-=3E_default?=
	=?utf8?q?=29=3A_merge_heads?=
Message-ID: <E1RgKLH-0006wU-QS@dinsdale.python.org>

http://hg.python.org/cpython/rev/1477d7de4473
changeset:   74196:1477d7de4473
parent:      74195:ac100a4e18b8
parent:      74194:cf57ef65bcd0
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 29 12:04:28 2011 -0600
summary:
  merge heads

files:
  Doc/library/shutil.rst  |   46 ++++-
  Lib/shutil.py           |  101 +++++++++---
  Lib/test/test_shutil.py |  219 ++++++++++++++++++++++++++++
  Misc/NEWS               |    5 +
  4 files changed, 333 insertions(+), 38 deletions(-)


diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -45,7 +45,7 @@
    be copied.
 
 
-.. function:: copyfile(src, dst)
+.. function:: copyfile(src, dst[, symlinks=False])
 
    Copy the contents (no metadata) of the file named *src* to a file named *dst*.
    *dst* must be the complete target file name; look at :func:`copy` for a copy that
@@ -56,37 +56,56 @@
    such as character or block devices and pipes cannot be copied with this
    function.  *src* and *dst* are path names given as strings.
 
+   If *symlinks* is true and *src* is a symbolic link, a new symbolic link will
+   be created instead of copying the file *src* points to.
+
    .. versionchanged:: 3.3
       :exc:`IOError` used to be raised instead of :exc:`OSError`.
+      Added *symlinks* argument.
 
 
-.. function:: copymode(src, dst)
+.. function:: copymode(src, dst[, symlinks=False])
 
    Copy the permission bits from *src* to *dst*.  The file contents, owner, and
-   group are unaffected.  *src* and *dst* are path names given as strings.
+   group are unaffected.  *src* and *dst* are path names given as strings.  If
+   *symlinks* is true, *src* a symbolic link and the operating system supports
+   modes for symbolic links (for example BSD-based ones), the mode of the link
+   will be copied.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copystat(src, dst)
+.. function:: copystat(src, dst[, symlinks=False])
 
    Copy the permission bits, last access time, last modification time, and flags
    from *src* to *dst*.  The file contents, owner, and group are unaffected.  *src*
-   and *dst* are path names given as strings.
+   and *dst* are path names given as strings.  If *src* and *dst* are both
+   symbolic links and *symlinks* true, the stats of the link will be copied as
+   far as the platform allows.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copy(src, dst)
+.. function:: copy(src, dst[, symlinks=False]))
 
    Copy the file *src* to the file or directory *dst*.  If *dst* is a directory, a
    file with the same basename as *src*  is created (or overwritten) in the
    directory specified.  Permission bits are copied.  *src* and *dst* are path
-   names given as strings.
+   names given as strings.  If *symlinks* is true, symbolic links won't be
+   followed but recreated instead -- this resembles GNU's :program:`cp -P`.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
-.. function:: copy2(src, dst)
+.. function:: copy2(src, dst[, symlinks=False])
 
    Similar to :func:`copy`, but metadata is copied as well -- in fact, this is just
    :func:`copy` followed by :func:`copystat`.  This is similar to the
-   Unix command :program:`cp -p`.
+   Unix command :program:`cp -p`.  If *symlinks* is true, symbolic links won't
+   be followed but recreated instead -- this resembles GNU's :program:`cp -P`.
 
+   .. versionchanged:: 3.3
+      Added *symlinks* argument.
 
 .. function:: ignore_patterns(\*patterns)
 
@@ -104,9 +123,9 @@
    :func:`copy2`.
 
    If *symlinks* is true, symbolic links in the source tree are represented as
-   symbolic links in the new tree, but the metadata of the original links is NOT
-   copied; if false or omitted, the contents and metadata of the linked files
-   are copied to the new tree.
+   symbolic links in the new tree and the metadata of the original links will
+   be copied as far as the platform allows; if false or omitted, the contents
+   and metadata of the linked files are copied to the new tree.
 
    When *symlinks* is false, if the file pointed by the symlink doesn't
    exist, a exception will be added in the list of errors raised in
@@ -140,6 +159,9 @@
       Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
       errors when *symlinks* is false.
 
+   .. versionchanged:: 3.3
+      Copy metadata when *symlinks* is false.
+
 
 .. function:: rmtree(path, ignore_errors=False, onerror=None)
 
diff --git a/Lib/shutil.py b/Lib/shutil.py
--- a/Lib/shutil.py
+++ b/Lib/shutil.py
@@ -82,8 +82,13 @@
     return (os.path.normcase(os.path.abspath(src)) ==
             os.path.normcase(os.path.abspath(dst)))
 
-def copyfile(src, dst):
-    """Copy data from src to dst"""
+def copyfile(src, dst, symlinks=False):
+    """Copy data from src to dst.
+
+    If optional flag `symlinks` is set and `src` is a symbolic link, a new
+    symlink will be created instead of copying the file it points to.
+
+    """
     if _samefile(src, dst):
         raise Error("`%s` and `%s` are the same file" % (src, dst))
 
@@ -98,54 +103,94 @@
             if stat.S_ISFIFO(st.st_mode):
                 raise SpecialFileError("`%s` is a named pipe" % fn)
 
-    with open(src, 'rb') as fsrc:
-        with open(dst, 'wb') as fdst:
-            copyfileobj(fsrc, fdst)
+    if symlinks and os.path.islink(src):
+        os.symlink(os.readlink(src), dst)
+    else:
+        with open(src, 'rb') as fsrc:
+            with open(dst, 'wb') as fdst:
+                copyfileobj(fsrc, fdst)
 
-def copymode(src, dst):
-    """Copy mode bits from src to dst"""
-    if hasattr(os, 'chmod'):
-        st = os.stat(src)
-        mode = stat.S_IMODE(st.st_mode)
-        os.chmod(dst, mode)
+def copymode(src, dst, symlinks=False):
+    """Copy mode bits from src to dst.
 
-def copystat(src, dst):
-    """Copy all stat info (mode bits, atime, mtime, flags) from src to dst"""
-    st = os.stat(src)
+    If the optional flag `symlinks` is set, symlinks aren't followed if and
+    only if both `src` and `dst` are symlinks. If `lchmod` isn't available (eg.
+    Linux), in these cases, this method does nothing.
+
+    """
+    if symlinks and os.path.islink(src) and os.path.islink(dst):
+        if hasattr(os, 'lchmod'):
+            stat_func, chmod_func = os.lstat, os.lchmod
+        else:
+            return
+    elif hasattr(os, 'chmod'):
+        stat_func, chmod_func = os.stat, os.chmod
+    else:
+        return
+
+    st = stat_func(src)
+    chmod_func(dst, stat.S_IMODE(st.st_mode))
+
+def copystat(src, dst, symlinks=False):
+    """Copy all stat info (mode bits, atime, mtime, flags) from src to dst.
+
+    If the optional flag `symlinks` is set, symlinks aren't followed if and
+    only if both `src` and `dst` are symlinks.
+
+    """
+    def _nop(*args):
+        pass
+
+    if symlinks and os.path.islink(src) and os.path.islink(dst):
+        stat_func = os.lstat
+        utime_func = os.lutimes if hasattr(os, 'lutimes') else _nop
+        chmod_func = os.lchmod if hasattr(os, 'lchmod') else _nop
+        chflags_func = os.lchflags if hasattr(os, 'lchflags') else _nop
+    else:
+        stat_func = os.stat
+        utime_func = os.utime if hasattr(os, 'utime') else _nop
+        chmod_func = os.chmod if hasattr(os, 'chmod') else _nop
+        chflags_func = os.chflags if hasattr(os, 'chflags') else _nop
+
+    st = stat_func(src)
     mode = stat.S_IMODE(st.st_mode)
-    if hasattr(os, 'utime'):
-        os.utime(dst, (st.st_atime, st.st_mtime))
-    if hasattr(os, 'chmod'):
-        os.chmod(dst, mode)
-    if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):
+    utime_func(dst, (st.st_atime, st.st_mtime))
+    chmod_func(dst, mode)
+    if hasattr(st, 'st_flags'):
         try:
-            os.chflags(dst, st.st_flags)
+            chflags_func(dst, st.st_flags)
         except OSError as why:
             if (not hasattr(errno, 'EOPNOTSUPP') or
                 why.errno != errno.EOPNOTSUPP):
                 raise
 
-def copy(src, dst):
+def copy(src, dst, symlinks=False):
     """Copy data and mode bits ("cp src dst").
 
     The destination may be a directory.
 
+    If the optional flag `symlinks` is set, symlinks won't be followed. This
+    resembles GNU's "cp -P src dst".
+
     """
     if os.path.isdir(dst):
         dst = os.path.join(dst, os.path.basename(src))
-    copyfile(src, dst)
-    copymode(src, dst)
+    copyfile(src, dst, symlinks=symlinks)
+    copymode(src, dst, symlinks=symlinks)
 
-def copy2(src, dst):
+def copy2(src, dst, symlinks=False):
     """Copy data and all stat info ("cp -p src dst").
 
     The destination may be a directory.
 
+    If the optional flag `symlinks` is set, symlinks won't be followed. This
+    resembles GNU's "cp -P src dst".
+
     """
     if os.path.isdir(dst):
         dst = os.path.join(dst, os.path.basename(src))
-    copyfile(src, dst)
-    copystat(src, dst)
+    copyfile(src, dst, symlinks=symlinks)
+    copystat(src, dst, symlinks=symlinks)
 
 def ignore_patterns(*patterns):
     """Function that can be used as copytree() ignore parameter.
@@ -212,7 +257,11 @@
             if os.path.islink(srcname):
                 linkto = os.readlink(srcname)
                 if symlinks:
+                    # We can't just leave it to `copy_function` because legacy
+                    # code with a custom `copy_function` may rely on copytree
+                    # doing the right thing.
                     os.symlink(linkto, dstname)
+                    copystat(srcname, dstname, symlinks=symlinks)
                 else:
                     # ignore dangling symlink if the flag is on
                     if not os.path.exists(linkto) and ignore_dangling_symlinks:
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -164,6 +164,197 @@
             self.assertTrue(issubclass(exc[0], OSError))
             self.errorState = 2
 
+    @unittest.skipUnless(hasattr(os, 'chmod'), 'requires os.chmod')
+    @support.skip_unless_symlink
+    def test_copymode_follow_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        os.chmod(src, stat.S_IRWXU|stat.S_IRWXG)
+        # file to file
+        os.chmod(dst, stat.S_IRWXO)
+        self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        shutil.copymode(src, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow src link
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src_link, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow dst link
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src, dst_link)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # follow both links
+        os.chmod(dst, stat.S_IRWXO)
+        shutil.copymode(src_link, dst)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+
+    @unittest.skipUnless(hasattr(os, 'lchmod'), 'requires os.lchmod')
+    @support.skip_unless_symlink
+    def test_copymode_symlink_to_symlink(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        os.chmod(src, stat.S_IRWXU|stat.S_IRWXG)
+        os.chmod(dst, stat.S_IRWXU)
+        os.lchmod(src_link, stat.S_IRWXO|stat.S_IRWXG)
+        # link to link
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src_link, dst_link, symlinks=True)
+        self.assertEqual(os.lstat(src_link).st_mode,
+                         os.lstat(dst_link).st_mode)
+        self.assertNotEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # src link - use chmod
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src_link, dst, symlinks=True)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+        # dst link - use chmod
+        os.lchmod(dst_link, stat.S_IRWXO)
+        shutil.copymode(src, dst_link, symlinks=True)
+        self.assertEqual(os.stat(src).st_mode, os.stat(dst).st_mode)
+
+    @unittest.skipIf(hasattr(os, 'lchmod'), 'requires os.lchmod to be missing')
+    @support.skip_unless_symlink
+    def test_copymode_symlink_to_symlink_wo_lchmod(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'quux')
+        write_file(src, 'foo')
+        write_file(dst, 'foo')
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        shutil.copymode(src_link, dst_link, symlinks=True)  # silent fail
+
+    @support.skip_unless_symlink
+    def test_copystat_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        dst_link = os.path.join(tmp_dir, 'qux')
+        write_file(src, 'foo')
+        src_stat = os.stat(src)
+        os.utime(src, (src_stat.st_atime,
+                       src_stat.st_mtime - 42.0))  # ensure different mtimes
+        write_file(dst, 'bar')
+        self.assertNotEqual(os.stat(src).st_mtime, os.stat(dst).st_mtime)
+        os.symlink(src, src_link)
+        os.symlink(dst, dst_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_link_stat = os.lstat(src_link)
+        # follow
+        if hasattr(os, 'lchmod'):
+            shutil.copystat(src_link, dst_link, symlinks=False)
+            self.assertNotEqual(src_link_stat.st_mode, os.stat(dst).st_mode)
+        # don't follow
+        shutil.copystat(src_link, dst_link, symlinks=True)
+        dst_link_stat = os.lstat(dst_link)
+        if hasattr(os, 'lutimes'):
+            for attr in 'st_atime', 'st_mtime':
+                # The modification times may be truncated in the new file.
+                self.assertLessEqual(getattr(src_link_stat, attr),
+                                     getattr(dst_link_stat, attr) + 1)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(src_link_stat.st_mode, dst_link_stat.st_mode)
+        if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'):
+            self.assertEqual(src_link_stat.st_flags, dst_link_stat.st_flags)
+        # tell to follow but dst is not a link
+        shutil.copystat(src_link, dst, symlinks=True)
+        self.assertTrue(abs(os.stat(src).st_mtime - os.stat(dst).st_mtime) <
+                        00000.1)
+
+    @support.skip_unless_symlink
+    def test_copy_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        write_file(src, 'foo')
+        os.symlink(src, src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        # don't follow
+        shutil.copy(src_link, dst, symlinks=False)
+        self.assertFalse(os.path.islink(dst))
+        self.assertEqual(read_file(src), read_file(dst))
+        os.remove(dst)
+        # follow
+        shutil.copy(src_link, dst, symlinks=True)
+        self.assertTrue(os.path.islink(dst))
+        self.assertEqual(os.readlink(dst), os.readlink(src_link))
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(os.lstat(src_link).st_mode,
+                             os.lstat(dst).st_mode)
+
+    @support.skip_unless_symlink
+    def test_copy2_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'foo')
+        dst = os.path.join(tmp_dir, 'bar')
+        src_link = os.path.join(tmp_dir, 'baz')
+        write_file(src, 'foo')
+        os.symlink(src, src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_stat = os.stat(src)
+        src_link_stat = os.lstat(src_link)
+        # follow
+        shutil.copy2(src_link, dst, symlinks=False)
+        self.assertFalse(os.path.islink(dst))
+        self.assertEqual(read_file(src), read_file(dst))
+        os.remove(dst)
+        # don't follow
+        shutil.copy2(src_link, dst, symlinks=True)
+        self.assertTrue(os.path.islink(dst))
+        self.assertEqual(os.readlink(dst), os.readlink(src_link))
+        dst_stat = os.lstat(dst)
+        if hasattr(os, 'lutimes'):
+            for attr in 'st_atime', 'st_mtime':
+                # The modification times may be truncated in the new file.
+                self.assertLessEqual(getattr(src_link_stat, attr),
+                                     getattr(dst_stat, attr) + 1)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(src_link_stat.st_mode, dst_stat.st_mode)
+            self.assertNotEqual(src_stat.st_mode, dst_stat.st_mode)
+        if hasattr(os, 'lchflags') and hasattr(src_link_stat, 'st_flags'):
+            self.assertEqual(src_link_stat.st_flags, dst_stat.st_flags)
+
+    @support.skip_unless_symlink
+    def test_copyfile_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src = os.path.join(tmp_dir, 'src')
+        dst = os.path.join(tmp_dir, 'dst')
+        dst_link = os.path.join(tmp_dir, 'dst_link')
+        link = os.path.join(tmp_dir, 'link')
+        write_file(src, 'foo')
+        os.symlink(src, link)
+        # don't follow
+        shutil.copyfile(link, dst_link, symlinks=True)
+        self.assertTrue(os.path.islink(dst_link))
+        self.assertEqual(os.readlink(link), os.readlink(dst_link))
+        # follow
+        shutil.copyfile(link, dst)
+        self.assertFalse(os.path.islink(dst))
+
     def test_rmtree_dont_delete_file(self):
         # When called on a file instead of a directory, don't delete it.
         handle, path = tempfile.mkstemp()
@@ -190,6 +381,34 @@
         actual = read_file((dst_dir, 'test_dir', 'test.txt'))
         self.assertEqual(actual, '456')
 
+    @support.skip_unless_symlink
+    def test_copytree_symlinks(self):
+        tmp_dir = self.mkdtemp()
+        src_dir = os.path.join(tmp_dir, 'src')
+        dst_dir = os.path.join(tmp_dir, 'dst')
+        sub_dir = os.path.join(src_dir, 'sub')
+        os.mkdir(src_dir)
+        os.mkdir(sub_dir)
+        write_file((src_dir, 'file.txt'), 'foo')
+        src_link = os.path.join(sub_dir, 'link')
+        dst_link = os.path.join(dst_dir, 'sub/link')
+        os.symlink(os.path.join(src_dir, 'file.txt'),
+                   src_link)
+        if hasattr(os, 'lchmod'):
+            os.lchmod(src_link, stat.S_IRWXU | stat.S_IRWXO)
+        if hasattr(os, 'lchflags') and hasattr(stat, 'UF_NODUMP'):
+            os.lchflags(src_link, stat.UF_NODUMP)
+        src_stat = os.lstat(src_link)
+        shutil.copytree(src_dir, dst_dir, symlinks=True)
+        self.assertTrue(os.path.islink(os.path.join(dst_dir, 'sub', 'link')))
+        self.assertEqual(os.readlink(os.path.join(dst_dir, 'sub', 'link')),
+                         os.path.join(src_dir, 'file.txt'))
+        dst_stat = os.lstat(dst_link)
+        if hasattr(os, 'lchmod'):
+            self.assertEqual(dst_stat.st_mode, src_stat.st_mode)
+        if hasattr(os, 'lchflags'):
+            self.assertEqual(dst_stat.st_flags, src_stat.st_flags)
+
     def test_copytree_with_exclude(self):
         # creating data
         join = os.path.join
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -422,6 +422,11 @@
 Library
 -------
 
+- Issue #12715: Add an optional symlinks argument to shutil functions
+  (copyfile, copymode, copystat, copy, copy2).  When that parameter is
+  true, symlinks aren't dereferenced and the operation instead acts on the
+  symlink itself (or creates one, if relevant).  Patch by Hynek Schlawack.
+
 - Add a flags parameter to select.epoll.
 
 - Issue #12798: Updated the mimetypes documentation.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Dec 29 19:07:31 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Thu, 29 Dec 2011 19:07:31 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_evidently_some_buildbots_do?=
 =?utf8?q?n=27t_have_python_3_installed?=
Message-ID: <E1RgKO3-0007Ap-TJ@dinsdale.python.org>

http://hg.python.org/cpython/rev/7010fa9bd190
changeset:   74197:7010fa9bd190
user:        Benjamin Peterson <benjamin at python.org>
date:        Thu Dec 29 12:07:21 2011 -0600
summary:
  evidently some buildbots don't have python 3 installed

files:
  Parser/asdl_c.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py
--- a/Parser/asdl_c.py
+++ b/Parser/asdl_c.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python3
+#! /usr/bin/env python
 """Generate C code from an ASDL description."""
 
 # TO DO

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Dec 30 05:32:56 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 30 Dec 2011 05:32:56 +0100
Subject: [Python-checkins] Daily reference leaks (7010fa9bd190): sum=0
Message-ID: <E1RgU9I-0001To-8t@ap.vmr.nerim.net>

results for 7010fa9bd190 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogS_Ohzs', '-x']

From brian at python.org  Fri Dec 30 20:29:36 2011
From: brian at python.org (Brian Curtin)
Date: Fri, 30 Dec 2011 13:29:36 -0600
Subject: [Python-checkins] cpython: Issue #12715: Add an optional
 symlinks argument to shutil functions (copyfile, 
In-Reply-To: <E1RgKC8-0006BJ-9N@dinsdale.python.org>
References: <E1RgKC8-0006BJ-9N@dinsdale.python.org>
Message-ID: <CAD+XWwpSMA0E6euhMZO5eZvWsYJCN6-KikMa9ryLt8cd3OkJ3Q@mail.gmail.com>

On Thu, Dec 29, 2011 at 11:55, antoine.pitrou
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/cf57ef65bcd0
> changeset: ? 74194:cf57ef65bcd0
> user: ? ? ? ?Antoine Pitrou <solipsis at pitrou.net>
> date: ? ? ? ?Thu Dec 29 18:54:15 2011 +0100
> summary:
> ?Issue #12715: Add an optional symlinks argument to shutil functions (copyfile, copymode, copystat, copy, copy2).
> When that parameter is true, symlinks aren't dereferenced and the operation
> instead acts on the symlink itself (or creates one, if relevant).
>
> Patch by Hynek Schlawack.
>
> files:
> ?Doc/library/shutil.rst ?| ? 46 ++++-
> ?Lib/shutil.py ? ? ? ? ? | ?101 +++++++++---
> ?Lib/test/test_shutil.py | ?219 ++++++++++++++++++++++++++++
> ?Misc/NEWS ? ? ? ? ? ? ? | ? ?5 +
> ?4 files changed, 333 insertions(+), 38 deletions(-)
>
>
> diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
> --- a/Doc/library/shutil.rst
> +++ b/Doc/library/shutil.rst
> @@ -45,7 +45,7 @@
> ? ?be copied.
>
>
> -.. function:: copyfile(src, dst)
> +.. function:: copyfile(src, dst[, symlinks=False])
>
> ? ?Copy the contents (no metadata) of the file named *src* to a file named *dst*.
> ? ?*dst* must be the complete target file name; look at :func:`copy` for a copy that
> @@ -56,37 +56,56 @@
> ? ?such as character or block devices and pipes cannot be copied with this
> ? ?function. ?*src* and *dst* are path names given as strings.
>
> + ? If *symlinks* is true and *src* is a symbolic link, a new symbolic link will
> + ? be created instead of copying the file *src* points to.
> +
> ? ?.. versionchanged:: 3.3
> ? ? ? :exc:`IOError` used to be raised instead of :exc:`OSError`.
> + ? ? ?Added *symlinks* argument.

Can we expect that readers on Windows know how os.symlink works, or
should the stipulations of os.symlink usage also be laid out or at
least linked to from there?

Basically, almost everyone is going to get an OSError if they call
this on Windows. You have to be on Windows Vista or beyond *and* the
calling process has to have the proper privileges (typically gained
through elevation - "Run as Administrator").

From python-checkins at python.org  Fri Dec 30 20:47:56 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Fri, 30 Dec 2011 20:47:56 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_use_booleans?=
Message-ID: <E1RgiQm-0004LJ-Hv@dinsdale.python.org>

http://hg.python.org/cpython/rev/d6f3e0b3c6fa
changeset:   74198:d6f3e0b3c6fa
branch:      3.2
parent:      74184:4964712a7269
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 30 13:47:25 2011 -0600
summary:
  use booleans

files:
  Doc/library/logging.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst
--- a/Doc/library/logging.rst
+++ b/Doc/library/logging.rst
@@ -65,7 +65,7 @@
    If this evaluates to false, logging messages are not passed to the handlers
    of ancestor loggers.
 
-   The constructor sets this attribute to 1.
+   The constructor sets this attribute to ``True``.
 
 
 .. method:: Logger.setLevel(lvl)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 30 20:47:57 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Fri, 30 Dec 2011 20:47:57 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1RgiQn-0004LU-G1@dinsdale.python.org>

http://hg.python.org/cpython/rev/f2b9f1d1dd6e
changeset:   74199:f2b9f1d1dd6e
parent:      74197:7010fa9bd190
parent:      74198:d6f3e0b3c6fa
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 30 13:47:49 2011 -0600
summary:
  merge 3.2

files:
  Doc/library/logging.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst
--- a/Doc/library/logging.rst
+++ b/Doc/library/logging.rst
@@ -65,7 +65,7 @@
    If this evaluates to false, logging messages are not passed to the handlers
    of ancestor loggers.
 
-   The constructor sets this attribute to 1.
+   The constructor sets this attribute to ``True``.
 
 
 .. method:: Logger.setLevel(lvl)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 30 20:56:28 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Fri, 30 Dec 2011 20:56:28 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_use_booleans?=
Message-ID: <E1RgiZ2-0004vS-Ti@dinsdale.python.org>

http://hg.python.org/cpython/rev/5adbc87f1028
changeset:   74200:5adbc87f1028
branch:      2.7
parent:      74191:9b681e0c04ed
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 30 13:47:25 2011 -0600
summary:
  use booleans

files:
  Doc/library/logging.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst
--- a/Doc/library/logging.rst
+++ b/Doc/library/logging.rst
@@ -67,7 +67,7 @@
    If this evaluates to false, logging messages are not passed to the handlers
    of ancestor loggers.
 
-   The constructor sets this attribute to 1.
+   The constructor sets this attribute to ``True``.
 
 
 .. method:: Logger.setLevel(lvl)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 30 21:27:38 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 30 Dec 2011 21:27:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Fix_no-op_tests?=
	=?utf8?q?_in_importlib=2E?=
Message-ID: <E1Rgj3C-0006zb-Ki@dinsdale.python.org>

http://hg.python.org/cpython/rev/cd30304713ae
changeset:   74201:cd30304713ae
branch:      3.2
parent:      74198:d6f3e0b3c6fa
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 30 21:25:15 2011 +0100
summary:
  Fix no-op tests in importlib.

files:
  Lib/importlib/test/source/test_file_loader.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/importlib/test/source/test_file_loader.py b/Lib/importlib/test/source/test_file_loader.py
--- a/Lib/importlib/test/source/test_file_loader.py
+++ b/Lib/importlib/test/source/test_file_loader.py
@@ -256,6 +256,8 @@
             with open(bytecode_path, 'rb') as file:
                 self.assertGreater(len(file.read()), 8)
 
+        self._test_magic_only(test)
+
     @source_util.writes_bytecode_files
     def test_bad_magic(self):
         # When the magic number is different, the bytecode should be
@@ -276,6 +278,8 @@
             with open(bc_path, 'rb') as file:
                 self.assertGreater(len(file.read()), 8)
 
+        self._test_partial_timestamp(test)
+
     @source_util.writes_bytecode_files
     def test_no_marshal(self):
         # When there is only the magic number and timestamp, raise EOFError.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Dec 30 21:27:39 2011
From: python-checkins at python.org (antoine.pitrou)
Date: Fri, 30 Dec 2011 21:27:39 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Fix_no-op_tests_in_importlib=2E?=
Message-ID: <E1Rgj3D-0006zl-Ik@dinsdale.python.org>

http://hg.python.org/cpython/rev/656fa057aa28
changeset:   74202:656fa057aa28
parent:      74199:f2b9f1d1dd6e
parent:      74201:cd30304713ae
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Fri Dec 30 21:26:08 2011 +0100
summary:
  Fix no-op tests in importlib.

files:
  Lib/importlib/test/source/test_file_loader.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/importlib/test/source/test_file_loader.py b/Lib/importlib/test/source/test_file_loader.py
--- a/Lib/importlib/test/source/test_file_loader.py
+++ b/Lib/importlib/test/source/test_file_loader.py
@@ -256,6 +256,8 @@
             with open(bytecode_path, 'rb') as file:
                 self.assertGreater(len(file.read()), 8)
 
+        self._test_magic_only(test)
+
     @source_util.writes_bytecode_files
     def test_bad_magic(self):
         # When the magic number is different, the bytecode should be
@@ -276,6 +278,8 @@
             with open(bc_path, 'rb') as file:
                 self.assertGreater(len(file.read()), 8)
 
+        self._test_partial_timestamp(test)
+
     @source_util.writes_bytecode_files
     def test_no_marshal(self):
         # When there is only the magic number and timestamp, raise EOFError.

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Dec 31 05:31:58 2011
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 31 Dec 2011 05:31:58 +0100
Subject: [Python-checkins] Daily reference leaks (656fa057aa28): sum=0
Message-ID: <E1Rgqbu-0001Mg-6M@ap.vmr.nerim.net>

results for 656fa057aa28 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloguYoz1d', '-x']

From python-checkins at python.org  Sat Dec 31 06:05:20 2011
From: python-checkins at python.org (eli.bendersky)
Date: Sat, 31 Dec 2011 06:05:20 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_Some_formatting?=
 =?utf8?q?_=26_grammar_fixes_for_the_multiprocessing_doc?=
Message-ID: <E1Rgr8C-0005E1-A8@dinsdale.python.org>

http://hg.python.org/cpython/rev/fb5ad61a2b35
changeset:   74203:fb5ad61a2b35
branch:      2.7
parent:      74200:5adbc87f1028
user:        Eli Bendersky <eliben at gmail.com>
date:        Sat Dec 31 07:05:12 2011 +0200
summary:
  Some formatting & grammar fixes for the multiprocessing doc

files:
  Doc/library/multiprocessing.rst |  34 ++++++++++----------
  1 files changed, 17 insertions(+), 17 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -282,7 +282,7 @@
 
    if __name__ == '__main__':
        pool = Pool(processes=4)              # start 4 worker processes
-       result = pool.apply_async(f, [10])     # evaluate "f(10)" asynchronously
+       result = pool.apply_async(f, [10])    # evaluate "f(10)" asynchronously
        print result.get(timeout=1)           # prints "100" unless your computer is *very* slow
        print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"
 
@@ -472,7 +472,7 @@
 
 If you use :class:`JoinableQueue` then you **must** call
 :meth:`JoinableQueue.task_done` for each task removed from the queue or else the
-semaphore used to count the number of unfinished tasks may eventually overflow
+semaphore used to count the number of unfinished tasks may eventually overflow,
 raising an exception.
 
 Note that one can also create a shared queue by using a manager object -- see
@@ -490,7 +490,7 @@
 
    If a process is killed using :meth:`Process.terminate` or :func:`os.kill`
    while it is trying to use a :class:`Queue`, then the data in the queue is
-   likely to become corrupted.  This may cause any other processes to get an
+   likely to become corrupted.  This may cause any other process to get an
    exception when it tries to use the queue later on.
 
 .. warning::
@@ -692,7 +692,7 @@
    (By default :data:`sys.executable` is used).  Embedders will probably need to
    do some thing like ::
 
-      setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
+      set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
 
    before they can create child processes.  (Windows only)
 
@@ -711,7 +711,7 @@
 Connection objects allow the sending and receiving of picklable objects or
 strings.  They can be thought of as message oriented connected sockets.
 
-Connection objects usually created using :func:`Pipe` -- see also
+Connection objects are usually created using :func:`Pipe` -- see also
 :ref:`multiprocessing-listeners-clients`.
 
 .. class:: Connection
@@ -722,7 +722,7 @@
       using :meth:`recv`.
 
       The object must be picklable.  Very large pickles (approximately 32 MB+,
-      though it depends on the OS) may raise a ValueError exception.
+      though it depends on the OS) may raise a :exc:`ValueError` exception.
 
    .. method:: recv()
 
@@ -732,7 +732,7 @@
 
    .. method:: fileno()
 
-      Returns the file descriptor or handle used by the connection.
+      Return the file descriptor or handle used by the connection.
 
    .. method:: close()
 
@@ -756,7 +756,7 @@
       If *offset* is given then data is read from that position in *buffer*.  If
       *size* is given then that many bytes will be read from buffer.  Very large
       buffers (approximately 32 MB+, though it depends on the OS) may raise a
-      ValueError exception
+      :exc:`ValueError` exception
 
    .. method:: recv_bytes([maxlength])
 
@@ -1329,7 +1329,7 @@
 >>>>>>>>>>>>>>>>>>>
 
 To create one's own manager, one creates a subclass of :class:`BaseManager` and
-use the :meth:`~BaseManager.register` classmethod to register new types or
+uses the :meth:`~BaseManager.register` classmethod to register new types or
 callables with the manager class.  For example::
 
    from multiprocessing.managers import BaseManager
@@ -1579,10 +1579,10 @@
 
    .. method:: apply(func[, args[, kwds]])
 
-      Equivalent of the :func:`apply` built-in function.  It blocks till the
-      result is ready.  Given this blocks, :meth:`apply_async` is better suited
-      for performing work in parallel. Additionally, the passed
-      in function is only executed in one of the workers of the pool.
+      Equivalent of the :func:`apply` built-in function.  It blocks until the
+      result is ready, so :meth:`apply_async` is better suited for performing
+      work in parallel. Additionally, *func* is only executed in one of the
+      workers of the pool.
 
    .. method:: apply_async(func[, args[, kwds[, callback]]])
 
@@ -1596,7 +1596,7 @@
    .. method:: map(func, iterable[, chunksize])
 
       A parallel equivalent of the :func:`map` built-in function (it supports only
-      one *iterable* argument though).  It blocks till the result is ready.
+      one *iterable* argument though).  It blocks until the result is ready.
 
       This method chops the iterable into a number of chunks which it submits to
       the process pool as separate tasks.  The (approximate) size of these
@@ -2046,7 +2046,7 @@
     On Windows many types from :mod:`multiprocessing` need to be picklable so
     that child processes can use them.  However, one should generally avoid
     sending shared objects to other processes using pipes or queues.  Instead
-    you should arrange the program so that a process which need access to a
+    you should arrange the program so that a process which needs access to a
     shared resource created elsewhere can inherit it from an ancestor process.
 
 Avoid terminating processes
@@ -2125,7 +2125,7 @@
            for i in range(10):
                 Process(target=f, args=(lock,)).start()
 
-Beware replacing sys.stdin with a "file like object"
+Beware of replacing :data:`sys.stdin` with a "file like object"
 
     :mod:`multiprocessing` originally unconditionally called::
 
@@ -2243,7 +2243,7 @@
 
 
 An example showing how to use queues to feed tasks to a collection of worker
-process and collect the results:
+processes and collect the results:
 
 .. literalinclude:: ../includes/mp_workers.py
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 06:09:38 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 31 Dec 2011 06:09:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_indicate_msg_is?=
	=?utf8?q?_an_argument?=
Message-ID: <E1RgrCM-0005MJ-7M@dinsdale.python.org>

http://hg.python.org/cpython/rev/ce349e1f923e
changeset:   74204:ce349e1f923e
branch:      3.2
parent:      74201:cd30304713ae
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 30 23:08:09 2011 -0600
summary:
  indicate msg is an argument

files:
  Doc/library/smtplib.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -281,10 +281,10 @@
       envelope used by the transport agents.  ``sendmail`` does not modify the
       message headers in any way.
 
-   msg may be a string containing characters in the ASCII range, or a byte
+   *msg* may be a string containing characters in the ASCII range, or a byte
    string.  A string is encoded to bytes using the ascii codec, and lone ``\r``
-   and ``\n`` characters are converted to ``\r\n`` characters.  A byte string
-   is not modified.
+   and ``\n`` characters are converted to ``\r\n`` characters.  A byte string is
+   not modified.
 
    If there has been no previous ``EHLO`` or ``HELO`` command this session, this
    method tries ESMTP ``EHLO`` first. If the server does ESMTP, message size and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 06:09:38 2011
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 31 Dec 2011 06:09:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_3=2E2?=
Message-ID: <E1RgrCM-0005MT-Vy@dinsdale.python.org>

http://hg.python.org/cpython/rev/dbd3013c2685
changeset:   74205:dbd3013c2685
parent:      74202:656fa057aa28
parent:      74204:ce349e1f923e
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri Dec 30 23:09:20 2011 -0600
summary:
  merge 3.2

files:
  Doc/library/smtplib.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -323,10 +323,10 @@
       envelope used by the transport agents.  ``sendmail`` does not modify the
       message headers in any way.
 
-   msg may be a string containing characters in the ASCII range, or a byte
+   *msg* may be a string containing characters in the ASCII range, or a byte
    string.  A string is encoded to bytes using the ascii codec, and lone ``\r``
-   and ``\n`` characters are converted to ``\r\n`` characters.  A byte string
-   is not modified.
+   and ``\n`` characters are converted to ``\r\n`` characters.  A byte string is
+   not modified.
 
    If there has been no previous ``EHLO`` or ``HELO`` command this session, this
    method tries ESMTP ``EHLO`` first. If the server does ESMTP, message size and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 06:22:36 2011
From: python-checkins at python.org (eli.bendersky)
Date: Sat, 31 Dec 2011 06:22:36 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_Some_formatting?=
 =?utf8?q?_=26_grammar_fixes_for_the_multiprocessing_doc?=
Message-ID: <E1RgrOu-00068q-5n@dinsdale.python.org>

http://hg.python.org/cpython/rev/2c3a769089d5
changeset:   74206:2c3a769089d5
branch:      3.2
parent:      74204:ce349e1f923e
user:        Eli Bendersky <eliben at gmail.com>
date:        Sat Dec 31 07:20:26 2011 +0200
summary:
  Some formatting & grammar fixes for the multiprocessing doc

files:
  Doc/library/multiprocessing.rst |  28 ++++++++++----------
  1 files changed, 14 insertions(+), 14 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -472,7 +472,7 @@
 
 If you use :class:`JoinableQueue` then you **must** call
 :meth:`JoinableQueue.task_done` for each task removed from the queue or else the
-semaphore used to count the number of unfinished tasks may eventually overflow
+semaphore used to count the number of unfinished tasks may eventually overflow,
 raising an exception.
 
 Note that one can also create a shared queue by using a manager object -- see
@@ -490,7 +490,7 @@
 
    If a process is killed using :meth:`Process.terminate` or :func:`os.kill`
    while it is trying to use a :class:`Queue`, then the data in the queue is
-   likely to become corrupted.  This may cause any other processes to get an
+   likely to become corrupted.  This may cause any other process to get an
    exception when it tries to use the queue later on.
 
 .. warning::
@@ -692,7 +692,7 @@
    (By default :data:`sys.executable` is used).  Embedders will probably need to
    do some thing like ::
 
-      setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
+      set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
 
    before they can create child processes.  (Windows only)
 
@@ -711,7 +711,7 @@
 Connection objects allow the sending and receiving of picklable objects or
 strings.  They can be thought of as message oriented connected sockets.
 
-Connection objects usually created using :func:`Pipe` -- see also
+Connection objects are usually created using :func:`Pipe` -- see also
 :ref:`multiprocessing-listeners-clients`.
 
 .. class:: Connection
@@ -732,7 +732,7 @@
 
    .. method:: fileno()
 
-      Returns the file descriptor or handle used by the connection.
+      Return the file descriptor or handle used by the connection.
 
    .. method:: close()
 
@@ -756,7 +756,7 @@
       If *offset* is given then data is read from that position in *buffer*.  If
       *size* is given then that many bytes will be read from buffer.  Very large
       buffers (approximately 32 MB+, though it depends on the OS) may raise a
-      ValueError exception
+      :exc:`ValueError` exception
 
    .. method:: recv_bytes([maxlength])
 
@@ -1329,7 +1329,7 @@
 >>>>>>>>>>>>>>>>>>>
 
 To create one's own manager, one creates a subclass of :class:`BaseManager` and
-use the :meth:`~BaseManager.register` classmethod to register new types or
+uses the :meth:`~BaseManager.register` classmethod to register new types or
 callables with the manager class.  For example::
 
    from multiprocessing.managers import BaseManager
@@ -1580,9 +1580,9 @@
    .. method:: apply(func[, args[, kwds]])
 
       Call *func* with arguments *args* and keyword arguments *kwds*.  It blocks
-      till the result is ready. Given this blocks, :meth:`apply_async` is better
-      suited for performing work in parallel. Additionally, the passed in
-      function is only executed in one of the workers of the pool.
+      until the result is ready. Given this blocks, :meth:`apply_async` is
+      better suited for performing work in parallel. Additionally, *func*
+      is only executed in one of the workers of the pool.
 
    .. method:: apply_async(func[, args[, kwds[, callback[, error_callback]]]])
 
@@ -1603,7 +1603,7 @@
    .. method:: map(func, iterable[, chunksize])
 
       A parallel equivalent of the :func:`map` built-in function (it supports only
-      one *iterable* argument though).  It blocks till the result is ready.
+      one *iterable* argument though).  It blocks until the result is ready.
 
       This method chops the iterable into a number of chunks which it submits to
       the process pool as separate tasks.  The (approximate) size of these
@@ -2060,7 +2060,7 @@
     On Windows many types from :mod:`multiprocessing` need to be picklable so
     that child processes can use them.  However, one should generally avoid
     sending shared objects to other processes using pipes or queues.  Instead
-    you should arrange the program so that a process which need access to a
+    you should arrange the program so that a process which needs access to a
     shared resource created elsewhere can inherit it from an ancestor process.
 
 Avoid terminating processes
@@ -2139,7 +2139,7 @@
            for i in range(10):
                 Process(target=f, args=(lock,)).start()
 
-Beware replacing sys.stdin with a "file like object"
+Beware of replacing :data:`sys.stdin` with a "file like object"
 
     :mod:`multiprocessing` originally unconditionally called::
 
@@ -2257,7 +2257,7 @@
 
 
 An example showing how to use queues to feed tasks to a collection of worker
-process and collect the results:
+processes and collect the results:
 
 .. literalinclude:: ../includes/mp_workers.py
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 06:22:38 2011
From: python-checkins at python.org (eli.bendersky)
Date: Sat, 31 Dec 2011 06:22:38 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
 =?utf8?q?=3A_Some_formatting_=26_grammar_fixes_for_the_multiprocessing_do?=
 =?utf8?q?c?=
Message-ID: <E1RgrOw-000690-EI@dinsdale.python.org>

http://hg.python.org/cpython/rev/73481bd9ca54
changeset:   74207:73481bd9ca54
parent:      74205:dbd3013c2685
parent:      74206:2c3a769089d5
user:        Eli Bendersky <eliben at gmail.com>
date:        Sat Dec 31 07:22:27 2011 +0200
summary:
  Some formatting & grammar fixes for the multiprocessing doc

files:
  Doc/library/multiprocessing.rst |  28 ++++++++++----------
  1 files changed, 14 insertions(+), 14 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -491,7 +491,7 @@
 
 If you use :class:`JoinableQueue` then you **must** call
 :meth:`JoinableQueue.task_done` for each task removed from the queue or else the
-semaphore used to count the number of unfinished tasks may eventually overflow
+semaphore used to count the number of unfinished tasks may eventually overflow,
 raising an exception.
 
 Note that one can also create a shared queue by using a manager object -- see
@@ -509,7 +509,7 @@
 
    If a process is killed using :meth:`Process.terminate` or :func:`os.kill`
    while it is trying to use a :class:`Queue`, then the data in the queue is
-   likely to become corrupted.  This may cause any other processes to get an
+   likely to become corrupted.  This may cause any other process to get an
    exception when it tries to use the queue later on.
 
 .. warning::
@@ -711,7 +711,7 @@
    (By default :data:`sys.executable` is used).  Embedders will probably need to
    do some thing like ::
 
-      setExecutable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
+      set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
 
    before they can create child processes.  (Windows only)
 
@@ -730,7 +730,7 @@
 Connection objects allow the sending and receiving of picklable objects or
 strings.  They can be thought of as message oriented connected sockets.
 
-Connection objects usually created using :func:`Pipe` -- see also
+Connection objects are usually created using :func:`Pipe` -- see also
 :ref:`multiprocessing-listeners-clients`.
 
 .. class:: Connection
@@ -751,7 +751,7 @@
 
    .. method:: fileno()
 
-      Returns the file descriptor or handle used by the connection.
+      Return the file descriptor or handle used by the connection.
 
    .. method:: close()
 
@@ -775,7 +775,7 @@
       If *offset* is given then data is read from that position in *buffer*.  If
       *size* is given then that many bytes will be read from buffer.  Very large
       buffers (approximately 32 MB+, though it depends on the OS) may raise a
-      ValueError exception
+      :exc:`ValueError` exception
 
    .. method:: recv_bytes([maxlength])
 
@@ -1353,7 +1353,7 @@
 >>>>>>>>>>>>>>>>>>>
 
 To create one's own manager, one creates a subclass of :class:`BaseManager` and
-use the :meth:`~BaseManager.register` classmethod to register new types or
+uses the :meth:`~BaseManager.register` classmethod to register new types or
 callables with the manager class.  For example::
 
    from multiprocessing.managers import BaseManager
@@ -1604,9 +1604,9 @@
    .. method:: apply(func[, args[, kwds]])
 
       Call *func* with arguments *args* and keyword arguments *kwds*.  It blocks
-      till the result is ready. Given this blocks, :meth:`apply_async` is better
-      suited for performing work in parallel. Additionally, the passed in
-      function is only executed in one of the workers of the pool.
+      until the result is ready. Given this blocks, :meth:`apply_async` is
+      better suited for performing work in parallel. Additionally, *func*
+      is only executed in one of the workers of the pool.
 
    .. method:: apply_async(func[, args[, kwds[, callback[, error_callback]]]])
 
@@ -1627,7 +1627,7 @@
    .. method:: map(func, iterable[, chunksize])
 
       A parallel equivalent of the :func:`map` built-in function (it supports only
-      one *iterable* argument though).  It blocks till the result is ready.
+      one *iterable* argument though).  It blocks until the result is ready.
 
       This method chops the iterable into a number of chunks which it submits to
       the process pool as separate tasks.  The (approximate) size of these
@@ -2102,7 +2102,7 @@
     On Windows many types from :mod:`multiprocessing` need to be picklable so
     that child processes can use them.  However, one should generally avoid
     sending shared objects to other processes using pipes or queues.  Instead
-    you should arrange the program so that a process which need access to a
+    you should arrange the program so that a process which needs access to a
     shared resource created elsewhere can inherit it from an ancestor process.
 
 Avoid terminating processes
@@ -2181,7 +2181,7 @@
            for i in range(10):
                 Process(target=f, args=(lock,)).start()
 
-Beware replacing sys.stdin with a "file like object"
+Beware of replacing :data:`sys.stdin` with a "file like object"
 
     :mod:`multiprocessing` originally unconditionally called::
 
@@ -2299,7 +2299,7 @@
 
 
 An example showing how to use queues to feed tasks to a collection of worker
-process and collect the results:
+processes and collect the results:
 
 .. literalinclude:: ../includes/mp_workers.py
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 12:59:58 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 31 Dec 2011 12:59:58 +0100
Subject: [Python-checkins] =?utf8?q?cpython=3A_fix_Exemples/Examples_typo?=
 =?utf8?q?=3B_thanks_to_Ori_Livneh_from_docs=40?=
Message-ID: <E1RgxbS-0002dE-Ss@dinsdale.python.org>

http://hg.python.org/cpython/rev/43b10c2d2553
changeset:   74208:43b10c2d2553
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 31 12:59:08 2011 +0100
summary:
  fix Exemples/Examples typo; thanks to Ori Livneh from docs@

files:
  Doc/library/packaging.pypi.simple.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/packaging.pypi.simple.rst b/Doc/library/packaging.pypi.simple.rst
--- a/Doc/library/packaging.pypi.simple.rst
+++ b/Doc/library/packaging.pypi.simple.rst
@@ -89,7 +89,7 @@
 
 
 
-Usage Exemples
+Usage Examples
 ---------------
 
 To help you understand how using the `Crawler` class, here are some basic

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 19:05:08 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 31 Dec 2011 19:05:08 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=283=2E2=29=3A_fix_some_broken?=
	=?utf8?q?_URLs?=
Message-ID: <E1Rh3Iq-0007WP-NT@dinsdale.python.org>

http://hg.python.org/cpython/rev/ed4f7d9e0e96
changeset:   74209:ed4f7d9e0e96
branch:      3.2
parent:      74206:2c3a769089d5
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 31 18:46:50 2011 +0100
summary:
  fix some broken URLs

files:
  Doc/howto/functional.rst |   6 +++---
  Doc/howto/webservers.rst |  10 +++++-----
  Doc/library/atexit.rst   |   2 --
  Doc/using/unix.rst       |   4 ++--
  4 files changed, 10 insertions(+), 12 deletions(-)


diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -1151,9 +1151,9 @@
 
 Mertz also wrote a 3-part series of articles on functional programming
 for IBM's DeveloperWorks site; see
-`part 1 <http://www-128.ibm.com/developerworks/library/l-prog.html>`__,
-`part 2 <http://www-128.ibm.com/developerworks/library/l-prog2.html>`__, and
-`part 3 <http://www-128.ibm.com/developerworks/linux/library/l-prog3.html>`__,
+`part 1 <http://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__,
+`part 2 <http://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and
+`part 3 <http://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__,
 
 
 Python documentation
diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst
--- a/Doc/howto/webservers.rst
+++ b/Doc/howto/webservers.rst
@@ -274,7 +274,7 @@
 Each web server requires a specific module.
 
 * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid
-  <http://fastcgi.coremail.cn/>`_.  ``mod_fastcgi`` is the original one, but it
+  <http://httpd.apache.org/mod_fcgid/>`_.  ``mod_fastcgi`` is the original one, but it
   has some licensing issues, which is why it is sometimes considered non-free.
   ``mod_fcgid`` is a smaller, compatible alternative.  One of these modules needs
   to be loaded by Apache.
@@ -365,7 +365,7 @@
 
 A really great WSGI feature is middleware.  Middleware is a layer around your
 program which can add various functionality to it.  There is quite a bit of
-`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already
+`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already
 available.  For example, instead of writing your own session management (HTTP
 is a stateless protocol, so to associate multiple HTTP requests with a single
 user your application must create and manage such state via a session), you can
@@ -396,9 +396,9 @@
 
 .. seealso::
 
-   A good overview of WSGI-related code can be found in the `WSGI wiki
-   <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers
-   <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application
+   A good overview of WSGI-related code can be found in the `WSGI homepage
+   <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers
+   <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application
    supporting WSGI.
 
    You might be interested in some WSGI-supporting modules already contained in
diff --git a/Doc/library/atexit.rst b/Doc/library/atexit.rst
--- a/Doc/library/atexit.rst
+++ b/Doc/library/atexit.rst
@@ -6,9 +6,6 @@
 .. moduleauthor:: Skip Montanaro <skip at pobox.com>
 .. sectionauthor:: Skip Montanaro <skip at pobox.com>
 
-**Source code:** :source:`Lib/atexit.py`
-
---------------
 
 The :mod:`atexit` module defines functions to register and unregister cleanup
 functions.  Functions thus registered are automatically executed upon normal
diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst
--- a/Doc/using/unix.rst
+++ b/Doc/using/unix.rst
@@ -26,11 +26,11 @@
 
 .. seealso::
 
-   http://www.linux.com/articles/60383
+   http://www.debian.org/doc/manuals/maint-guide/first.en.html
       for Debian users
    http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html
       for OpenSuse users
-   http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html
+   http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html
       for Fedora users
    http://www.slackbook.org/html/package-management-making-packages.html
       for Slackware users

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 19:05:09 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 31 Dec 2011 19:05:09 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=282=2E7=29=3A_fix_some_broken?=
	=?utf8?q?_URLs?=
Message-ID: <E1Rh3Ir-0007WZ-Fc@dinsdale.python.org>

http://hg.python.org/cpython/rev/e9f517f8ee87
changeset:   74210:e9f517f8ee87
branch:      2.7
parent:      74203:fb5ad61a2b35
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 31 18:13:59 2011 +0100
summary:
  fix some broken URLs

files:
  Doc/bugs.rst                |   6 ++++--
  Doc/howto/webservers.rst    |  10 +++++-----
  Doc/library/carbon.rst      |   4 ++--
  Doc/library/easydialogs.rst |   2 +-
  Doc/using/unix.rst          |   4 ++--
  5 files changed, 14 insertions(+), 12 deletions(-)


diff --git a/Doc/bugs.rst b/Doc/bugs.rst
--- a/Doc/bugs.rst
+++ b/Doc/bugs.rst
@@ -57,12 +57,14 @@
 
 Each bug report will be assigned to a developer who will determine what needs to
 be done to correct the problem.  You will receive an update each time action is
-taken on the bug.  See http://www.python.org/dev/workflow/ for a detailed
-description of the issue workflow.
+taken on the bug.
 
 
 .. seealso::
 
+   `Python Developer's Guide <http://docs.python.org/devguide/>`_
+      Detailed description of the issue workflow and developers tools.
+
    `How to Report Bugs Effectively <http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>`_
       Article which goes into some detail about how to create a useful bug report.
       This describes what kind of information is useful and why it is useful.
diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst
--- a/Doc/howto/webservers.rst
+++ b/Doc/howto/webservers.rst
@@ -274,7 +274,7 @@
 Each web server requires a specific module.
 
 * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid
-  <http://fastcgi.coremail.cn/>`_.  ``mod_fastcgi`` is the original one, but it
+  <http://httpd.apache.org/mod_fcgid/>`_.  ``mod_fastcgi`` is the original one, but it
   has some licensing issues, which is why it is sometimes considered non-free.
   ``mod_fcgid`` is a smaller, compatible alternative.  One of these modules needs
   to be loaded by Apache.
@@ -364,7 +364,7 @@
 
 A really great WSGI feature is middleware.  Middleware is a layer around your
 program which can add various functionality to it.  There is quite a bit of
-`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already
+`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already
 available.  For example, instead of writing your own session management (HTTP
 is a stateless protocol, so to associate multiple HTTP requests with a single
 user your application must create and manage such state via a session), you can
@@ -395,9 +395,9 @@
 
 .. seealso::
 
-   A good overview of WSGI-related code can be found in the `WSGI wiki
-   <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers
-   <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application
+   A good overview of WSGI-related code can be found in the `WSGI homepage
+   <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers
+   <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application
    supporting WSGI.
 
    You might be interested in some WSGI-supporting modules already contained in
diff --git a/Doc/library/carbon.rst b/Doc/library/carbon.rst
--- a/Doc/library/carbon.rst
+++ b/Doc/library/carbon.rst
@@ -14,7 +14,7 @@
 in Python (input and output buffers, especially).  All methods and functions
 have a :attr:`__doc__` string describing their arguments and return values, and
 for additional description you are referred to `Inside Macintosh
-<http://developer.apple.com/documentation/macos8/mac8.html>`_ or similar works.
+<http://developer.apple.com/legacy/mac/library/#documentation/macos8/mac8.html>`_ or similar works.
 
 These modules all live in a package called :mod:`Carbon`. Despite that name they
 are not all part of the Carbon framework: CF is really in the CoreFoundation
@@ -515,7 +515,7 @@
 
 .. seealso::
 
-   `Scrap Manager <http://developer.apple.com/documentation/mac/MoreToolbox/MoreToolbox-109.html>`_
+   `Scrap Manager <http://developer.apple.com/legacy/mac/library/documentation/mac/MoreToolbox/MoreToolbox-109.html>`_
       Apple's documentation for the Scrap Manager gives a lot of useful information
       about using the Scrap Manager in applications.
 
diff --git a/Doc/library/easydialogs.rst b/Doc/library/easydialogs.rst
--- a/Doc/library/easydialogs.rst
+++ b/Doc/library/easydialogs.rst
@@ -143,7 +143,7 @@
 
 .. seealso::
 
-   `Navigation Services Reference <http://developer.apple.com/documentation/Carbon/Reference/Navigation_Services_Ref/>`_
+   `Navigation Services Reference <http://developer.apple.com/legacy/mac/library/#documentation/Carbon/Conceptual/NavServicesIntro/ns_intro_carb/ns_into_carb.html>`_
       Programmer's reference documentation for the Navigation Services, a part of the
       Carbon framework.
 
diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst
--- a/Doc/using/unix.rst
+++ b/Doc/using/unix.rst
@@ -26,11 +26,11 @@
 
 .. seealso::
 
-   http://www.linux.com/articles/60383
+   http://www.debian.org/doc/manuals/maint-guide/first.en.html
       for Debian users
    http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html
       for OpenSuse users
-   http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html
+   http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html
       for Fedora users
    http://www.slackbook.org/html/package-management-making-packages.html
       for Slackware users

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Dec 31 19:05:10 2011
From: python-checkins at python.org (sandro.tosi)
Date: Sat, 31 Dec 2011 19:05:10 +0100
Subject: [Python-checkins] =?utf8?q?cpython_=28merge_3=2E2_-=3E_default=29?=
	=?utf8?q?=3A_merge_with_3=2E2?=
Message-ID: <E1Rh3Is-0007We-9r@dinsdale.python.org>

http://hg.python.org/cpython/rev/0ddb0bb63550
changeset:   74211:0ddb0bb63550
parent:      74208:43b10c2d2553
parent:      74209:ed4f7d9e0e96
user:        Sandro Tosi <sandro.tosi at gmail.com>
date:        Sat Dec 31 19:03:30 2011 +0100
summary:
  merge with 3.2

files:
  Doc/howto/functional.rst |   6 +++---
  Doc/howto/webservers.rst |  10 +++++-----
  Doc/library/atexit.rst   |   2 --
  Doc/using/unix.rst       |   4 ++--
  4 files changed, 10 insertions(+), 12 deletions(-)


diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -1151,9 +1151,9 @@
 
 Mertz also wrote a 3-part series of articles on functional programming
 for IBM's DeveloperWorks site; see
-`part 1 <http://www-128.ibm.com/developerworks/library/l-prog.html>`__,
-`part 2 <http://www-128.ibm.com/developerworks/library/l-prog2.html>`__, and
-`part 3 <http://www-128.ibm.com/developerworks/linux/library/l-prog3.html>`__,
+`part 1 <http://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__,
+`part 2 <http://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and
+`part 3 <http://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__,
 
 
 Python documentation
diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst
--- a/Doc/howto/webservers.rst
+++ b/Doc/howto/webservers.rst
@@ -274,7 +274,7 @@
 Each web server requires a specific module.
 
 * Apache has both `mod_fastcgi <http://www.fastcgi.com/drupal/>`_ and `mod_fcgid
-  <http://fastcgi.coremail.cn/>`_.  ``mod_fastcgi`` is the original one, but it
+  <http://httpd.apache.org/mod_fcgid/>`_.  ``mod_fastcgi`` is the original one, but it
   has some licensing issues, which is why it is sometimes considered non-free.
   ``mod_fcgid`` is a smaller, compatible alternative.  One of these modules needs
   to be loaded by Apache.
@@ -365,7 +365,7 @@
 
 A really great WSGI feature is middleware.  Middleware is a layer around your
 program which can add various functionality to it.  There is quite a bit of
-`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already
+`middleware <http://www.wsgi.org/en/latest/libraries.html>`_ already
 available.  For example, instead of writing your own session management (HTTP
 is a stateless protocol, so to associate multiple HTTP requests with a single
 user your application must create and manage such state via a session), you can
@@ -396,9 +396,9 @@
 
 .. seealso::
 
-   A good overview of WSGI-related code can be found in the `WSGI wiki
-   <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers
-   <http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application
+   A good overview of WSGI-related code can be found in the `WSGI homepage
+   <http://www.wsgi.org/en/latest/index.html>`_, which contains an extensive list of `WSGI servers
+   <http://www.wsgi.org/en/latest/servers.html>`_ which can be used by *any* application
    supporting WSGI.
 
    You might be interested in some WSGI-supporting modules already contained in
diff --git a/Doc/library/atexit.rst b/Doc/library/atexit.rst
--- a/Doc/library/atexit.rst
+++ b/Doc/library/atexit.rst
@@ -6,9 +6,6 @@
 .. moduleauthor:: Skip Montanaro <skip at pobox.com>
 .. sectionauthor:: Skip Montanaro <skip at pobox.com>
 
-**Source code:** :source:`Lib/atexit.py`
-
---------------
 
 The :mod:`atexit` module defines functions to register and unregister cleanup
 functions.  Functions thus registered are automatically executed upon normal
diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst
--- a/Doc/using/unix.rst
+++ b/Doc/using/unix.rst
@@ -26,11 +26,11 @@
 
 .. seealso::
 
-   http://www.linux.com/articles/60383
+   http://www.debian.org/doc/manuals/maint-guide/first.en.html
       for Debian users
    http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html
       for OpenSuse users
-   http://docs.fedoraproject.org/drafts/rpm-guide-en/ch-creating-rpms.html
+   http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html
       for Fedora users
    http://www.slackbook.org/html/package-management-making-packages.html
       for Slackware users

-- 
Repository URL: http://hg.python.org/cpython