[Python-checkins] r87135 - in python/branches/py3k: Doc/library/codecs.rst Lib/test/test_codecs.py

victor.stinner python-checkins at python.org
Wed Dec 8 23:25:45 CET 2010


Author: victor.stinner
Date: Wed Dec  8 23:25:45 2010
New Revision: 87135

Log:
Issue #10546: UTF-16-LE and UTF-16-BE *do* support non-BMP characters

Fix the doc and add tests.


Modified:
   python/branches/py3k/Doc/library/codecs.rst
   python/branches/py3k/Lib/test/test_codecs.py

Modified: python/branches/py3k/Doc/library/codecs.rst
==============================================================================
--- python/branches/py3k/Doc/library/codecs.rst	(original)
+++ python/branches/py3k/Doc/library/codecs.rst	Wed Dec  8 23:25:45 2010
@@ -1114,9 +1114,9 @@
 +-----------------+--------------------------------+--------------------------------+
 | utf_16          | U16, utf16                     | all languages                  |
 +-----------------+--------------------------------+--------------------------------+
-| utf_16_be       | UTF-16BE                       | all languages (BMP only)       |
+| utf_16_be       | UTF-16BE                       | all languages                  |
 +-----------------+--------------------------------+--------------------------------+
-| utf_16_le       | UTF-16LE                       | all languages (BMP only)       |
+| utf_16_le       | UTF-16LE                       | all languages                  |
 +-----------------+--------------------------------+--------------------------------+
 | utf_7           | U7, unicode-1-1-utf-7          | all languages                  |
 +-----------------+--------------------------------+--------------------------------+

Modified: python/branches/py3k/Lib/test/test_codecs.py
==============================================================================
--- python/branches/py3k/Lib/test/test_codecs.py	(original)
+++ python/branches/py3k/Lib/test/test_codecs.py	Wed Dec  8 23:25:45 2010
@@ -544,6 +544,12 @@
         self.assertRaises(UnicodeDecodeError, codecs.utf_16_le_decode,
                           b"\xff", "strict", True)
 
+    def test_nonbmp(self):
+        self.assertEqual("\U00010203".encode(self.encoding),
+                         b'\x00\xd8\x03\xde')
+        self.assertEqual(b'\x00\xd8\x03\xde'.decode(self.encoding),
+                         "\U00010203")
+
 class UTF16BETest(ReadTest):
     encoding = "utf-16-be"
 
@@ -566,6 +572,12 @@
         self.assertRaises(UnicodeDecodeError, codecs.utf_16_be_decode,
                           b"\xff", "strict", True)
 
+    def test_nonbmp(self):
+        self.assertEqual("\U00010203".encode(self.encoding),
+                         b'\xd8\x00\xde\x03')
+        self.assertEqual(b'\xd8\x00\xde\x03'.decode(self.encoding),
+                         "\U00010203")
+
 class UTF8Test(ReadTest):
     encoding = "utf-8"
 


More information about the Python-checkins mailing list