[Python-checkins] cpython: Improve / clean up the PEP 393 description
antoine.pitrou
python-checkins at python.org
Mon Oct 24 00:18:45 CEST 2011
http://hg.python.org/cpython/rev/5880584cafe6
changeset: 73089:5880584cafe6
user: Antoine Pitrou <solipsis at pitrou.net>
date: Mon Oct 24 00:14:43 2011 +0200
summary:
Improve / clean up the PEP 393 description
files:
Doc/whatsnew/3.3.rst | 36 +++++++++++++++++--------------
1 files changed, 20 insertions(+), 16 deletions(-)
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -52,25 +52,27 @@
PEP 393: Flexible String Representation
=======================================
-[Abstract copied from the PEP: The Unicode string type is changed to support
-multiple internal representations, depending on the character with the largest
-Unicode ordinal (1, 2, or 4 bytes). This allows a space-efficient
-representation in common cases, but gives access to full UCS-4 on all systems.
-For compatibility with existing APIs, several representations may exist in
-parallel; over time, this compatibility should be phased out.]
+The Unicode string type is changed to support multiple internal
+representations, depending on the character with the largest Unicode ordinal
+(1, 2, or 4 bytes) in the represented string. This allows a space-efficient
+representation in common cases, but gives access to full UCS-4 on all
+systems. For compatibility with existing APIs, several representations may
+exist in parallel; over time, this compatibility should be phased out.
-PEP 393 is fully backward compatible. The legacy API should remain
-available at least five years. Applications using the legacy API will not
-fully benefit of the memory reduction, or worse may use a little bit more
-memory, because Python may have to maintain two versions of each string (in
-the legacy format and in the new efficient storage).
+On the Python side, there should be no downside to this change.
-XXX Add list of changes introduced by :pep:`393` here:
+On the C API side, PEP 393 is fully backward compatible. The legacy API
+should remain available at least five years. Applications using the legacy
+API will not fully benefit of the memory reduction, or - worse - may use
+a bit more memory, because Python may have to maintain two versions of each
+string (in the legacy format and in the new efficient storage).
+
+Changes introduced by :pep:`393` are the following:
* Python now always supports the full range of Unicode codepoints, including
non-BMP ones (i.e. from ``U+0000`` to ``U+10FFFF``). The distinction between
narrow and wide builds no longer exists and Python now behaves like a wide
- build.
+ build, even under Windows.
* The storage of Unicode strings now depends on the highest codepoint in the string:
@@ -86,7 +88,8 @@
XXX The result should be moved in the PEP and a small summary about
performances and a link to the PEP should be added here.
-* Some of the problems visible on narrow builds have been fixed, for example:
+* With the death of narrow builds, the problems specific to narrow builds have
+ also been fixed, for example:
* :func:`len` now always returns 1 for non-BMP characters,
so ``len('\U0010FFFF') == 1``;
@@ -94,10 +97,11 @@
* surrogate pairs are not recombined in string literals,
so ``'\uDBFF\uDFFF' != '\U0010FFFF'``;
- * indexing or slicing a non-BMP characters doesn't return surrogates anymore,
+ * indexing or slicing non-BMP characters returns the expected value,
so ``'\U0010FFFF'[0]`` now returns ``'\U0010FFFF'`` and not ``'\uDBFF'``;
- * several other functions in the stdlib now handle correctly non-BMP codepoints.
+ * several other functions in the standard library now handle correctly
+ non-BMP codepoints.
* The value of :data:`sys.maxunicode` is now always ``1114111`` (``0x10FFFF``
in hexadecimal). The :c:func:`PyUnicode_GetMax` function still returns
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list