[issue34484] Unicode HOWTO incorrectly refers to Private Use Area for surrogateescape
New submission from Mark Dickinson <dickinsm@gmail.com>: The Unicode HOWTO currently has contains this text in the "Files in an Unknown Encoding" section [1]:
The surrogateescape error handler will decode any non-ASCII bytes as code points in the Unicode Private Use Area ranging from U+DC80 to U+DCFF. These private code points will then be turned back into the same bytes when the surrogateescape error handler is used when encoding the data and writing it back out.
Unless I'm missing something, the subrange U+DC80 to U+DCFF of the low surrogates is *not* a Private Use Area. There *is* a kinda-sorta PUA in the high surrogates from U+DB80 to U+DBFF (because the only valid codepoints that use these surrogates in their UTF-16 encoding are the codepoints in planes 15 and 16, which are almost entirely PUA codepoints), but that's not what the surrogateescape handler is using. [1] https://docs.python.org/3/howto/unicode.html#files-in-an-unknown-encoding ---------- assignee: docs@python components: Documentation messages: 323976 nosy: docs@python, mark.dickinson priority: normal severity: normal status: open title: Unicode HOWTO incorrectly refers to Private Use Area for surrogateescape versions: Python 3.6, Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
Mark Dickinson <dickinsm@gmail.com> added the comment: For history, this text was introduced as a result of issue #4163. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
Mark Dickinson <dickinsm@gmail.com> added the comment: Whoops. Sorry, that should be #4153. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
A.M. Kuchling <amk@amk.ca> added the comment: Corrected in the unicode-howto-update branch being developed for issue #20906. ---------- assignee: docs@python -> akuchling nosy: +akuchling _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
Change by A.M. Kuchling <amk@amk.ca>: ---------- stage: -> patch review _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
A.M. Kuchling <amk@amk.ca> added the comment: New changeset 97c288df614dd7856f5a0336925f56a7a2a5bc74 by Andrew Kuchling in branch 'master': bpo-20906: Various revisions to the Unicode howto (#8394) https://github.com/python/cpython/commit/97c288df614dd7856f5a0336925f56a7a2a... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- keywords: +patch pull_requests: +12154 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset 84fa6b9e5932af981cb299c0c5ac80b9cc37c3fa by Miss Islington (bot) in branch '3.7': bpo-20906: Various revisions to the Unicode howto (GH-8394) https://github.com/python/cpython/commit/84fa6b9e5932af981cb299c0c5ac80b9cc3... ---------- nosy: +miss-islington _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
Mark Dickinson <dickinsm@gmail.com> added the comment: Thanks for the fix. @akuchling: safe to close this issue? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
A.M. Kuchling <amk@amk.ca> added the comment: Yes, I think this issue can now be closed. ---------- stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue34484> _______________________________________
participants (3)
-
A.M. Kuchling -
Mark Dickinson
-
miss-islington