[docs] [issue13997] Clearly explain the bare minimum Python 3 users should know about Unicode

[Nick Coghlan]

Nick Coghlan <ncoghlan at gmail.com> added the comment:

If such use cases are indeed better handled as bytes, then that's what should be documented. However, there are some text processing assumptions that no longer hold when using bytes instead of strings (such as "x[0:1] == x[0]"). You also can't safely pass such byte sequences to various other APIs (e.g. urllib.parse will happily process surrogate escaped text without corrupting them, but will throw UnicodeDecodeError for bytes sequences that aren't pure 7-bit ASCII).

Using surrogateescape instead means that you're only going to have problems if you go to encode the data to an encoding other than the source one. That's basically the things work in Python 2 with 8-bit strings.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13997>
_______________________________________