[Python-ideas] Processing surrogates in

[Stephen J. Turnbull]

Chris Barker writes:

 > I've read many of the rants about UTF-16, but in fact, it's really
 > not any worse than UTF-8

Yes, it is.  It's not ASCII compatible.  You can safely use the usual
libc string APIs on UTF-8 (except for any that might return only part
of a string), but not on UTF-16 (nulls).  This is a pretty big
advantage for UTF-8 in practice.