[issue2382] [Py3k] SyntaxError cursor shifted if multibyte character is in line.

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

For an easier review, I splitted my patch in multiple small patches:
 - unicode_utf8size.patch: create _PyUnicode_UTF8Size() function: 
Number of bytes needed to encode the unicode character as UTF-8
 - unicode_width.patch: create PyUnicode_Width(): Number of column 
needed to represent the string in the current locale. -1 is returned 
in case of an error.
 - adjust_offset.patch: Change unit of SyntaxError.offset, convert 
utf8 offset to unicode offset
 - print_exception.patch: process error text as an unicode string 
(instead of a byte string), convert offset from characters 
to "columns"

Dependencies:
 - adjust_offset.patch depends on unicode_utf8size.patch
 - print_exception.patch depends on unicode_width.patch

Changes since issue2382.patch:
 - PyUnicode_Width() doesn't change the locale
 - PyUnicode_Width() uses WideCharToMultiByte() on MS_WINDOWS, and 
wcswidth() otherwise (before: do nothing if HAVE_WCSWIDTH is not 
definied)
 - the offset was converted from utf8 index to unicode index only in 
print_error_text(), not on SyntaxError creation
 - _PyUnicode_UTF8Size() and PyUnicode_Width() are public

----------
Added file: http://bugs.python.org/file13356/unicode_utf8size.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2382>
_______________________________________