Unicode support of the curses module in Python 3.3
Hi, I changed many functions of the curses module in Python 3.3 to improve its Unicode support: - new functions: curses.unget_wch() and window.get_wch() - new attribute: window.encoding - the default encoding is now the locale encoding instead of UTF-8 - use the C functions *_wch() and *wstr() when available instead of *ch() and *str() functions. For example, the Python function addstr() calls waddwstr() and addch(str) calls wadd_wch() (addch(int) and addch(bytes) are still calling waddch()) Most new features related to Unicode now depends if the Python curses module is linked to the C libncursesw library or not... and the Python module is not linked to libncursesw if the libreadline library is linked to libncurses module. How the readline library is linked to libncurses/libncursesw is not a new problem but it may become more annoying than before. I hope that most Linux distro are/will link readline to libncursesw. For example, if the Python curses module is not linked to libncursesw, get_wch() is not available and addch("é") raises an OverflowError if the locale encoding is UTF-8 (because "é".encode("utf-8") is longer than 1 byte). I introduced two bugs: get_wch() didn't support keycodes (like curses.KEY_UP) and addch() didn't work anymore with special characters like curses.ACS_HLINE. These issues are referenced as #15785 and #14223 in the bug tracker, and I pushed fixes: c58789634d22 and 27b5bd5f0e4c. I hope that Georg will accept them in Python 3.3 final! I didn't find these bugs myself because I only used dummy scripts to test my changes. Does anyone know "real world" applications using the curses module and supporting Python 3? Can you please test them with non-ASCII characters and the last development version of Python 3.3? So please try to test the curses module before Python 3.3 final with your favorite application! Victor
On 01/09/12 23:44, Victor Stinner wrote:
Hi,
I changed many functions of the curses module in Python 3.3 to improve its Unicode support: [...]
Thank you.
For example, if the Python curses module is not linked to libncursesw, get_wch() is not available and addch("é") raises an OverflowError if the locale encoding is UTF-8 (because "é".encode("utf-8") is longer than 1 byte).
OverflowError? That is very surprising. I wouldn't guess that calling addch could raise OverflowError. Could you use a less surprising exception, or at least make sure that it is clearly and obviously documented? -- Steven
For example, if the Python curses module is not linked to libncursesw, get_wch() is not available and addch("é") raises an OverflowError if the locale encoding is UTF-8 (because "é".encode("utf-8") is longer than 1 byte).
OverflowError? That is very surprising. I wouldn't guess that calling addch could raise OverflowError.
Could you use a less surprising exception, or at least make sure that it is clearly and obviously documented?
Which exception would you expect? ValueError? Another error? I used the same exception for addch(int) and addch(str). addch(int) already raises an OverflowError on Python 3.2 if the value doesn't fit in a C long type (which may be bigger than chtype, Python 3.3 is more strict), so applications don't need to be changed for the "new" error (on addch(str)). Victor
participants (2)
-
Steven D'Aprano -
Victor Stinner