Problem with curses and UTF-8

[Thomas Dickey]

Ian Ward <ian at excess.org> wrote:
> Martin v. Löwis wrote:

>> If that was Python's configure: don't do that. Instead, hack setup.py
>> to make it change the compiler/linker settings, or even edit the
>> compiler/linker line manually at first.

> Ok, that compiled.

same here - though it was not immediately not clear which copy of ncurses it's
using (not the shared libraries since I installed those with tracing - a
little odd for it to use the static library, but that's what the access time
tells me).

To check on that (since I wanted to read the ncurses trace),
I ran strace and ltrace to look for clues.

> Now when I run the same test:

> import curses
> s = curses.initscr()
> s.addstr('\xc3\x85 U+00C5 LATIN CAPITAL LETTER A WITH RING ABOVE\n')
> s.addstr('\xc3\xa5 U+00F5 LATIN SMALL LETTER O WITH TILDE')
> s.refresh()
> s.getstr()
> curses.endwin()

Testing this, and looking to see what's going on, I notice that python
is doing a

	setlocale(LC_ALL, "C");

before the addstr is actually called.  (ncurses never sets the locale;
it calls setlocale in one place to ask what it is).

That makes ncurses think it's not really doing UTF-8, of course.  What I
see on the screen is the U+00C5 comes out with a box and a "~E" (the
latter being ncurses' representation in POSIX for \0x85).

> This is what I see:

>   +00C5 LATIN CAPITAL LETTER A WITH RING ABOVE
>   +00F5 LATIN SMALL LETTER O WITH TILDE


> so, the UTF-8 characters didn't appear and the " U" at the beginning 
> became just " ".

well - running in uxterm I see the second line properly.  But some more
tinkering is needed to make python work properly.

-- 
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net