Tkinter and locale

[Martin v. Loewis]

Fredrik Juhlin <laz at strakt.com> writes:

> Normally, I have the environment variable LC_CTYPE set to
> "sv_SE.iso-8859-1" to be able to use latin1 chars in various programs.
> It seems, however, as if TK does not like this, because it makes it
> impossible for me to enter latin1 chars in i.e. entry widgets (pasting
> them in works just fine, though).
> I experimented for a while and soon found that if I just set LC_CTYPE to
> "C" instead, TK worked just great.
> 
> This is where I get confused: 
>  * Why wouldn't TK work properly with my original setting? 

This is difficult to answer (and was even more difficult to
research). The short answer is: use sv_SE.ISO-8859-1 instead. For the
long answer, see below.

> And even more confusing:
>  * Why _would_ TK work with LC_CTYPE=C? From what I've understood from the
>    man pages, C is 7-bit ASCII and if anything should not understand
>    latin1 characters.

In the C locale, X11 will assume that all data is ISO-8859-1. This is
an X11 specific interpretation of the world.

Here is all the glory details: To process a KeyPress event, Tcl
performs

    if (winPtr->dispPtr->useInputMethods
	    && (winPtr->inputContext != NULL)
	    && (eventPtr->type == KeyPress)) {
	len = XmbLookupString(winPtr->inputContext, &eventPtr->xkey,
		Tcl_DStringValue(&buf), Tcl_DStringLength(&buf),
		(KeySym *) NULL, &status);
	/*
	 * If the buffer wasn't big enough, grow the buffer and try again.
	 */

	if (status == XBufferOverflow) {
	    Tcl_DStringSetLength(&buf, len);
	    len = XmbLookupString(winPtr->inputContext, &eventPtr->xkey,
		    Tcl_DStringValue(&buf), len, (KeySym *) NULL, &status);
	}
	if ((status != XLookupChars) && (status != XLookupBoth)) {
	    len = 0;
	}
    } else {
	len = XLookupString(&eventPtr->xkey, Tcl_DStringValue(&buf),
		Tcl_DStringLength(&buf), (KeySym *) NULL,
		(XComposeStatus *) NULL);
    }

(see tkUnixKey.c). The core of this is that it either does
XmbLookupString, which honors locale settings, or XLookupString, which
fails for non-ASCII characters (according to the man page, it ought to
succeed for all Latin-1 characters, but ask google for XLookupString
and LC_CTYPE to get a rough overview of the limitations).

The preferred way to convert keysyms into strings is to use
XmbLookupString, which is done when the inputContext of the window is
set. This, in turn, happens in tkEvent.c, but only if
dispPtr->inputMethod is set, which happens in tkWindow.c:OpenIM, which
starts

    if ((modifier_list = XSetLocaleModifiers("")) == NULL) {
	goto error;
    }

    dispPtr->inputMethod = XOpenIM(dispPtr->display, NULL, NULL, NULL);
    if (dispPtr->inputMethod == NULL) {
	return;
    }

Now, if you set the LC_CTYPE locale to sv_SE.iso-8859-1,
XSetLocaleModifiers will return NULL. It does so because
XSupportsLocale fails. This fails because this locale is not known to
X11; the proper X11 name for the locale is sv_SE.ISO8859-1. 

If you have been following carefully, you may wonder why
sv_SE.ISO-8859-1 is also supported by X11; this is because
<X11>/lib/locale/locale.alias has the entries

sv_SE                              sv_SE.ISO8859-1
sv_SE.88591                        sv_SE.ISO8859-1
sv_SE.88591.en                     sv_SE.ISO8859-1
sv_SE.iso88591                     sv_SE.ISO8859-1
sv_SE.ISO-8859-1                   sv_SE.ISO8859-1
sv_SE.ISO_8859-1                   sv_SE.ISO8859-1

As you can see, a number of alternative spellings of your locale are
supported; sv_SE.iso-8859-1 is not among them. Correcting the spelling
of your locale (as shown above) will make your program work.

If you set the locale to C, X11 will recognize the locale name also,
and Tk will properly use XmbLookupString.

HTH,
Martin