[Python-Dev] IDLE and non-ASCII characters

[M.-A. Lemburg]

Martin von Loewis wrote:
> 
> Thanks to a bug report I got, I noticed for the first time that you
> cannot enter non-ASCII characters in IDLE anymore. Eg. at the shell
> prompt, you may get
> 
> >>> s='дц'
> UnicodeError: ASCII encoding error: ordinal not in range(128)
> 
> Likewise, when trying to save a file that has non-ASCII characters,
> you get a traceback.
> 
> Now, I think I understand all the causes of the problem (Tkinter
> returning Unicode objects, and so on). However, I'm curious whether
> anybody has proposals on how to deal with it.
> 
> For saving text files, if Python had an encoding directive, things
> might be easier :-) For the shell prompt, I've no idea how to solve
> this best.
> 
> So any suggestions are welcome.

I have a bug report assigned to myself which indicates similar
problems with _tkinter and Tk/Tcl. There were other problem
reports on the German Python mailing list going in the same
direction too.

The basic problem seems to be that Tk/Tcl applies too much
magic to the text widget contents in order to find out the
used encoding and this can easily cause the whole encoding
mechanism to fail.

A Tk/Tcl expert should really look into this and fix _tkinter.c
to aid Tk/Tcl in not mixing up the encodings (e.g. it would
probably be a good idea to recode Python 8bit-strings into
whatever encoding Tk/Tcl assumes as default).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/