Unicode filenames
Peter Otten
__peter__ at web.de
Sat Dec 7 04:10:56 EST 2019
Bob van der Poel wrote:
> I have some files which came off the net with, I'm assuming, unicode
> characters in the names. I have a very short program which takes the
> filename and puts into an emacs buffer, and then lets me add information
> to that new file (it's a poor man's DB).
>
> Next, I can look up text in the file and open the saved filename.
> Everything works great until I hit those darn unicode filenames.
>
> Just to confuse me even more, the error seems to be coming from a bit of
> tkinter code:
> if sresults.has_key(textAtCursor):
> bookname = os.path.expanduser(sresults[textAtCursor].strip())
>
> which generates
>
> UnicodeWarning: Unicode equal comparison failed to convert both
> arguments
> to Unicode - interpreting them as being unequal if
> sresults.has_key(textAtCursor):
>
> I really don't understand the business about "both arguments". Not sure
> how to proceed here. Hoping for a guideline!
I cannot provoke the error with dict.has_key() over here, only with direct
comparisons:
>>> u"a" == u"ä"
False
>>> u"a" == "ä"
__main__:1: UnicodeWarning: Unicode equal comparison failed to convert both
arguments to Unicode - interpreting them as being unequal
False
The problem is that you are mixing strings of type str and type unicode, and
generally speaking the remedy is to use unicode throughout. In your case
this means opening files with io.open() or codecs.open() instead of the
builtin, and invoking os.listdir() with a unicode argument.
I don't remember about Tkinter, I think it provides ascii-only strings as
str and everything else as unicode. If that's correct you could play it safe
with a conversion function:
def ensure_unicode(s):
if isinstance(s, bytes):
return s.decode("ascii")
return s
Your other option is to live with the *warning* -- it's not an error, just a
reminder that you have to rethink your types once you switch to Python 3.
You can also switch off the message with
python -W ignore::UnicodeWarning yourscript
or by setting the PYTHONWARNINGS environment variable.
More information about the Python-list
mailing list