Hi, Slowly, we get recurrent questions about Python3 and unicode. It's maybe time to start a FAQ? Here is an ugly draft to start it ;-) (1) Exit on undecodable command line arguments $ LANG=en_GB.UTF-8 python3.0 test.py $'\xff' Could not convert argument 2 to string$ Is it an expected behaviour? Yes! Example of the question: http://bugs.python.org/issue3023 (2) Undecodable filenames os.listdir(str)->str raises an exception on undecodable filenames. Solution: use os.listdir(bytes)->bytes. To display the filename to the user, use a function like: import sys def humanFilename(filename): encoding = sys.getfilesystemencoding() return filename.encode(encoding, "replace") See also http://bugs.python.org/issue3187 (3) Bytes environment variables Python 3.0 only supports decodable variables for os.environ. Undecodable variables are skipped for the creation of os.environ but original variables still exist at the C level. $ A=$(echo -e "\xff") B=c ./python Python 3.1a0 (py3k:67973M, Dec 31 2008, 00:51:49)
import os os.environ.get('A'), os.environ.get('B') (None, 'c') retcode=os.system('echo -n $A|hexdump -C') 00000000 ff |.| 00000001 retcode=os.system('echo -n $B|hexdump -C') 00000000 63 |c| 00000001
Discussion to support bytes environment variables: http://mail.python.org/pipermail/python-dev/2008-December/083856.html -- Victor Stinner aka haypo http://www.haypocalc.com/blog/