unicode to ascii converting

Peter Wilkinson pwilkinson at videotron.ca
Fri Aug 6 19:23:41 CEST 2004

Hello tlistmembers,

I am using the encoding function to convert unicode to ascii. At one point 
this code was working just fine, however, now it has broken.

I am reading a text file that has is in unicode (I am unsure of which 
flavour or bit depth). as I read in the file one line at a time 
(readlines()) it converts to ascii. Simple enough. At the same time I am 
copressing to bz2 with the bz2 module but that works just fine.  The code 
is and error reported appears below. I am unsure what to do.

I assume that because it is reporting that ordinal is not in range, that 
something to do with the character width that I am reading?

Peter W.

def encode_file(file_path, encode_type, compress='N'):
     Changes encoding of file
     new_encode = encode_type
     old_file_path = file_path + '.old'
     new_file_path = file_path
     file_in  = file(old_file_path,'r')

     if compress == 'Y' or compress == 'y':
         bz_file_path = file_path + '.bz2'
         bz_file_out  = bz2.BZ2File(bz_file_path, 'w')
         for line in file_in.readlines():

         file_out = file(file_path,'w')
         for line in file_in.readlines():


ERROR Reported:

Traceback (most recent call last):
   File "C:\Program Files\ActiveState Komodo 2.5\callkomodo\kdb.py", line 
433, in _do_start
     self.kdb.run(code_ob, locals, locals)
   File "C:\Python23\lib\bdb.py", line 350, in run
     exec cmd in globals, locals
   File "C:\Python23\Lib\site-packages\xBio\Scripts\unicodeToAscii.py", 
line 158, in ?
   File "C:\Python23\Lib\site-packages\xBio\Scripts\unicodeToAscii.py", 
line 75, in main
     encode_file(fileToProcess, options.encode,  'Y')
   File "C:\Python23\Lib\site-packages\xBio\Scripts\unicodeToAscii.py", 
line 144, in encode_file
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: 
ordinal not in range(128)

More information about the Python-list mailing list