csv.DictReader and unicode

[Peter Otten]

Laszlo Nagy wrote:

> This program
> 
> fin = codecs.open(fname,"r",encoding="UTF-8")
> eader = csv.DictReader(fin)
> for values in reader:
>     pass
> 
> results in:
> 
>   File "run.py", line 23, in process_file
>     for values in reader:
>   File "/usr/local/lib/python2.5/csv.py", line 83, in next
>     row = self.reader.next()
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in
> position 13: ordinal not in range(128)
> 
> As you can see the exception is thrown in csv.py. How it is possible?
> The csv.DictReader should not use ascii codec for anything, because the
> file encoding is UTF-8.

The csv module doesn't support unicode. Read the values as byte strings and
decode afterwards.

Peter