should writing Unicode files be so slow

Antoine Pitrou solipsis at
Sat Mar 20 01:19:07 CET 2010

Le Fri, 19 Mar 2010 17:18:17 +0000, djc a écrit :
> changing
> with open(filename, 'rU') as tabfile: to
> with, 'rU', 'utf-8', 'backslashreplace') as
> tabfile:
> and
> with open(outfile, 'wt') as out_part: to
> with, 'w', 'utf-8') as out_part:
> causes a program that runs  in
> 43 seconds to take 4 minutes to process the same data. (and the object it returns) is slow as it is written in 
pure Python.

Accelerated reading and writing of unicode files is available in Python 
2.7 and 3.1, using the new `io` module.



More information about the Python-list mailing list