[Python-Dev] Python 3.0.1 (io-in-c)

Antoine Pitrou solipsis at pitrou.net
Wed Jan 28 01:15:27 CET 2009

Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> What kind of input are you using for the Text tests?  I'm kind of surprised
that the conversion to Unicode results in such a dramatic slowdown, if you're
feeding it plain text (characters 0x00 through 0x7f).

It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On
this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s
(on the same machine on which I ran the benchmarks).

You can find the test here:

More information about the Python-Dev mailing list