[Python-Dev] Python 3.0.1 (io-in-c)
solipsis at pitrou.net
Wed Jan 28 01:15:27 CET 2009
Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> What kind of input are you using for the Text tests? I'm kind of surprised
that the conversion to Unicode results in such a dramatic slowdown, if you're
feeding it plain text (characters 0x00 through 0x7f).
It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On
this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s
(on the same machine on which I ran the benchmarks).
You can find the test here:
More information about the Python-Dev