[Python-Dev] Python 3.0.1 (io-in-c)

[Antoine Pitrou]

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> It would be helpful to limit this cost as much as possible - maybe
> that's simply ensuring that the default encoding for open is (in the
> majority of cases) a highly-optimised one whose costs *don't* dominate
> in the way you describe

As I pointed out, utf-8, utf-16 and latin1 decoders have already been optimized
in py3k. For *pure ASCII* input, utf-8 decoding is blazingly fast (1GB/s here).
The dataset for iobench isn't pure ASCII though, and that's why it's not as fast.

People are invited to test their own workloads with the io-c branch and report
performance figures (and possible bugs). There are so many possibilities that
the benchmark figures given by a generic tool can only be indicative.

Regards

Antoine.