[Python-3000] Pre-PEP: Easy Text File Decoding

[Paul Prescod]

On 9/10/06, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> ...
> > Modern I/O is astonishingly fast anyhow. On my computer it takes five
> > seconds to decode a quarter gigabyte of UTF-8 text through Python.
>
> Maybe we shouldn't be that presomptuous. Modern I/O is fast but memory
> is not infinite. That quarter gigabyte will have swapped out other
> data/code in order to make some place in the filesystem cache.

Not really. It works in 16k chunks.

> Also, Python is often used on more modest hardware.

People writing programs to deal with vast amounts of data on modest
computers are trying to do something advanced and should not use the
quick and dirty guessing algorithms. We're not trying to hit 100% of
programmers and situations. Not even close. The PEP was very explicit
about that fact.

 Paul Presocd