[Python-3000] Pre-PEP: Easy Text File Decoding

Paul Prescod paul at prescod.net
Mon Sep 11 06:42:01 CEST 2006

On 9/10/06, Marcin 'Qrczak' Kowalczyk <qrczak at knm.org.pl> wrote:
> Other than that, guessing the encoding from the contents of the text
> stream, especially statistical guessing basing on well-formed UTF-8
> non-ASCII characters, shouldn't be encouraged, because it's effect is
> not predictable.

My thinking has evolved. The "guess" mode should "virtually" try
different decodings until one succeeds. In the worst case this might
involve decoding the whole file twice (once for detection and once for
application processing).

In general, your proposal is too far from the goals that were given to
me by Guido for me to really evaluate it as an alternative. Guido's
goal was that quick and dirty text processing should "just work" for
newbies and encoding-disintererested expert programmers. I don't think
that your proposal achieves that.

 Paul Prescod

More information about the Python-3000 mailing list