[Python-Dev] Encoding detection in the standard library?

Michael Foord fuzzyman at voidspace.org.uk
Mon Apr 21 18:50:43 CEST 2008

skip at pobox.com wrote:
>     David> Is there some sort of text encoding detection module is the
>     David> standard library?  And, if not, is there any reason not to add
>     David> one?
> No, there's not.  I suspect the fact that you can't correctly determine the
> encoding of a chunk of text 100% of the time mitigates against it.

The only approach I know of is a heuristic based approach. e.g.


(Which was 'borrowed' from docutils in the first place.)

Michael Foord
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

More information about the Python-Dev mailing list