fun with unicode files

"Martin v. Löwis" martin at
Tue Aug 24 22:25:40 CEST 2004

Thomas Heller wrote:
> I wonder: do I really have to check for the BOM manually, or is there a
> Python function which does that?

If it can also be ASCII (or ansi?), then yes, you need to manually check
for the BOM. This is because you need to make an explicit decision in
the fallback case - Python cannot know whether it is ASCII if it is
not UTF-16. For example, it might also be Latin-1 or UTF-8 if it is not
UTF-16, or, say, iso-2022-jp.


More information about the Python-list mailing list