[Python-Dev] Improve open() to support reading file starting with an unicode BOM

[Antoine Pitrou]

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.