[Python-Dev] Improve open() to support reading file starting with an unicode BOM

[Antoine Pitrou]

Martin v. Löwis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.