[Python-ideas] Py3 unicode impositions

Nick Coghlan ncoghlan at gmail.com
Sun Feb 12 05:19:12 CET 2012

On Sun, Feb 12, 2012 at 1:19 PM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
> On Feb 11, 2012, at 5:10 PM, Eric Snow wrote:
>> So something like this:
>>    import functools, builtins
>>    open = builtins.open = functools.partial(open, encoding="ascii",
>> errors="surrogateescape")
> We could pack it in and call it something like "python2open". :-)

An open_ascii() builtin isn't as crazy as it may initially sound -
it's not at all uncommon to have a file that's almost certainly in
some ASCII compatible encoding like utf-8, latin-1 or one of the other
extended ASCII encodings, but you don't know which one specifically.

By offering open_ascii(), we'd be making it trivial to process such
files without blowing up (or having to figure out exactly *which*
ASCII compatible encoding you have). When you wrote them back to disk,
if you'd added any non-ASCII chars of your own, you'd get a
UnicodeEncodeError, but any encoded data from the original would be
reproduced in the original encoding.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list