Read text files with a specific encoding? using fileinput?

[Tim Evans]

Skip Montanaro <skip at pobox.com> writes:

>     Jim> I could write my own replacement for fileinput that did that, but
>     Jim> currently the fileinput module does all the opening, so absent an
>     Jim> open hook or a way to tell it what encoding to use, I'm not sure
>     Jim> that will help.
> 
> Sorry, my bad.  I forgot that fileinput is rather unpythonic in its
> treatment of files (accepting just names and not file objects, and even
> getting the names from an implicit source).
> 
> Skip

Telling it how to open files can be done:

-----------------------------------------------
import fileinput, codecs

def my_open(filename, mode):
    return codecs.open(filename, mode, 'utf8')

fileinput.open = my_open

for line in fileinput.input():
    print repr(line)
-----------------------------------------------

Making it read utf8 from stdin will be a little harder, but changing
'fileinput.sys' to a new object with hand-crafted 'stdin' and 'argv'
attributes should work.

-- 
Tim Evans