UnicodeEncodeError when piping stdout, but not when printing directly to the console

[Peter Otten]

Adam Funk wrote:

> (I'm using Python 2.7.2+ on Ubuntu.)
> 
> When I'm running my program in an xterm, the print command with an
> argument containing unicode works fine (it correctly detects my UTF-8
> environment).  But when I run it with a pipe or redirect to a file (|
> or >), unicode strings fail with the following (for example):
> 
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in
> position 21: ordinal not in range(128)
> 
> How can I force python (preferably within my python program, rather
> than having to set something externally) to treat stdout as UTF-8?


$ cat force_utf8.py
# -*- coding: utf-8 -*-
import sys

if sys.stdout.encoding is None:
    import codecs
    writer = codecs.getwriter("utf-8")
    sys.stdout = writer(sys.stdout)

print u"Ähnlich üblich nötig"

$ python force_utf8.py
Ähnlich üblich nötig

$ python force_utf8.py | cat
Ähnlich üblich nötig