[New-bugs-announce] [issue7470] logger.StreamHandler emit encoding fallback is wrong

Christian Boos report at bugs.python.org
Thu Dec 10 14:14:35 CET 2009

New submission from Christian Boos <cboos at neuf.fr>:

For a stream with a "poor" encoding, such as sys.stderr on Windows where 
encoding is cp437, the encoding of an unicode message will fail 
(expected) and then a fallback encoding to UTF-8 will be done:

837                 except UnicodeError:
838                     stream.write(fs % msg.encode("UTF-8"))

However, that fallback won't work, as at this point, fs was already 
converted to unicode and is now u'%s\n', so the (msg.encode("UTF-8")) 
str will be decoded to unicode again using the default encoding, which 
will likely fail with a UnicodeDecodeError if msg contains non-ascii 

The solution would be to keep using fs as "%s\n" in this line.

This is similar to issue6991, but not exactly the same and it only 
happens for Python 2.7. Using logging_error.py, I've tested Python 2.3 
to Python 2.6 (works) and Python 2.7a1 (fails), current trunk must have 
the same issue as 2.7a1. Patch follows.

components: Library (Lib)
files: logging_error.py
messages: 96201
nosy: cboos, vinay.sajip
severity: normal
status: open
title: logger.StreamHandler emit encoding fallback is wrong
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file15518/logging_error.py

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list