[New-bugs-announce] [issue7470] logger.StreamHandler emit encoding fallback is wrong

[Christian Boos]

New submission from Christian Boos <cboos at neuf.fr>:

For a stream with a "poor" encoding, such as sys.stderr on Windows where 
encoding is cp437, the encoding of an unicode message will fail 
(expected) and then a fallback encoding to UTF-8 will be done:

(in 
http://code.python.org/hg/trunk/file/bd98b2c097fe/Lib/logging/__init__.p
y#l837):
837                 except UnicodeError:
838                     stream.write(fs % msg.encode("UTF-8"))

However, that fallback won't work, as at this point, fs was already 
converted to unicode and is now u'%s\n', so the (msg.encode("UTF-8")) 
str will be decoded to unicode again using the default encoding, which 
will likely fail with a UnicodeDecodeError if msg contains non-ascii 
characters.

The solution would be to keep using fs as "%s\n" in this line.

This is similar to issue6991, but not exactly the same and it only 
happens for Python 2.7. Using logging_error.py, I've tested Python 2.3 
to Python 2.6 (works) and Python 2.7a1 (fails), current trunk must have 
the same issue as 2.7a1. Patch follows.

----------
components: Library (Lib)
files: logging_error.py
messages: 96201
nosy: cboos, vinay.sajip
severity: normal
status: open
title: logger.StreamHandler emit encoding fallback is wrong
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file15518/logging_error.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7470>
_______________________________________