logging of strings with broken encoding

[David Smith]

Thomas Guettler wrote:
> Hi,
> 
> I have bug in my code, which results in the same error has this one:
> 
> https://bugs.launchpad.net/bzr/+bug/295653
> {{{
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/logging/__init__.py", line 765, in emit
>     self.stream.write(fs % msg.encode("UTF-8"))
>   ..
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 8: ordinal not in range(128)
> }}}
> 
> I run Python 2.6. In SVN the code is the same (StreamHandler ... def emit...):
> http://svn.python.org/view/python/branches/release26-maint/Lib/logging/__init__.py?revision=72507&view=markup
> 
> I think msg.encode("UTF-8", 'backslashreplace') would be better here.
> 
> What do you think?
> 
> Should I fill a bugreport?
> 
>   Thomas
> 
> 

I think you have to decode it first using the strings original encoding
whether that be cp1252 or mac-roman or any of the other 8-bit encodings.
 Once that's done, you can encode in UTF-8

--David