Logging library unicode problem

Victor Lin Bornstub at gmail.com
Wed Aug 13 12:08:09 CEST 2008

I'm writting a application using python standard logging system. I
encounter some problem with unicode message passed to logging library.
I found that unicode message will be messed up by logging handler.

piese of StreamHandler:

                    self.stream.write(fs % msg)
                except UnicodeError:
                    self.stream.write(fs % msg.encode("UTF-8"))

It just write the message to stream. If there is some unicode error,
it would rewrite msg with utf8 encoding.

I write some code to try:

    import sys
    print u'中文字測試'
    print sys.stdout.encoding

result of that program:

Traceback (most recent call last):
  File "update_stockprice.py", line 92, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-1: ordin
al not in range(128)

It show that....

1. print statement encode what it get with stream.encoding?
2. stream.write don't do anything like encoding, just write it
(because it might be binary data?)

So the problem is : the StreamHandler of standard logging library use
stream.write to log message, if there is unicode error, unicode string
will be encode to utf8. This behavior mess my unicode up.

Here I modify the code of StreamHandler:

                    print >> self.stream, msg
                    #self.stream.write(fs % msg)
                except UnicodeError:
                    self.stream.write(fs % msg.encode("UTF-8"))

I replace stream.write with print statement, so that it will try to
use stream.encoding to encode msg. Now everything works fine.

My question is :
Could the behavior of StreamHandler be considered as a bug?
If it is, how to report this bug?
Is my solution correct?
Are there any side effect will caused by doing so?
If the code I write is fine, and solve that problem, how to report it
to Python's project?
I think this could be helpful for people who also encountered this

Victor Lin.

More information about the Python-list mailing list