I've got a set of patches which I want to make to logging: 1. Added the record being processed as a parameter to handleError. 2. Handler.handle returns the result of applying the filter to the record. 3. Added a seek(0, 2) in RotatingFileHandler before the tell() call. This is because under Windows, tell() returns 0 until the first actual write. 4. Renamed warn and WARN to warning and WARNING. This may break existing code, but the standard Python module will use warning/WARNING rather than warn/WARN. The fatal and FATAL synonyms for critical and CRITICAL have also been removed. 5. Added a defaultEncoding attribute and some simple support for encoding Unicode messages. 6. Added process ID to the list of LogRecord attributes. 7. Modified Logger.removeHandler so that it does not close the handler on removal. As far as point 4 is concerned, I feel that the last discussion on python-dev was inconclusive. The consensus seemed to say WARNING and CRITICAL were OK, but then there was some doubt about WARNING due to the number of changes Zope would need. Though Zope is an important application, I'm not sure Zope changes should be the yardstick for this sort of change;
Vinay Sajip wrote:
For point 5, I'm not sure how best to handle Unicode support, so I'm canvassing suggestions. The API allows overriding of the message formatting logic at various stages - LogRecord, Handler or Formatter. This makes it fairly easy to provide arbitrary custom logic for Unicode encoding, etc. The change I've made allows for simple default processing - a "defaultEncoding" attribute defaulted to "latin_1" has been added to logging.
You should use sys.getdefaultencoding() for this. Python's default encoding is "ascii", BTW, not "latin-1".
Currently, the message format string is determined via LogRecord.getMessage(), which does a str() on the passed in message argument passed to the logging call. Under the proposed change:
- If the running Python has no Unicode support, the system does a str(), as it does currently. - If the message argument is not a Unicode string, the system does a str(), as it does currently. - If the message argument is a Unicode string (type(msg) == types.UnicodeType), the system does a msg.encode(defaultEncoding).
Perhaps there's some other reason, but why do you apply special treatment to Unicode ? Unicode objects behave pretty much the same as strings do, so the special casing doesn't seem necessary. I'd only do the str() call on objects which are not string types.
The resulting string is treated as the format string and the % operator applied with the arguments to obtain the final message string.
-- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 15 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 45 days left EuroPython 2003, Charleroi, Belgium: 129 days left
You should use sys.getdefaultencoding() for this. Python's default encoding is "ascii", BTW, not "latin-1".
OK, but supposing I log a message with the format string u"Marc-Andr\xe9". If I use "ascii" as the encoding, an exception is raised because the code for e-acute is > 128. How best should this situation be handled? I assumed that people would be using Unicode to log messages in other languages with accented characters etc. which are typically outside the ASCII range. Regards, Vinay Sajip
Vinay Sajip wrote:
You should use sys.getdefaultencoding() for this. Python's default encoding is "ascii", BTW, not "latin-1".
OK, but supposing I log a message with the format string u"Marc-Andr\xe9". If I use "ascii" as the encoding, an exception is raised because the code for e-acute is > 128. How best should this situation be handled? I assumed that people would be using Unicode to log messages in other languages with accented characters etc. which are typically outside the ASCII range.
I don't know how you have implemented logging to files or stream, but in general it's better to let the stream get the Unicode and let it decide what to do, e.g. you could have a log file open in 'latin-1' mode (via codecs.open()) and then see log entries as Latin-1. To make things a little more robust (ie. to prevent the log message from getting lost), I'd suggest to use try: except UnicodeError: around the log writing code. The except clause should then encode the log message to UTF-8 and write the 8-bit string in a second try. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 16 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 44 days left EuroPython 2003, Charleroi, Belgium: 128 days left
I've got a set of patches which I want to make to logging:
Please upload to SF ASAP -- I'd like to do another alpha next Tuesday. Please send forward context or unified diffs relative to current CVS!
1. Added the record being processed as a parameter to handleError.
OK.
2. Handler.handle returns the result of applying the filter to the record.
OK.
3. Added a seek(0, 2) in RotatingFileHandler before the tell() call. This is because under Windows, tell() returns 0 until the first actual write.
OK.
4. Renamed warn and WARN to warning and WARNING. This may break existing code, but the standard Python module will use warning/WARNING rather than warn/WARN.
OK, but please leave warn/WARN in as synonyms.
The fatal and FATAL synonyms for critical and CRITICAL have also been removed.
NOT OK. Please leave them in as synonyms.
5. Added a defaultEncoding attribute and some simple support for encoding Unicode messages.
See Marc-Andre's reply.
6. Added process ID to the list of LogRecord attributes.
OK.
7. Modified Logger.removeHandler so that it does not close the handler on removal.
OK.
As far as point 4 is concerned, I feel that the last discussion on python-dev was inconclusive. The consensus seemed to say WARNING and CRITICAL were OK, but then there was some doubt about WARNING due to the number of changes Zope would need. Though Zope is an important application, I'm not sure Zope changes should be the yardstick for this sort of change; From looking for instances of WARN and warn in the .py files from a recent read-only checkout of Zope, I couldn't see all that many places where changes would be needed. Have I missed something?
Did you check Zope 3 too? It's got more of these I think. But if you submit patches for Python, I can take care of Zope 3. --Guido van Rossum (home page: http://www.python.org/~guido/)
I've got a set of patches which I want to make to logging:
Please upload to SF ASAP -- I'd like to do another alpha next Tuesday.
Please send forward context or unified diffs relative to current CVS!
Uploaded, see http://sf.net/tracker/index.php?func=detail&aid=687683&group_id=5470&atid=30 5470 I assigned it to Neal, hope that's OK. Regards, Vinay Sajip
participants (3)
-
Guido van Rossum -
M.-A. Lemburg -
Vinay Sajip