[issue12168] SysLogHandler incorrectly appents \000 to messages

[Carl Crowder]

New submission from Carl Crowder <carl.crowder at gmail.com>:

logging.handlers.SysLogHandler contains this class variable and comment:

    # curious: when talking to the unix-domain '/dev/log' socket, a
    # zero-terminator seems to be required.  this string is placed
    # into a class variable so that it can be overridden if
    # necessary.
    log_format_string = '<%d>%s\000'

And separately, in emit:

    msg = self.format(record) + '\000'

The assumption here is that a null character must be appended to delimit the syslog message. In RFC5424, there is no mention of a message delimiter, and in fact the previous syslog RFC, RFC3164, specifically states:

>  The MSG part will fill the remainder of the syslog packet.  This will usually contain some additional information of the process that generated the message, and then the text of the message.  There is no ending delimiter to this part.

I believe this comment and behaviour is due to an older version of syslogd. Checking the manpage for an older version of rsyslog for example includes this piece of information [1]:

> There is probably one important consideration when installing rsyslogd. It is dependent on proper formatting of messages by the syslog function. The functioning of the syslog function in the shared libraries changed somewhere in the region of libc.so.4.[2-4].n.   The specific change was to null-terminate the message before transmitting it to the /dev/log socket. Proper functioning of this  version of rsyslogd is dependent on null-termination of the message.

I'm running Ubuntu 11.04 with rsyslogd 4.6.4 (that is, the standard version). In the manpage for this version of rsyslogd, there is no reference to null-terminators. Removing "+ '\000'" from the SysLogHandler results in messages still being received correctly.

Problem behaviour:
1) When running any RFC compliant syslog receiver that handles syslog messages, such as flume[2], this null character is not stripped as it is not expected to be present. Current versions of syslog cope because previously they assumed it existed.
2) The log_format_string class variable is not actually used anywhere, so it cannot be overridden usefully.

Removing the null terminator will cause older typical versions of syslogd to fail to receive messages, however including it causes any normal receiver that does not implement the non-standard behaviour to receive an additional unwanted null.

Suggestion for a fix is either to properly use the log_format_string class variable, or to allow an optional "append_null" argument to the SysLogHandler constructor. By default, this should be True, as it will continue to work with the main use case, which is unix syslog demons. Having the option will allow other use cases to also use the SysLogHandler.

[1] http://manpages.ubuntu.com/manpages/hardy/man8/rsyslogd.8.html#contenttoc8
[2] http://www.cloudera.com/blog/category/flume/

----------
components: Library (Lib)
messages: 136743
nosy: Carl.Crowder
priority: normal
severity: normal
status: open
title: SysLogHandler incorrectly appents \000 to messages
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12168>
_______________________________________