[Mailman-Developers] EOL handling in Mailman

[John W. Baxter]

On 3/19/2005 9:59, "Mark Sapiro" <msapiro at value.net> wrote:

> Sylvain Beucler wrote:
>> 
>> I'm a Savane (gna.org/p/savane) developer, and we noticed that when we
>> send mail with newlines in the \r\n format, Mailman converts them to
>> \n\n.
>> 
>> I would like to get your point on what we should do to fix the
>> problem. I had a look at RFC 822; the EOL (end of line) description is
>> not very clear, but it seems that the standard EOL convention is \r\n
>> (CRLF).
> 
> RFC 822 is obsolete. The current standards are RFC 2821 SMTP and RFC
> 2822 Internet Message Format. Both these standards are very clear. For
> example RFC 2821 says:
> 
> <quote>
> 2.3.7 Lines
> 
>     SMTP commands and, unless altered by a service extension, message
> data, are transmitted in "lines". Lines consist of zero or more data
> characters terminated by the sequence ASCII character "CR" (hex value
> 0D) followed immediately by ASCII character "LF" (hex value 0A). This
> termination sequence is denoted as <CRLF> in this document. Conforming
> implementations MUST NOT recognize or generate any other character or
> character sequence as a line terminator. Limits MAY be imposed on line
> lengths by servers (see section 4.5.3).
> 
>     In addition, the appearance of "bare" "CR" or "LF" characters in
> text (i.e., either without the other) has a long history of causing
> problems in mail implementations and applications that use the mail
> system as a tool. SMTP client implementations MUST NOT transmit these
> characters except when they are intended as line terminators and then
> MUST, as indicated above, transmit them only as a <CRLF> sequence.
> </quote>
> 
> RFC 2822 says
> 
> <quote>
> 2.3 Body
> 
>     The body of a message is simply lines of US-ASCII characters. The
> only two limitations on the body are as follows:
> 
>    - CR and LF MUST only occur together as CRLF; they MUST NOT appear
>      independently in the body.
> 
>    - Lines of characters in the body MUST be limited to 998 characters,
>      and SHOULD be limited to 78 characters, excluding the CRLF.
> 
>     Note: As was stated earlier, there are other standards documents,
> specifically the MIME documents [RFC2045, RFC2046, RFC2048, RFC2049]
> that extend this standard to allow for different sorts of message
> bodies. Again, these mechanisms are beyond the scope of this document.
> </quote>
> 
> 
>> The issue can be fixed if I make Savane convert all \r\n to \n before
>> to send the notifications, but I am not sure this is very
>> compliant. Anyway I do think this is a bug in Mailman. What do you
>> think we (Mailman and Savane) should do?
> 
> I don't see any issues with mail delivered from Mailman having any
> "doubling" of line terminators resulting in extra blank lines.
> 
> How does Savane inject its messages into the internet mail transport
> system? If it is sending via SMTP, then it MUST send line terminators
> as <CRLF> (or \r\n). If this results in doubling, then the receiving
> SMTP server is non compliant. If it is sending messages via some non
> SMTP intermediary, then it needs to format the messages as expected by
> the intermediary.
> 
> Does Savane perhaps pipe outgoing messages directly to the Mailman
> wrapper in the same way that an incoming MTA might do? If so, this is
> _not_ an SMTP transfer and end of line sequences should be those of
> the platform, i.e just \n for Unix.
> 
> --
> Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> 
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> http://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: 
> 
http://mail.python.org/mailman/options/mailman-developers/jwblist%40olympus.ne>
t

Note that 2821 is talking about how messages are transmitted over the wire
as part of the SMTP transaction.  An MTA will usually present the message to
mailboxes, scripts, etc in the native form expected on the platform (for
Unix, newline = ASCII linefeed).  Mailman is very likely operating on that
expectation (in particular, Mailman is not one end of an SMTP conversation,
so the SMTP RFC doesn't apply).

I believe things will work much better if Savane presents Unix-like data to
the Mailman scripts, not SMTP data.

The same is likely true of at least some other entities that Savane talks
to.

  --John