[Mailman-Users] Problem with inappropriate breaks in messages

[Mark Sapiro]

Chris Malme wrote:
>
>That is correct, I did it manually (or rather, with a quick script I wrote),
>preceding each message text line that begins with a "From " with a ">". This
>enabled the mbox to be imported into the Mailman Archive without splitting
>messages as it did when I first tried it. However, those "From " lines that
>I manually escaped are showing clearly in the Archive *with* the escape
>character - i.e. as ">From".


That is the normal way of dealing with messages containing From_ in the
message body. It's not just Mailman or pipermail, and it's problematic
to unescape them for display, because while the escaping is normal,
there is no standard for escaping/unescaping so when you see >From_ in
a message, you don't know if it is an escaped From_, a quoted From_ or
a literal >From_.


>Furthermore any new emails to the list that
>have a newline/From in the message text are archiving incorrectly.
>
>> So the question is why is this happening with new messages? Again, what
>> Mailman version is this?
>
>Sorry, I should have stated earlier. This is Mailman version 2.1.9, which is
>the version that the folk who support my VPS (using Plesk) have rolled out.


The issue with unescaped From_ in the body causing archive corruption
was fixed long before Mailman 2.1.9


>To go to a more recent version is not impossible, but not trivial for me (a
>Linux VPS newbie), so I wanted to see if it was the solution before rolling
>my sleeves up.


You shouldn't need to. Mailman 2.1.9 should not have this problem.


>> Also note that escaping From_ by preceding it with '>' is the accepted
>> way to deal with this. Many MUAs will do it before sending the message
>> and MDAs will do it too before delivering a message. It is unusual to
>> be able to pass a From_ through email from end to end without it being
>> escaped to >From_ somewhere between source and destination.
>
>That is what is puzzling me. I am able to send an email from Thunderbird
>through my VPS's mail server and see it go straight into the archive
>unescaped, splitting into multiple messages at every incidence of
>newline/From. I can also do it from Gmail.


It doesn't happen with every MUA/MTA/MDA, but for example, I just sent
such a message from Tbird 3.0.3 via Exim on localhost to a mailbox on
localhost, and the From_ was unescaped in Tbird's Sent folder, but it
was escaped in the recipient mailbox.


>I have a Test list set up that I am happy for anyone to access, if it might
>shed any light on the matter. The Test list does not have the imported
>archive, but it demonstrates the same behaviour regarding new messages.
>
>I'm happy to post the list URL if that is appropriate.


I believe you, so I don't think I need to see the test list. The
question is why isn't Mailman escaping the From_ when it archives and
sends the message.

It actually relies on the Python email library to do this, but Mailman
2.1.9 should install its own version of the email package in Mailman's
pythonlib/ directory, and this should always escape From_ lines when
converting an email.Message.Message object to text. Why it doesn't is
the question.

Also curious is that I think you said the problem occurs with
"text\nFrom " in the body, but not with "text\n\nFrom ". If I
understood that correctly, that is really strange.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan