Mailman 3 April 2007 - Mailman-coders

[ mailman-Bugs-1661574 ] arch corrupts archives, but only for the last month
by SourceForge.net April 2, 2007

April 2, 2007

Bugs item #1661574, was opened at 2007-02-16 06:54 Message generated for change (Comment added) made by msapiro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1661574&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: command line scripts Group: 2.1 (stable) >Status: Pending Resolution: None Priority: 9 Private: No Submitted By: David A. Desrosiers (desrod) Assigned to: Nobody/Anonymous (nobody) Summary: arch corrupts archives, but only for the last month Initial Comment: I think this problem has been reported before in previous versions, and its back again in 2.1.9. When I regenerate archives for our lists, if ANY message contains a '<' character in the body, Mailman splits it as a new message, and everything after that gets corrupted. This means if someone pastes some XML into the body of a message (which happens quite often on our lists) or some HTML, or the headers of an email, Mailman will break it, but *ONLY* for the latest month's messages, even if the message that started it, was months or years ago. If a message sent in April of 2003 includes an '<' as the first character anywhere in the body of the message, February 2007's archive will be corrupted. You can see the results of this over here: http://lists.plkr.org/pipermail/plucker-list/2007-February/thread.html And also here: http://lists.plkr.org/pipermail/plucker-dev/2007-February/thread.html The raw mbox files are fine, every message is intact. I don't see this problem on other lists I maintain, it only seems to affect lists where HTML or XML or mail headers are pasted into the body of the message. I'd call this grave, because its odd how it just dumps itself on the latest month's archive, when the latest month's messages don't even have the problem. ---------------------------------------------------------------------- >Comment By: Mark Sapiro (msapiro) Date: 2007-04-02 09:20 Message: Logged In: YES user_id=1123998 Originator: NO No response after 6 weeks. I'm setting status to Pending which will automatically close in 2 more weeks. ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-02-16 08:58 Message: Logged In: YES user_id=1123998 Originator: NO What am I looking for at <http://lists.plkr.org/pipermail/plucker-list/2007-February/thread.html>? It looks OK to me. <http://lists.plkr.org/pipermail/plucker-dev/2007-February/thread.html> returns a 404. There is an issue in that if the body of some message in the archives/private/<listname>.mbox/<listname>.mbox file (or whatever mbox is input to bin/arch) contains a line that begins with "From ", the archiver takes that line as an mbox message separator and the message is truncated at that point, and the rest of the message is seen as a new message without a date so it is archived with the current date. It sounds like that may be what you are seeing, but it has nothing to do with a '<' as the first character of a line. It has to do with 'unescaped' 'From ' lines in the bodies of messages. Mailman currently precedes any 'From ' at the beginning of a body line with a '>' making it '>From ' in the .mbox and avoiding the problem, but old .mbox files and .mbox files from other sources may have unescaped 'From ' lines. There is a bin/cleanarch script distributed with Mailman to help 'fix' old .mbox files with this problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1661574&group_…

1 0

[ mailman-Bugs-1658920 ] charset proble,
by SourceForge.net April 2, 2007

April 2, 2007

Bugs item #1658920, was opened at 2007-02-13 05:06 Message generated for change (Comment added) made by msapiro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Web/CGI Group: 2.1 (stable) Status: Open Resolution: None Priority: 5 Private: No Submitted By: dudeua (dudeua) Assigned to: Mark Sapiro (msapiro) Summary: charset proble, Initial Comment: Hello, I need help. I have mailman 2.1.9 .... if I receive mail in charset, different that koi8, than mailman in admins area shows this mail in "quoted-printable" charset ... But after approving this email, via web-interface email shows with correct charset. How to fix charset? ---------------------------------------------------------------------- >Comment By: Mark Sapiro (msapiro) Date: 2007-04-02 08:56 Message: Logged In: YES user_id=1123998 Originator: NO I don't know why the result is different in these two cases. In the case of quoted-printable encoding, all non-us-ascii-printable bytes are encoded as '=xx' where xx is the hex value of the corresponding byte. The 'decode=True' argument added by the patch causes these '=xx' codes to be converted back to the original bytes for display in the admindb interface. In the case of 8bit encoding, all bytes are represented as themselves and the 'decode=True' argument does nothing. The result should be the same. ---------------------------------------------------------------------- Comment By: dudeua (dudeua) Date: 2007-04-02 06:30 Message: Logged In: YES user_id=1718234 Originator: YES Hi msapiro, Thanks for your patch! all work fine. but today i meet ne problem. When mesasge comes with this headers: Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable All fine. But when message is in this encoding: Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 8bit it's body is ugly. Please advice me ... Thanks! ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-02-13 10:53 Message: Logged In: YES user_id=1123998 Originator: NO The screenshot is unnecessary. I understood exactly what the issue was from your original description. I know you only see the problem in the "message excerpt" in the admindb message detail. I tried to explain that the reason for this is that this excerpt is not decoded. It would be possible to patch Mailman/Cgi/admindb.py with the attached patch to decode this "message excerpt" before displaying it, but then you would have the issue that the characters in the box in your screenshot would be Windows CP1251 characters which would probably still be garbled when displayed in the character set of the rest of the page. Also, my suggestion to set admin_immed_notify to 1 was not intended to 'correct' this display. It causes this message to be also sent to you in an email so that if it is necessary for you to see the message text in order to decide what action to take with the message, you can see the text in the email notice. File Added: admindb.patch.txt ---------------------------------------------------------------------- Comment By: dudeua (dudeua) Date: 2007-02-13 08:55 Message: Logged In: YES user_id=1718234 Originator: YES set admin_immed_notify at /mailman/bun/config ? It already setted to 1; Pleasee see attacged screenshot. I can't use MUA (mail agent), because I meet problem with encoding only at admin area. Thanks for help. File Added: ggg.jpg ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-02-13 08:36 Message: Logged In: YES user_id=1123998 Originator: NO Quoted-printable is not a character set. It is an encoding. That is, it is a way of representing data which contains non-printable, us-ascii characters using only the us-ascii character set as required by RFC2822 for email messages. The issue here is that encoded message bodies are not decoded for display in the admindb message detail. This may change in the future, but then there _will_ be character set and content-type issues. These are more complicated as in general, different message parts may have different character sets and may not even be text. Currently, if you really need to see the decoded text, you need to set admin_immed_notify to Yes so you get an email notice of the held message and use an MUA (mail client) that shows you the decoded message in the notice. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…

1 0

[ mailman-Bugs-1658920 ] charset proble,
by SourceForge.net April 2, 2007

April 2, 2007

Bugs item #1658920, was opened at 2007-02-13 13:06 Message generated for change (Comment added) made by dudeua You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Web/CGI Group: 2.1 (stable) Status: Open Resolution: None Priority: 5 Private: No Submitted By: dudeua (dudeua) Assigned to: Mark Sapiro (msapiro) Summary: charset proble, Initial Comment: Hello, I need help. I have mailman 2.1.9 .... if I receive mail in charset, different that koi8, than mailman in admins area shows this mail in "quoted-printable" charset ... But after approving this email, via web-interface email shows with correct charset. How to fix charset? ---------------------------------------------------------------------- >Comment By: dudeua (dudeua) Date: 2007-04-02 13:30 Message: Logged In: YES user_id=1718234 Originator: YES Hi msapiro, Thanks for your patch! all work fine. but today i meet ne problem. When mesasge comes with this headers: Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable All fine. But when message is in this encoding: Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 8bit it's body is ugly. Please advice me ... Thanks! ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-02-13 18:53 Message: Logged In: YES user_id=1123998 Originator: NO The screenshot is unnecessary. I understood exactly what the issue was from your original description. I know you only see the problem in the "message excerpt" in the admindb message detail. I tried to explain that the reason for this is that this excerpt is not decoded. It would be possible to patch Mailman/Cgi/admindb.py with the attached patch to decode this "message excerpt" before displaying it, but then you would have the issue that the characters in the box in your screenshot would be Windows CP1251 characters which would probably still be garbled when displayed in the character set of the rest of the page. Also, my suggestion to set admin_immed_notify to 1 was not intended to 'correct' this display. It causes this message to be also sent to you in an email so that if it is necessary for you to see the message text in order to decide what action to take with the message, you can see the text in the email notice. File Added: admindb.patch.txt ---------------------------------------------------------------------- Comment By: dudeua (dudeua) Date: 2007-02-13 16:55 Message: Logged In: YES user_id=1718234 Originator: YES set admin_immed_notify at /mailman/bun/config ? It already setted to 1; Pleasee see attacged screenshot. I can't use MUA (mail agent), because I meet problem with encoding only at admin area. Thanks for help. File Added: ggg.jpg ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2007-02-13 16:36 Message: Logged In: YES user_id=1123998 Originator: NO Quoted-printable is not a character set. It is an encoding. That is, it is a way of representing data which contains non-printable, us-ascii characters using only the us-ascii character set as required by RFC2822 for email messages. The issue here is that encoded message bodies are not decoded for display in the admindb message detail. This may change in the future, but then there _will_ be character set and content-type issues. These are more complicated as in general, different message parts may have different character sets and may not even be text. Currently, if you really need to see the decoded text, you need to set admin_immed_notify to Yes so you get an email notice of the held message and use an MUA (mail client) that shows you the decoded message in the notice. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…

1 0