Bugs item #815297, was opened at 2003-10-01 03:42
Message generated for change (Comment added) made by chrissamuel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=815297&group_i…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: security/privacy
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 8
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: Breaking signatures in message/rfc822 attachement!
Initial Comment:
Mailman _must_ not touch MIME-parts which are nested
more deeply in the mail. As tested with Mailman 2.1.2,
header lines will be sometimes reformatted in
message/rfc822 attachments which will break the OpenPGP
signature
(also conforming to the PGP/MIME standard) on that part.
I'm attaching a simple email with on long header.
Forward this as MIME part and sign it sending it
through Mailman,
the signature will be broken.
This is an email security affecting bug, because if people
start believing that a *BAD* signature does not mean much,
because they get many broken by mailman, they will not
react
to a seriously manipulated email anymore!
----------------------------------------------------------------------
Comment By: Chris Samuel (chrissamuel)
Date: 2007-04-10 10:34
Message:
Logged In: YES
user_id=1581966
Originator: NO
It may be that bug number 1605144 is also related as that too is caused by
headers being wrapped by the Python email module automatically wrapping
headers at 78 characters.
This is certainly breaking my PGP/MIME messages being sent from KMail
through Mailman, and shows up in the MIME headers being wrapped and the
original headers in attached message/rfc822 attachments getting munged.
For example:
@@ -56,7 +103,9 @@
Content-Type: message/rfc822;
name="forwarded message"
Content-Transfer-Encoding: 7bit
-Content-Description: "Rachana Ananthakrishnan" <ranantha(a)mcs.anl.gov>:
[security-announce] Globus Security Advisory 2007-02: GSI-OpenSSH
vulnerability
+Content-Description: "Rachana Ananthakrishnan" <ranantha(a)mcs.anl.gov>:
+ [security-announce] Globus Security Advisory 2007-02:
+ GSI-OpenSSH vulnerability
Content-Disposition: inline
and, further on:
Received: from localhost (localhost [127.0.0.1])
by mailbouncer.mcs.anl.gov (Postfix) with ESMTP id 73FB112AC5
- for <glbs-security-announce(a)mailbouncer.mcs.anl.gov>; Mon, 9 Apr
2007 10:23:46 -0500 (CDT)
+ for <glbs-security-announce(a)mailbouncer.mcs.anl.gov>;
+ Mon, 9 Apr 2007 10:23:46 -0500 (CDT)
Received: from mcs.anl.gov (cliff.mcs.anl.gov [140.221.9.17])
As I wrote in my comment on bug 1605144 it appears that by passing through
maxheaderlen set to 0 to all calls of Generator in the email module then
you shouldn't get this wrapping behaviour, though I don't know when this
appeared in Python.
----------------------------------------------------------------------
Comment By: Bernhard Reiter (ber)
Date: 2005-11-27 01:58
Message:
Logged In: YES
user_id=113859
This is still a serious bug.
I guess that the real fix will be to rewrite the email
and mime handling classes to at least additionally save
an original version of the different email parts
without stripping and further formatting.
----------------------------------------------------------------------
Comment By: Bernhard Reiter (ber)
Date: 2004-05-11 04:15
Message:
Logged In: YES
user_id=113859
There is another possibility when
mailman breaks the signature and that is
when the signed part contains
an empty header with _two_ spaces after
the colon, like forward and sign an email with
X-Empty-Header-with-two-spaces:<space><space>
patch 933757 does not remedy this, though.
----------------------------------------------------------------------
Comment By: Bernhard Reiter (ber)
Date: 2004-04-13 03:17
Message:
Logged In: YES
user_id=113859
I have created a patch to address the problem.
[ 933757 ] fix for [815297] signatures break
https://sourceforge.net/tracker/index.php?func=detail&aid=933757&group_id=1…
----------------------------------------------------------------------
Comment By: Marc Mutz (mmutz)
Date: 2003-10-04 01:54
Message:
Logged In: YES
user_id=82377
This is not limited to message/rfc822 at all:
As a specific example, create a message with an attachment
and add the header
Content-Disposition: attachment; filename="more-than-70-chars.
txt"
(all in a single line), then send it through a mailman-managed ml.
Result: mailman "fixes" the message to look like
Content-Disposition: attachment;
\tfilename="more-than-70-chars.txt"
It even does that inside a multipart/signed part, and this is
where it breaks the signature verification.
----------------------------------------------------------------------
Comment By: Bernhard Reiter (ber)
Date: 2003-10-01 03:46
Message:
Logged In: YES
user_id=113859
Here is the email signed by myself and broken
after delivery through mailman. Check the "To:" header line.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=815297&group_i…
Bugs item #1605144, was opened at 2006-11-29 21:28
Message generated for change (Comment added) made by chrissamuel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1605144&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: mail delivery
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: David Woodhouse (dwmw2)
Assigned to: Nobody/Anonymous (nobody)
Summary: mailman corrupts RFC2047-encoded headers
Initial Comment:
Given an input like this:
Subject: =?UTF-8?Q?[MTD]=20NAND:=20CAF=C3=89=20NAND=20driver=20cleanup,=20fix=20ECC=20on=20reading=20empty=20flash?=
Mailman appears to emit mail like this:
Subject: =?UTF-8?Q?[MTD]=20NAND:=20CAF=C3=89=20NAND=20driver=20cleanup,
=20fix=20ECC=20on=20reading=20empty=20flash?=
The input was RFC2047-compliant. The output isn't.
----------------------------------------------------------------------
Comment By: Chris Samuel (chrissamuel)
Date: 2007-04-10 10:27
Message:
Logged In: YES
user_id=1581966
Originator: NO
You can disable header wrapping in the module (if I am looking at the
correct Python docs) according to this page:
http://docs.python.org/lib/module-email.generator.html
It implies that by passing through maxheaderlen set to 0 to all calls of
Generator then you shouldn't get this wrapping behaviour, though I don't
know when this appeared in Python.
I believe this may also be the the cause of Mailman breaking my PGP/MIME
messages as diff'ing the saved original and the version that comes back
shows that the only differences are for long MIME headers and for
reformatting of the headers in the message/rfc822 attached email.
I am not sure if this is related to 815297, but it sure looks like it.
Caveat: I am not a Python programmer, just a Postmaster..
----------------------------------------------------------------------
Comment By: Tokio Kikuchi (tkikuchi)
Date: 2006-11-30 23:46
Message:
Logged In: YES
user_id=67709
Originator: NO
This is derived from the python email module behavior that try to keep a
header line within 78 characters. Mailman parses the message first and do
something like adding subject prefix or message body header/footer then
regenerate RFC-2822 message. Email module thinks your subject has two part
structure separated by a comma and split by CRLF. I am not very sure but
current version of email doesn't distinguish structured and unstructured
headers defined in 2.2.1 and 2.2.2 of RFC-2822. Anyway, It is safer to
shorten the header lines within 78 charcters.
FYI, email module generates your subject header like this:
Subject:
=?utf-8?q?=5BMTD=5D_NAND=3A_CAF=C3=89_NAND_driver_cleanup=2C_fix?=
=?utf-8?q?_ECC_on_reading_empty_flash?=
----------------------------------------------------------------------
Comment By: David Woodhouse (dwmw2)
Date: 2006-11-30 01:00
Message:
Logged In: YES
user_id=81423
Originator: YES
Your thunderbird also refuses to send this:
To: Some people : ;
Bcc: foo(a)bar.org, baz(a)turnip.com
Thunderbird isn't necessarily the best test of what's valid :)
The pertinent question is: why is mailman munging this _anyway_? Why can't
it just pass the header through as it was originally sent? If I put line
breaks in and lined things up sensibly like a SpamAssassin report does, why
should that be mangled by mailman?
----------------------------------------------------------------------
Comment By: Harald Hoyer (Red Hat) (saturn_de)
Date: 2006-11-30 00:46
Message:
Logged In: YES
user_id=5540
Originator: NO
Hmm, my thunderbird encodes "," with =2C
----------------------------------------------------------------------
Comment By: David Woodhouse (dwmw2)
Date: 2006-11-30 00:15
Message:
Logged In: YES
user_id=81423
Originator: YES
That's only for the charset (UTF-8) and the encoding (Q). The comma
appears in the encoded-text, and should be fine (since this is a Subject:
header and hence comes under paragraph (1) of §5.
encoded-word = "=?" charset "?" encoding "?" encoded-text "?="
charset = token ; see section 3
encoding = token ; see section 4
token = 1*<Any CHAR except SPACE, CTLs, and especials>
especials = "(" / ")" / "<" / ">" / "@" / "," / ";" / ":" / "
<"> / "/" / "[" / "]" / "?" / "." / "="
encoded-text = 1*<Any printable ASCII character other than "?"
or SPACE>
; (but see "Use of encoded-words in message
; headers", section 5)
----------------------------------------------------------------------
Comment By: Harald Hoyer (Red Hat) (saturn_de)
Date: 2006-11-29 23:21
Message:
Logged In: YES
user_id=5540
Originator: NO
Hmm, I don't thin "," is allowed unencoded...
token = 1*<Any CHAR except SPACE, CTLs, and especials>
especials = "(" / ")" / "<" / ">" / "@" / "," / ";" / ":" / "
<"> / "/" / "[" / "]" / "?" / "." / "="
----------------------------------------------------------------------
Comment By: David Woodhouse (dwmw2)
Date: 2006-11-29 22:53
Message:
Logged In: YES
user_id=81423
Originator: YES
Hm, good point; thanks. I've fixed the script which generates mail for
each commit to the Linux kernel git tree, and it should no longer generate
encoded-words longer than 75 characters.
I still see this input...
Subject:
=?UTF-8?Q?[MTD]_NAND:_CAF=C3=89_NAND_driver_cleanup,_fix_ECC_on_reading?=
=?UTF-8?Q?_empty_flash?=
and this output...
Subject: =?UTF-8?Q?[MTD]_NAND:_CAF=C3=89_NAND_driver_cleanup,
_fix_ECC_on_reading?= =?UTF-8?Q?_empty_flash?=
The comma is allowed, and doesn't have to be '=2C', does it? See §4.2 (3)
and §5 (1).
----------------------------------------------------------------------
Comment By: Harald Hoyer (Red Hat) (saturn_de)
Date: 2006-11-29 21:44
Message:
Logged In: YES
user_id=5540
Originator: NO
http://www.ietf.org/rfc/rfc2047.txt
An 'encoded-word' may not be more than 75 characters long, including
'charset', 'encoding', 'encoded-text', and delimiters. If it is
desirable to encode more text than will fit in an 'encoded-word' of
75 characters, multiple 'encoded-word's (separated by CRLF SPACE) may
be used.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1605144&group_…
Bugs item #1696053, was opened at 2007-04-07 17:24
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1696053&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Dubi (dovzamir)
Assigned to: Nobody/Anonymous (nobody)
Summary: Hebrew Translation Attached!
Initial Comment:
Attached are the Hebrew files for Mailman 2.1.9
I added a parameter to the LC_DESCRIPTION function, and these are the files that are affected by the addition of 'direction' to the 'LC_DESCRIPTION'
function. They have been changed, and diffs are attached...
./Mailman/Cgi/create.py
./Mailman/Defaults.py.in
./Mailman/Defaults.py
./Mailman/Utils.py
./messages/es/README.es
./messages/ja/README.ja
./messages/ja/doc/Defaults.py.in
./messages/eu/README.eu
Likewise, all translation files are attached.
There is still a problem displaying command line output in the correct direction. Still looking for a solution...
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1696053&group_…
Feature Requests item #1695897, was opened at 2007-04-06 21:30
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1695897&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jim Popovitch (jimpop)
Assigned to: Nobody/Anonymous (nobody)
Summary: patch to rename log files in order to work with logrotate
Initial Comment:
Mailman needs to have different log file names in order to work properly with logrotate. A proper Logrotate conf file for Mailman needs a postrotate command in order to restart Mailman after log files have been rotated. See FAQ 04.007 for more details (http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1695897&group_…
Feature Requests item #491675, was opened at 2001-12-11 13:43
Message generated for change (Comment added) made by jimpop
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Deleted
Resolution: None
Priority: 5
Private: No
Submitted By: Luigi Rosa (lrosa)
Assigned to: Nobody/Anonymous (nobody)
Summary: logrotate config file
Initial Comment:
During the install procedure you could create the
logrotate script in this way:
IF exists /etc/logrotate.d
IF NOT exists /etc/logrotate.d/mailman
Create /etc/logrotate.d/mailman
FI
FI
The /etc/logrotate.d/mailman file could be something
like this:
# Logrotate file for Mailman 2.0.x
$HOME/log/digest {
monthly
}
$HOME/log/error {
monthly
}
$HOME/log/post {
weekly
}
$HOME/log/smtp {
weekly
}
$HOME/log/subscribe {
monthly
}
$HOME/log/vette {
monthly
}
Of course, instead of $HOME you should place the
installation path of Mailman
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:29
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:28
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:27
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Feature Requests item #491675, was opened at 2001-12-11 13:43
Message generated for change (Comment added) made by jimpop
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Deleted
Resolution: None
Priority: 5
Private: No
Submitted By: Luigi Rosa (lrosa)
Assigned to: Nobody/Anonymous (nobody)
Summary: logrotate config file
Initial Comment:
During the install procedure you could create the
logrotate script in this way:
IF exists /etc/logrotate.d
IF NOT exists /etc/logrotate.d/mailman
Create /etc/logrotate.d/mailman
FI
FI
The /etc/logrotate.d/mailman file could be something
like this:
# Logrotate file for Mailman 2.0.x
$HOME/log/digest {
monthly
}
$HOME/log/error {
monthly
}
$HOME/log/post {
weekly
}
$HOME/log/smtp {
weekly
}
$HOME/log/subscribe {
monthly
}
$HOME/log/vette {
monthly
}
Of course, instead of $HOME you should place the
installation path of Mailman
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:28
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:27
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Feature Requests item #491675, was opened at 2001-12-11 13:43
Message generated for change (Comment added) made by jimpop
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Deleted
Resolution: None
Priority: 5
Private: No
Submitted By: Luigi Rosa (lrosa)
Assigned to: Nobody/Anonymous (nobody)
Summary: logrotate config file
Initial Comment:
During the install procedure you could create the
logrotate script in this way:
IF exists /etc/logrotate.d
IF NOT exists /etc/logrotate.d/mailman
Create /etc/logrotate.d/mailman
FI
FI
The /etc/logrotate.d/mailman file could be something
like this:
# Logrotate file for Mailman 2.0.x
$HOME/log/digest {
monthly
}
$HOME/log/error {
monthly
}
$HOME/log/post {
weekly
}
$HOME/log/smtp {
weekly
}
$HOME/log/subscribe {
monthly
}
$HOME/log/vette {
monthly
}
Of course, instead of $HOME you should place the
installation path of Mailman
----------------------------------------------------------------------
Comment By: Jim Popovitch (jimpop)
Date: 2007-04-06 21:27
Message:
Logged In: YES
user_id=3142
Originator: NO
Reopening. Mailman needs to have different log file names in order to
work _properly_ with logrotate. A proper Logrotate conf file for Mailman
needs a postrotate command in order to restart Mailman after log files have
been rotated. See FAQ 04.007 for more details
(http://www.python.org/cgi-bin/faqw-mm.py?req=edit&file=faq04.007.htp)
Here is a patch against v2.1.9
-------------------------
+++ Logging/Logger.py 2007-04-06 21:11:18.600639072 -0400
--- Logging/Logger.py.old 2007-04-06 21:10:05.672725800 -0400
@@ -43,7 +43,6 @@
Otherwise, the file is created only when there are writes
pending.
"""
self.__filename = os.path.join(mm_cfg.LOG_DIR, category)
+ self.__filename += '.log'
self.__fp = None
self.__nofail = nofail
self.__encoding = LOG_ENCODING or sys.getdefaultencoding()
-------------------------
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=350103&aid=491675&group_i…
Bugs item #1658920, was opened at 2007-02-13 05:06
Message generated for change (Comment added) made by msapiro
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Web/CGI
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dudeua (dudeua)
Assigned to: Mark Sapiro (msapiro)
Summary: charset proble,
Initial Comment:
Hello, I need help.
I have mailman 2.1.9 .... if I receive mail in charset, different
that koi8, than mailman in admins area shows this mail in
"quoted-printable" charset ...
But after approving this email, via web-interface email shows with
correct charset.
How to fix charset?
----------------------------------------------------------------------
>Comment By: Mark Sapiro (msapiro)
Date: 2007-04-03 09:43
Message:
Logged In: YES
user_id=1123998
Originator: NO
>mailman admin page shows in KOI-8 charser ....
>thouse message body with this headers
>Content-Type: text/plain; charset=windows-1251
>Content-Transfer-Encoding: 8bit
>
>shows ugly, with wrong charset.
>When I change page charset from koi-8 to windows cp1251, mailman admin
>page is ugly, but message is OK .....
This is exactly what I would expect to happen in all cases. I suggested
this would occur when I wrote on 2007-02-13 10:53:
>It would be possible to patch
>Mailman/Cgi/admindb.py with the attached patch to decode this "message
>excerpt" before displaying it, but then you would have the issue that
the
>characters in the box in your screenshot would be Windows CP1251
characters
>which would probably still be garbled when displayed in the character
set
>of the rest of the page.
The only question is why doesn't this occur when the message is
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: quoted-printable
since all the patch does is convert the quoted-printable encoding back to
the original 8bit.
At some point in the future, we expect that mailman will represent
everything internally in Unicode. At that time, this issue may be fixed.
Until then I don't anticipate this behavior to change.
The only advice I can offer is that in cases where it is important to see
the message body correctly in order to decide what action to take on a
particular held message, you either switch character sets in your browser
when you need to see the message body, or you refer to the original message
attached to the 'held message notice' email sent to the moderators which
should display properly in your email client.
----------------------------------------------------------------------
Comment By: dudeua (dudeua)
Date: 2007-04-03 02:07
Message:
Logged In: YES
user_id=1718234
Originator: YES
mailman admin page shows in KOI-8 charser ....
thouse message body with this headers
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: 8bit
shows ugly, with wrong charset.
When I change page charset from koi-8 to windows cp1251, mailman admin
page is ugly, but message is OK .....
Please note, others messages body with headers
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: quoted-printable
are shown OK in mailman admin page with charset koi-8
Thanks
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-04-02 08:56
Message:
Logged In: YES
user_id=1123998
Originator: NO
I don't know why the result is different in these two cases. In the case
of quoted-printable encoding, all non-us-ascii-printable bytes are encoded
as '=xx' where xx is the hex value of the corresponding byte. The
'decode=True' argument added by the patch causes these '=xx' codes to be
converted back to the original bytes for display in the admindb interface.
In the case of 8bit encoding, all bytes are represented as themselves and
the 'decode=True' argument does nothing. The result should be the same.
----------------------------------------------------------------------
Comment By: dudeua (dudeua)
Date: 2007-04-02 06:30
Message:
Logged In: YES
user_id=1718234
Originator: YES
Hi msapiro,
Thanks for your patch! all work fine. but today i meet ne problem.
When mesasge comes with this headers:
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: quoted-printable
All fine.
But when message is in this encoding:
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: 8bit
it's body is ugly.
Please advice me ...
Thanks!
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-02-13 10:53
Message:
Logged In: YES
user_id=1123998
Originator: NO
The screenshot is unnecessary. I understood exactly what the issue was
from your original description.
I know you only see the problem in the "message excerpt" in the admindb
message detail. I tried to explain that the reason for this is that this
excerpt is not decoded. It would be possible to patch
Mailman/Cgi/admindb.py with the attached patch to decode this "message
excerpt" before displaying it, but then you would have the issue that the
characters in the box in your screenshot would be Windows CP1251 characters
which would probably still be garbled when displayed in the character set
of the rest of the page.
Also, my suggestion to set admin_immed_notify to 1 was not intended to
'correct' this display. It causes this message to be also sent to you in an
email so that if it is necessary for you to see the message text in order
to decide what action to take with the message, you can see the text in the
email notice.
File Added: admindb.patch.txt
----------------------------------------------------------------------
Comment By: dudeua (dudeua)
Date: 2007-02-13 08:55
Message:
Logged In: YES
user_id=1718234
Originator: YES
set admin_immed_notify at /mailman/bun/config ? It already setted to 1;
Pleasee see attacged screenshot.
I can't use MUA (mail agent), because I meet problem with encoding only at
admin area.
Thanks for help.
File Added: ggg.jpg
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-02-13 08:36
Message:
Logged In: YES
user_id=1123998
Originator: NO
Quoted-printable is not a character set. It is an encoding. That is, it is
a way of representing data which contains non-printable, us-ascii
characters using only the us-ascii character set as required by RFC2822 for
email messages.
The issue here is that encoded message bodies are not decoded for display
in the admindb message detail. This may change in the future, but then
there _will_ be character set and content-type issues. These are more
complicated as in general, different message parts may have different
character sets and may not even be text.
Currently, if you really need to see the decoded text, you need to set
admin_immed_notify to Yes so you get an email notice of the held message
and use an MUA (mail client) that shows you the decoded message in the
notice.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…
Bugs item #1658920, was opened at 2007-02-13 13:06
Message generated for change (Comment added) made by dudeua
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Web/CGI
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dudeua (dudeua)
Assigned to: Mark Sapiro (msapiro)
Summary: charset proble,
Initial Comment:
Hello, I need help.
I have mailman 2.1.9 .... if I receive mail in charset, different
that koi8, than mailman in admins area shows this mail in
"quoted-printable" charset ...
But after approving this email, via web-interface email shows with
correct charset.
How to fix charset?
----------------------------------------------------------------------
>Comment By: dudeua (dudeua)
Date: 2007-04-03 09:07
Message:
Logged In: YES
user_id=1718234
Originator: YES
mailman admin page shows in KOI-8 charser ....
thouse message body with this headers
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: 8bit
shows ugly, with wrong charset.
When I change page charset from koi-8 to windows cp1251, mailman admin
page is ugly, but message is OK .....
Please note, others messages body with headers
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: quoted-printable
are shown OK in mailman admin page with charset koi-8
Thanks
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-04-02 15:56
Message:
Logged In: YES
user_id=1123998
Originator: NO
I don't know why the result is different in these two cases. In the case
of quoted-printable encoding, all non-us-ascii-printable bytes are encoded
as '=xx' where xx is the hex value of the corresponding byte. The
'decode=True' argument added by the patch causes these '=xx' codes to be
converted back to the original bytes for display in the admindb interface.
In the case of 8bit encoding, all bytes are represented as themselves and
the 'decode=True' argument does nothing. The result should be the same.
----------------------------------------------------------------------
Comment By: dudeua (dudeua)
Date: 2007-04-02 13:30
Message:
Logged In: YES
user_id=1718234
Originator: YES
Hi msapiro,
Thanks for your patch! all work fine. but today i meet ne problem.
When mesasge comes with this headers:
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: quoted-printable
All fine.
But when message is in this encoding:
Content-Type: text/plain; charset=windows-1251
Content-Transfer-Encoding: 8bit
it's body is ugly.
Please advice me ...
Thanks!
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-02-13 18:53
Message:
Logged In: YES
user_id=1123998
Originator: NO
The screenshot is unnecessary. I understood exactly what the issue was
from your original description.
I know you only see the problem in the "message excerpt" in the admindb
message detail. I tried to explain that the reason for this is that this
excerpt is not decoded. It would be possible to patch
Mailman/Cgi/admindb.py with the attached patch to decode this "message
excerpt" before displaying it, but then you would have the issue that the
characters in the box in your screenshot would be Windows CP1251 characters
which would probably still be garbled when displayed in the character set
of the rest of the page.
Also, my suggestion to set admin_immed_notify to 1 was not intended to
'correct' this display. It causes this message to be also sent to you in an
email so that if it is necessary for you to see the message text in order
to decide what action to take with the message, you can see the text in the
email notice.
File Added: admindb.patch.txt
----------------------------------------------------------------------
Comment By: dudeua (dudeua)
Date: 2007-02-13 16:55
Message:
Logged In: YES
user_id=1718234
Originator: YES
set admin_immed_notify at /mailman/bun/config ? It already setted to 1;
Pleasee see attacged screenshot.
I can't use MUA (mail agent), because I meet problem with encoding only at
admin area.
Thanks for help.
File Added: ggg.jpg
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-02-13 16:36
Message:
Logged In: YES
user_id=1123998
Originator: NO
Quoted-printable is not a character set. It is an encoding. That is, it is
a way of representing data which contains non-printable, us-ascii
characters using only the us-ascii character set as required by RFC2822 for
email messages.
The issue here is that encoded message bodies are not decoded for display
in the admindb message detail. This may change in the future, but then
there _will_ be character set and content-type issues. These are more
complicated as in general, different message parts may have different
character sets and may not even be text.
Currently, if you really need to see the decoded text, you need to set
admin_immed_notify to Yes so you get an email notice of the held message
and use an MUA (mail client) that shows you the decoded message in the
notice.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1658920&group_…
Bugs item #1661574, was opened at 2007-02-16 06:54
Message generated for change (Comment added) made by msapiro
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1661574&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: command line scripts
Group: 2.1 (stable)
>Status: Pending
Resolution: None
Priority: 9
Private: No
Submitted By: David A. Desrosiers (desrod)
Assigned to: Nobody/Anonymous (nobody)
Summary: arch corrupts archives, but only for the last month
Initial Comment:
I think this problem has been reported before in previous versions, and its back again in 2.1.9.
When I regenerate archives for our lists, if ANY message contains a '<' character in the body, Mailman splits it as a new message, and everything after that gets corrupted.
This means if someone pastes some XML into the body of a message (which happens quite often on our lists) or some HTML, or the headers of an email, Mailman will break it, but *ONLY* for the latest month's messages, even if the message that started it, was months or years ago.
If a message sent in April of 2003 includes an '<' as the first character anywhere in the body of the message, February 2007's archive will be corrupted.
You can see the results of this over here:
http://lists.plkr.org/pipermail/plucker-list/2007-February/thread.html
And also here:
http://lists.plkr.org/pipermail/plucker-dev/2007-February/thread.html
The raw mbox files are fine, every message is intact.
I don't see this problem on other lists I maintain, it only seems to affect lists where HTML or XML or mail headers are pasted into the body of the message.
I'd call this grave, because its odd how it just dumps itself on the latest month's archive, when the latest month's messages don't even have the problem.
----------------------------------------------------------------------
>Comment By: Mark Sapiro (msapiro)
Date: 2007-04-02 09:20
Message:
Logged In: YES
user_id=1123998
Originator: NO
No response after 6 weeks. I'm setting status to Pending which will
automatically close in 2 more weeks.
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2007-02-16 08:58
Message:
Logged In: YES
user_id=1123998
Originator: NO
What am I looking for at
<http://lists.plkr.org/pipermail/plucker-list/2007-February/thread.html>?
It looks OK to me.
<http://lists.plkr.org/pipermail/plucker-dev/2007-February/thread.html>
returns a 404.
There is an issue in that if the body of some message in the
archives/private/<listname>.mbox/<listname>.mbox file (or whatever mbox is
input to bin/arch) contains a line that begins with "From ", the archiver
takes that line as an mbox message separator and the message is truncated
at that point, and the rest of the message is seen as a new message without
a date so it is archived with the current date.
It sounds like that may be what you are seeing, but it has nothing to do
with a '<' as the first character of a line. It has to do with 'unescaped'
'From ' lines in the bodies of messages.
Mailman currently precedes any 'From ' at the beginning of a body line
with a '>' making it '>From ' in the .mbox and avoiding the problem, but
old .mbox files and .mbox files from other sources may have unescaped 'From
' lines.
There is a bin/cleanarch script distributed with Mailman to help 'fix' old
.mbox files with this problem.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1661574&group_…