Mailman-coders September 2004

mailman-coders@python.org

2 participants
67 discussions

[ mailman-Feature Requests-1024289 ] Add support for $Orig and $REF for Lotus Notes < 5.0.9
by SourceForge.net 14 Sep '04

14 Sep '04

Feature Requests item #1024289, was opened at 2004-09-08 10:35 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1024289&group_… >Category: None >Group: None Status: Open Resolution: None Priority: 5 Submitted By: yoda (shobhanc) >Assigned to: Nobody/Anonymous (nobody) Summary: Add support for $Orig and $REF for Lotus Notes < 5.0.9 Initial Comment: Lotus Notes below 6.25 doesnt set the 'In-Reply-To' or 'References' header. but it sets 'Orig' and 'REF' headers. Mailman should be able to process 'Orig' and 'REF' headers if 'In-Reply-To' is not set for Lotus Notes. Does anyone have any idea how to do this..? Thanks ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-09-14 13:06 Message: Logged In: YES user_id=67709 No patch. Requesting a feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=350103&aid=1024289&group_…

1 0

[ mailman-Patches-1024289 ] Add support for $Orig and $REF for Lotus Notes < 5.0.9
by SourceForge.net 14 Sep '04

14 Sep '04

Patches item #1024289, was opened at 2004-09-08 10:35 Message generated for change (Settings changed) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1024289&group_… Category: Pipermail Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: yoda (shobhanc) >Assigned to: Tokio Kikuchi (tkikuchi) Summary: Add support for $Orig and $REF for Lotus Notes < 5.0.9 Initial Comment: Lotus Notes below 6.25 doesnt set the 'In-Reply-To' or 'References' header. but it sets 'Orig' and 'REF' headers. Mailman should be able to process 'Orig' and 'REF' headers if 'In-Reply-To' is not set for Lotus Notes. Does anyone have any idea how to do this..? Thanks ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1024289&group_…

1 0

[ mailman-Patches-1027882 ] filter attachments by filename extensions
by SourceForge.net 14 Sep '04

14 Sep '04

Patches item #1027882, was opened at 2004-09-14 12:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1027882&group_… Category: mail delivery Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: filter attachments by filename extensions Initial Comment: Mailman's filter attachment feature is useful but many MUA's are sending same application files with different content-type: eg. application/pdf or application/octet-stream for PDF. Viruses also fake the content-type for executables as audio etc. So, we need to filter attachments by filename extensions by which windows assumes their associated applications. This patch enables list owners to define filename extensions to filter (block) or pass the attached file. After applying patch, configure and make install, you may have to 'bin/update -f' to force the list attribute upgrading. You may also have to check the config.pck schema version (DATA_FILE_VERSION) in Version.py and make sure the number is incremented. (90 -> 91) The old number may be different if you have applied other patches. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1027882&group_…

1 0

[ mailman-Patches-601117 ] add sequencial number in subject prefix
by SourceForge.net 14 Sep '04

14 Sep '04

Patches item #601117, was opened at 2002-08-28 10:07 Message generated for change (Comment added) made by shobhanc You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=601117&group_i… Category: mail delivery Group: Mailman 2.2 / 3.0 Status: Open Resolution: None Priority: 7 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: add sequencial number in subject prefix Initial Comment: This patch for 'CookHeaders.py' add an ability to add a sequencial number in the subject prefix. You can define a subject prefix like: [listname %d] Then, the subject line of delivered mail becomes: Subject: [listname 123] Hoge hoge When someone replied this mail, mailman receives a messages with: Subject: Re: [listname 123] Hoge hoge Then, this patch removes [listname \d+] part and deliver it with: Subject: [listname 124] Re: Hoge hoge And next, another person replies with Subject: Re: [listname 124] Re: Hoge hoge Then, (magically!) you get: Subject: [listname 125] Re: Hoge hoge Not with Re: Re: Hoge hoge. Looks like complicated but this patch has been working well with Japanese-enhanced Mailman for more than a year. Without %d, this patch works like current version, I believe. ---------------------------------------------------------------------- Comment By: shobhan challa (shobhanc) Date: 2004-09-14 11:16 Message: Logged In: YES user_id=598275 Hi tkikuchi, There are some gibberish text within the patch, are they Japanesh chars..?? If possible can you look at this tracker and give your suggestions.https://sourceforge.net/tracker/index.php?func=detail&aid=10242… This is basically I want to implement threading in Mailman. Yes Mailman implements threading I agree but when the mail replies are sent using Lotus Notes 5.0.8/5.0.9 the threading doesn't work because Lotus Notes doesn't set the "In-Reply-To" header. Thanks for your time ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-05-25 02:28 Message: Logged In: YES user_id=67709 You can download the patch for 2.1.5 from http://mm.tkikuchi.net/mailman-2.1.5+patch.20040516.gz TK ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-02-08 11:10 Message: Logged In: YES user_id=67709 This patch for 2.1.3 is also applicable to 2.1.4. Though one patch fails to apply, it can safely be neglected. You can also get a newer and more convenient patch at: http://mm.tkikuchi.net/mailman-2.1.4+patch.20040123.gz Cheers, TK ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2003-09-29 06:21 Message: Logged In: YES user_id=67709 Now the patch includes instruction in General admin page. (in detail part). In source dir you should type: patch -p0 < /path/to/seqnum.patch ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2003-05-18 07:01 Message: Logged In: YES user_id=67709 Here is update of my patch for 2.1.2 ---------------------------------------------------------------------- Comment By: Mitchell Marks (jojoba2) Date: 2003-05-17 22:42 Message: Logged In: YES user_id=718482 Is there a version vor MM 2.1.2? I'm trying the older one but doesn't seem to work. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2003-02-27 10:33 Message: Logged In: YES user_id=67709 sorry for daily update but there was an error in i18n handling of '(no subject)' ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2003-02-26 06:25 Message: Logged In: YES user_id=67709 update of patch for 2.1.1 if the subject is all ascii, then cat the prefix and subject before it is instantiated. Hope to solve prefix only subject 1st line. Note: with this patch without $d, Re: [prefix] is now mungled to [prefix] Re:. This is useful if someone post new issue as follow up to old subject like; New subject bah bah was Re: [prefix] foo ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2003-02-14 16:33 Message: Logged In: YES user_id=67709 Hi, I am pleased you like this patch. Here is my current version attached. ---------------------------------------------------------------------- Comment By: Fabio Rossetti (fabiorossetti) Date: 2003-02-14 15:02 Message: Logged In: YES user_id=693899 If you need a quick fix to make the patch work with 2.1.1 here's my take (it's the diff file adapted to work with 2.1.1 CookHeaders.py) 17a18 > (sequence version) 221c222,223 < prefix = mlist.subject_prefix --- > prefix = mlist.subject_prefix.strip(); > if not prefix: return 237,243c239,250 < if prefix and subject: < pattern = re.escape(prefix.strip()) < for decodedsubj, charset in headerbits: < if re.search(pattern, decodedsubj, re.IGNORECASE): < # The subject's already got the prefix, so don't change it < return < del msg['subject'] --- > headerstring = '' > fws = '' > cset = None > for (s, c) in headerbits: > headerstring += fws + s > if c == None or c == 'us-ascii': > fws = ' ' > cset = Utils.GetCharSet(mlist.preferred_language) > else: > fws = '' > cset = c > # Note: searching prefix in subject is REMOVED. (seq version) 245a253,275 > else: > subject = headerstring > # If the subject_prefix contains '%d', it is replaced with the > # mailing list sequential number. Also, if the prefix is closed with > # [],(), or {}, the prefix in the responding post subject will be cared. > # sequential number format allows '%05d' like pattern. > p = re.compile('%\d*d') > if p.search(prefix,1): > # prefix have number, so we should search prefix w/number > # in subject. > prefix_pattern = p.sub(r'\s*\d+\s*', prefix) > else: > prefix_pattern = prefix > prefix_pattern = re.sub('([\[\(\{])', '\\\g<1>', prefix_pattern) > subject = re.sub(prefix_pattern, '', subject) > subject = re.compile('(RE:\s*)+', re.I).sub('Re: ', subject, 1) > # and substitute %d in prefix with post_id > try: > prefix = prefix % mlist.post_id > except: > pass > # Note that trailing space was stripped in seq version (TK) > prefix += ' ' 248,262c278,288 < for s, c in headerbits: < # Once again, convert the string to unicode. < if c is None: < c = Charset('iso-8859-1') < if not isinstance(c, Charset): < c = Charset(c) < if not _isunicode(s): < codec = c.input_codec or 'ascii' < try: < s = unicode(s, codec, 'replace') < except LookupError: < # Unknown codec, is this default reasonable? < s = unicode(s, Utils.GetCharSet (mlist.preferred_language), < 'replace') < h.append(s, c) --- > # in seq version, subject header is already concatnated > if not _isunicode(subject): > try: > subject = unicode(subject, cset, 'replace') > except LookupError: > # unknown codec > cset = Utils.GetCharSet(mlist.preferred_language) > subject = unicode(subject, cset, 'replace') > subject = subject.encode(cset) > h.append(subject, cset) > del msg['subject'] ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2002-12-20 09:31 Message: Logged In: YES user_id=67709 update for 2.1b6+ ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2002-10-31 10:59 Message: Logged In: YES user_id=67709 I have uploaded the patch with the same name as the old one. Please download the upper one because it's newer. Sorry for the inconvenience. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2002-10-31 10:53 Message: Logged In: YES user_id=67709 Patch ID 625482 (i18n List-Id) and this was merged for 2.1b4+ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=601117&group_i…

1 0

[ mailman-Patches-891491 ] Scrubber.py patch
by SourceForge.net 13 Sep '04

13 Sep '04

Patches item #891491, was opened at 2004-02-06 01:26 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i… Category: Pipermail Group: Mailman 2.1 Status: Open Resolution: None Priority: 8 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: Scrubber.py patch Initial Comment: Scrubber.py has number of bugs for processing various types of attachment and languages and many have submitted patches to fix them. This patch item is opened to collect such patches for convenience. This patch corrects: - if an attached text is composed by win notepad, it has no charset specified and actual charset may be different from message/list charset. This sometimes cause error in composing digest message. - sometimes, null charset is represented by '' as well as None. - embedded rfc-2822 message is lost if you don't use msg.walk() - special problem with japanese charsets. - t (stringfied part) may be None which you can't append a '\n'. ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-09-14 00:25 Message: Logged In: YES user_id=67709 Your sample email did not reproduce error. Anyway, I managed to make a message which can cause the error and confirmed the last fix was valid. Thank you for your cooperation. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-13 18:23 Message: Logged In: YES user_id=113859 The patch looks better, but I probably can only test it when I updated my Mailman patches again. I have directly send you an email that should trigger the bug so you can test. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-09-11 23:45 Message: Logged In: YES user_id=67709 Sorry for the problem. I have updated this patch (2004-09-12 version) Please try and report this patch because I can't fully reproduce the problem (in Japanese environment). ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-11 18:04 Message: Logged In: YES user_id=113859 scrubber.patch 2004-08-24 has a problem. Charset(lcset).output_charset can return None according to the documentation. E.g. when no conversion is needed. This would assign Non to lcset_out . Later this leads to an exception in try: if len(charset) == 0: charset = 'us-ascii' t = t.encode(charset, 'replace') except (UnicodeError, LookupError, ValueError): t = t.encode(lcset, 'replace') because: TypeError: len() of unsized object ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-08-24 07:58 Message: Logged In: YES user_id=67709 added a fix for the case of lcset_out <> lcset. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-04-14 18:53 Message: Logged In: YES user_id=113859 Thanks for working on the patch. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-02-25 10:42 Message: Logged In: YES user_id=67709 uploading revised patch. Now fixes a few more bugs which try to decode scrub plain text message and result in mojibake. Also, japanese filename tend to become so long that system limit may exceeded because of mime encoding, so add an option not to use the filename in the message but to use 'attachment' as filename. Because this patch spans two files (Defaults.py.in and Handlers/Scrubber.py) you have to cd mailman and patch -p1 < this_patch. (Well, I think it is -p1. If it didn't work, try -p0 ;-) ---------------------------------------------------------------------- Comment By: Jonathan Larmour (jifl) Date: 2004-02-22 17:25 Message: Logged In: YES user_id=817601 I strongly recommend applying this patch. I received a mail bounce on a list with an empty charset in a part (i.e. "charset=") and it caused /var/mailman/cron/senddigest and thus all digest processing to fail because of this error: Traceback (most recent call last): File "/var/mailman/cron/senddigests", line 94, in ? main() File "/var/mailman/cron/senddigests", line 86, in main mlist.send_digest_now() File "/var/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 123, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 295, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 308, in process t = t.encode(charset, 'replace') File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function mod = __import__(modname,globals(),locals(),'*') which is something this patch fixes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i…

1 0

[ mailman-Patches-891491 ] Scrubber.py patch
by SourceForge.net 13 Sep '04

13 Sep '04

Patches item #891491, was opened at 2004-02-06 02:26 Message generated for change (Comment added) made by ber You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i… Category: Pipermail Group: Mailman 2.1 Status: Open Resolution: None Priority: 8 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: Scrubber.py patch Initial Comment: Scrubber.py has number of bugs for processing various types of attachment and languages and many have submitted patches to fix them. This patch item is opened to collect such patches for convenience. This patch corrects: - if an attached text is composed by win notepad, it has no charset specified and actual charset may be different from message/list charset. This sometimes cause error in composing digest message. - sometimes, null charset is represented by '' as well as None. - embedded rfc-2822 message is lost if you don't use msg.walk() - special problem with japanese charsets. - t (stringfied part) may be None which you can't append a '\n'. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-13 20:23 Message: Logged In: YES user_id=113859 The patch looks better, but I probably can only test it when I updated my Mailman patches again. I have directly send you an email that should trigger the bug so you can test. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-09-12 01:45 Message: Logged In: YES user_id=67709 Sorry for the problem. I have updated this patch (2004-09-12 version) Please try and report this patch because I can't fully reproduce the problem (in Japanese environment). ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-11 20:04 Message: Logged In: YES user_id=113859 scrubber.patch 2004-08-24 has a problem. Charset(lcset).output_charset can return None according to the documentation. E.g. when no conversion is needed. This would assign Non to lcset_out . Later this leads to an exception in try: if len(charset) == 0: charset = 'us-ascii' t = t.encode(charset, 'replace') except (UnicodeError, LookupError, ValueError): t = t.encode(lcset, 'replace') because: TypeError: len() of unsized object ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-08-24 09:58 Message: Logged In: YES user_id=67709 added a fix for the case of lcset_out <> lcset. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-04-14 20:53 Message: Logged In: YES user_id=113859 Thanks for working on the patch. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-02-25 11:42 Message: Logged In: YES user_id=67709 uploading revised patch. Now fixes a few more bugs which try to decode scrub plain text message and result in mojibake. Also, japanese filename tend to become so long that system limit may exceeded because of mime encoding, so add an option not to use the filename in the message but to use 'attachment' as filename. Because this patch spans two files (Defaults.py.in and Handlers/Scrubber.py) you have to cd mailman and patch -p1 < this_patch. (Well, I think it is -p1. If it didn't work, try -p0 ;-) ---------------------------------------------------------------------- Comment By: Jonathan Larmour (jifl) Date: 2004-02-22 18:25 Message: Logged In: YES user_id=817601 I strongly recommend applying this patch. I received a mail bounce on a list with an empty charset in a part (i.e. "charset=") and it caused /var/mailman/cron/senddigest and thus all digest processing to fail because of this error: Traceback (most recent call last): File "/var/mailman/cron/senddigests", line 94, in ? main() File "/var/mailman/cron/senddigests", line 86, in main mlist.send_digest_now() File "/var/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 123, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 295, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 308, in process t = t.encode(charset, 'replace') File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function mod = __import__(modname,globals(),locals(),'*') which is something this patch fixes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i…

1 0

[ mailman-Bugs-975768 ] senddigests doesn't handle consecutive unparseable messages
by SourceForge.net 13 Sep '04

13 Sep '04

Bugs item #975768, was opened at 2004-06-19 09:07 Message generated for change (Comment added) made by thomas_ah You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=975768&group_i… Category: mail delivery Group: 2.1 (stable) Status: Open Resolution: None Priority: 5 Submitted By: Andreas Voegele (voegelas) Assigned to: Nobody/Anonymous (nobody) Summary: senddigests doesn't handle consecutive unparseable messages Initial Comment: The following code fragment in send_i18n_digests in Mailman/ Handlers/ToDigest.py doesn't take into account that there could be several consecutive messages that are unparseable: msg = mbox.next() while msg is not None: if msg == '': # It was an unparseable message msg = mbox.next() [...] # Get the next message in the digest mailbox msg = mbox.next() I'm not that familiar with Python but I think that the following code should fix the problem: while 1: msg = mbox.next() if msg is None: break if msg == '': # It was an unparseable message continue [...] ---------------------------------------------------------------------- Comment By: Thomas Arendsen Hein (thomas_ah) Date: 2004-09-13 14:30 Message: Logged In: YES user_id=839582 This bug is still present and hit our mailman installation last friday. The problem occurs if the unparseable message is the last one, too. Here is a patch against current CVS, effectively doing the same as Andreas Voegele suggested, but with cleaner code. Unfortunately this tracker doesn't allow me to attach it, because I didn't submit this bug. diff -u -r2.28 ToDigest.py --- Mailman/Handlers/ToDigest.py 15 Aug 2003 21:06:28 -0000 2.28 +++ Mailman/Handlers/ToDigest.py 13 Sep 2004 12:22:03 -0000 @@ -210,6 +210,7 @@ if msg == '': # It was an unparseable message msg = mbox.next() + continue msgcount += 1 messages.append(msg) # Get the Subject header ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=975768&group_i…

1 0

[ mailman-Patches-1026977 ] check attachment header by SpamDetect.py
by SourceForge.net 12 Sep '04

12 Sep '04

Patches item #1026977, was opened at 2004-09-13 02:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1026977&group_… Category: list administration Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: check attachment header by SpamDetect.py Initial Comment: SpamDetect.py and Privacy -> Spam interface can check incoming message haeder by regular expressions and discard/reject/hold the message. It can prevent viruses by blocking multipart messages but many users are now want to pass multipart. This patch extents the check into the attachment header so that viruses which have .exe/.pif file can be prevented by the same interface. Note that cheking is limited to the second level of multiparts because collecting all the header will cause infinite loop because owner notification message is also checked by this module. Usage: after applying this patch and restart mailman, go to the privacy/spam admin page and insert a rule content-.*name.*\.(exe|pif|com|vbs) to be discarded. Cheers, ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1026977&group_…

1 0

[ mailman-Patches-891491 ] Scrubber.py patch
by SourceForge.net 11 Sep '04

11 Sep '04

Patches item #891491, was opened at 2004-02-06 01:26 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i… Category: Pipermail Group: Mailman 2.1 Status: Open Resolution: None Priority: 8 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: Scrubber.py patch Initial Comment: Scrubber.py has number of bugs for processing various types of attachment and languages and many have submitted patches to fix them. This patch item is opened to collect such patches for convenience. This patch corrects: - if an attached text is composed by win notepad, it has no charset specified and actual charset may be different from message/list charset. This sometimes cause error in composing digest message. - sometimes, null charset is represented by '' as well as None. - embedded rfc-2822 message is lost if you don't use msg.walk() - special problem with japanese charsets. - t (stringfied part) may be None which you can't append a '\n'. ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-09-11 23:45 Message: Logged In: YES user_id=67709 Sorry for the problem. I have updated this patch (2004-09-12 version) Please try and report this patch because I can't fully reproduce the problem (in Japanese environment). ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-11 18:04 Message: Logged In: YES user_id=113859 scrubber.patch 2004-08-24 has a problem. Charset(lcset).output_charset can return None according to the documentation. E.g. when no conversion is needed. This would assign Non to lcset_out . Later this leads to an exception in try: if len(charset) == 0: charset = 'us-ascii' t = t.encode(charset, 'replace') except (UnicodeError, LookupError, ValueError): t = t.encode(lcset, 'replace') because: TypeError: len() of unsized object ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-08-24 07:58 Message: Logged In: YES user_id=67709 added a fix for the case of lcset_out <> lcset. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-04-14 18:53 Message: Logged In: YES user_id=113859 Thanks for working on the patch. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-02-25 10:42 Message: Logged In: YES user_id=67709 uploading revised patch. Now fixes a few more bugs which try to decode scrub plain text message and result in mojibake. Also, japanese filename tend to become so long that system limit may exceeded because of mime encoding, so add an option not to use the filename in the message but to use 'attachment' as filename. Because this patch spans two files (Defaults.py.in and Handlers/Scrubber.py) you have to cd mailman and patch -p1 < this_patch. (Well, I think it is -p1. If it didn't work, try -p0 ;-) ---------------------------------------------------------------------- Comment By: Jonathan Larmour (jifl) Date: 2004-02-22 17:25 Message: Logged In: YES user_id=817601 I strongly recommend applying this patch. I received a mail bounce on a list with an empty charset in a part (i.e. "charset=") and it caused /var/mailman/cron/senddigest and thus all digest processing to fail because of this error: Traceback (most recent call last): File "/var/mailman/cron/senddigests", line 94, in ? main() File "/var/mailman/cron/senddigests", line 86, in main mlist.send_digest_now() File "/var/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 123, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 295, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 308, in process t = t.encode(charset, 'replace') File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function mod = __import__(modname,globals(),locals(),'*') which is something this patch fixes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i…

1 0

[ mailman-Patches-891491 ] Scrubber.py patch
by SourceForge.net 11 Sep '04

11 Sep '04

Patches item #891491, was opened at 2004-02-06 02:26 Message generated for change (Comment added) made by ber You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i… Category: Pipermail Group: Mailman 2.1 Status: Open Resolution: None Priority: 8 Submitted By: Tokio Kikuchi (tkikuchi) Assigned to: Nobody/Anonymous (nobody) Summary: Scrubber.py patch Initial Comment: Scrubber.py has number of bugs for processing various types of attachment and languages and many have submitted patches to fix them. This patch item is opened to collect such patches for convenience. This patch corrects: - if an attached text is composed by win notepad, it has no charset specified and actual charset may be different from message/list charset. This sometimes cause error in composing digest message. - sometimes, null charset is represented by '' as well as None. - embedded rfc-2822 message is lost if you don't use msg.walk() - special problem with japanese charsets. - t (stringfied part) may be None which you can't append a '\n'. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-09-11 20:04 Message: Logged In: YES user_id=113859 scrubber.patch 2004-08-24 has a problem. Charset(lcset).output_charset can return None according to the documentation. E.g. when no conversion is needed. This would assign Non to lcset_out . Later this leads to an exception in try: if len(charset) == 0: charset = 'us-ascii' t = t.encode(charset, 'replace') except (UnicodeError, LookupError, ValueError): t = t.encode(lcset, 'replace') because: TypeError: len() of unsized object ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-08-24 09:58 Message: Logged In: YES user_id=67709 added a fix for the case of lcset_out <> lcset. ---------------------------------------------------------------------- Comment By: Bernhard Reiter (ber) Date: 2004-04-14 20:53 Message: Logged In: YES user_id=113859 Thanks for working on the patch. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2004-02-25 11:42 Message: Logged In: YES user_id=67709 uploading revised patch. Now fixes a few more bugs which try to decode scrub plain text message and result in mojibake. Also, japanese filename tend to become so long that system limit may exceeded because of mime encoding, so add an option not to use the filename in the message but to use 'attachment' as filename. Because this patch spans two files (Defaults.py.in and Handlers/Scrubber.py) you have to cd mailman and patch -p1 < this_patch. (Well, I think it is -p1. If it didn't work, try -p0 ;-) ---------------------------------------------------------------------- Comment By: Jonathan Larmour (jifl) Date: 2004-02-22 18:25 Message: Logged In: YES user_id=817601 I strongly recommend applying this patch. I received a mail bounce on a list with an empty charset in a part (i.e. "charset=") and it caused /var/mailman/cron/senddigest and thus all digest processing to fail because of this error: Traceback (most recent call last): File "/var/mailman/cron/senddigests", line 94, in ? main() File "/var/mailman/cron/senddigests", line 86, in main mlist.send_digest_now() File "/var/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 123, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/mailman/Mailman/Handlers/ToDigest.py", line 295, in send_i18n_digests msg = scrubber(mlist, msg) File "/var/mailman/Mailman/Handlers/Scrubber.py", line 308, in process t = t.encode(charset, 'replace') File "/usr/lib/python2.2/encodings/__init__.py", line 51, in search_function mod = __import__(modname,globals(),locals(),'*') which is something this patch fixes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=891491&group_i…

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

Mailman-coders September 2004