Mailman 3 September 2003 - Mailman-Developers

Bounce removal parameters default values
by Greg Stark 23 Oct '03

23 Oct '03

I find I am being removed from mailman mailing lists left and right. I believe the default values for the bounce removal should be reconsidered. It's possible that you haven't had many users in my situation and so haven't really had a chance to tune these parameters on the low end yet. But they clearly aren't working for me at a few sites. My particular situation is that my site has seen fit to filter viruses by refusing delivery. This causes a bounce from the remote MTA every time someone sends me an Outlook virus. Why my site administrators felt this was necessary is a question for another day, it's not like I use Outlook or like my spam filters wouldn't have thrown these messages away anyways, but whatever. The net result is that some small fraction of messages to me bounce and list management software notices this. The only reason I became aware of the problem was because ezmlm also does this type of processing but it sends a warning message before removing users. It only removes you if the warning message itself bounces. In fact it sends two such warning messages and only removes the user if *both* bounce. This provides the user with a chance to react to the first message and fix the problem -- if they ever see the message. I could beg for a similar feature in mailman, but I'm not sure it's necessary. But I am sure it's necessary to tune the bounce processing parameters. The relatively few bounces I'm generating shouldn't be causing me to get removed when all the real messages are being delivered fine. It seems the legitimate messages that are correctly delivered should reset the count of bounces to 0. Reading the source it seems it has to see DEFAULT_MAX_POSTS_BETWEEN_BOUNCES such legitimate posts between messages. I'm fairly convinced this parameter should always be 0. If any successful delivery occurs the user should never be removed due to bounces. What I don't understand is how DEFAULT_MAX_POSTS_BETWEEN_BOUNCES relates to the parameters I see in the admin. None of the parameters in the admin corresponds to this. How is it calculated? -- greg

6 15

Mailman & CPanel?
by Barry Warsaw 21 Oct '03

21 Oct '03

I recently received a message about problems with Mailman and CPanel. I don't know anything about CPanel, or why Mailman and CPanel might not be working together, but that doesn't mean I wouldn't be interested in helping to make sure this is still a viable solution. I know a lot of people use Mailman under CPanel, and I think that's a worthwhile thing to keep available. I'm perplexed by something the message said though: After many complaints, support at the host company said this: "Well, mailman has just released a statement that they will not support CPanel-based mailman installations anymore due to the fact that the CPanel developers have not correctly implemented their solution and causing it to fail, so there isn't much we can do either if both mailman an cpanel developers do not support it. CPanel will most likely be discontinuing it from all servers, too." There are many things I don't understand about this, but let's start with "mailman has just released a statement...". I'm pretty sure /I/ didn't release such a statement, so unless someone has hijacked or forged my address, I don't believe this is accurate. :) Does anybody know anything about this? I'd like to get the record corrected: I have nothing against supporting CPanel in principle, and I'd be willing to learn more about what the problems were, and if there was anything we could do in Mailman to make this combination continue to work. Cheers, -Barry

7 10

Problem with MM after power outage
by Bill Bradford 10 Oct '03

10 Oct '03

After power was restored and the machine came up, Mailman (2.1.2) is now spitting out errors: Traceback (most recent call last): File "/usr/local/mailman/bin/qrunner", line 270, in ? main() File "/usr/local/mailman/bin/qrunner", line 230, in main qrunner.run() File "/usr/local/mailman/Mailman/Queue/Runner.py", line 59, in run filecnt = self._oneloop() File "/usr/local/mailman/Mailman/Queue/Runner.py", line 88, in _oneloop msg, msgdata = self._switchboard.dequeue(filebase) File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 144, in dequeue data = self._ext_read(dbfile) File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 246, in _ext_read dict = marshal.load(fp) ValueError: bad marshal data Any suggestions? Worked fine before power got yanked... Bill -- bill bradford mrbill(a)mrbill.net austin, texas

7 28

MailMan Archives and more cPanel Control
by Gregory A. Clark 01 Oct '03

01 Oct '03

I can only create and maintain my mailing list options via cPanel. The actual mailman files are not in a user accessible area for me. I have private archival set on. When I look at the generated archive page there is: 1. a link to download the full raw archive 2. a link "Downloadable version [ Gzip'd Text xx KB ]" Is there any development initiative to allow more control of the MailMail archiving via cPanel? Is there any current way I can suppress these, given my obvious lack of system privileges? I don't want these archive download links to appear. I have the following at my current web hosting account: MailMan Version 2.1.2 Pipermail 0.09 cPanel Version 7.4.2 MySQL Version 4.0.15 PHP Version 4.3.3 Regards, Greg

2 1

Mailman/pipermail/MHonArc
by Richard Barrett 01 Oct '03

01 Oct '03

This topic got some airtime on the list recently and I thought I would share my thoughts on it. Mailman/pipermail has got some strengths and some weaknesses. Ditto MHonArc. Being greedy and unable to wait for the Mailman 3's wizzy archiver/archive-search solution I decided to try for the best of both worlds from what code is currently available. The result is a patch for Mailman 2.1.3 (the MM 2.1.2 solution was strictly in-house) which creates a Mailman/pipermail/MHonArc integration/fusion. This may be rubbish or the greatest thing since sliced bread. That being the case I have yet to upload the patch to sourceforge. Anybody with an opinion on such an action is welcome to (politely) give their opinion about it. My description of the patch and the patch is available at: http://www.openinfo.co.uk/mailman/patches/mhonarc/index.html This patch of mine works but is a crude hack and I would really love to work on a really good replacement for pipermail + search engine. As an aside, I have been using Perl since 1993 and have a love-hate relationship with it. It works extremely well and has advanced our capabilities but it offends most of my sensibilities. There has to be be a better solution than MHonArc (and pipermail) but Earl Hood does do such a good job with Perl. Enough rant/navel-fluff-checking. ----------------------------------------------------------------------- Richard Barrett http://www.openinfo.co.uk

1 0

MM 2.1.3 version Mailman-htdig integration patches available
by Richard Barrett 01 Oct '03

01 Oct '03

I have uploaded updated files to sourceforge for the following patches and bugs I maintain, which include my Mailman-HTdig integration patches. You can reach them through: http://sourceforge.net/tracker/?group_id=103 Mailman - Patches ----------------- #444879 - Archive indexer control to improve index #444884 - Integration of Mailman & htdig for archi #644797 - Revised mailer exit status #644810 - Sendmail mailer in Python #760567 - moderation request message content Mailman - Bugs -------------- #777444 - mailmanctl doesn't setgroups when run as root They are also available through: http://www.openinfo.co.uk/mailman/index.html If you encounter any problems with downloading or using these patches then let me know. ----------------------------------------------------------------------- Richard Barrett http://www.openinfo.co.uk

2 1

Not sleeping for locks?
by Pug Bainter 30 Sep '03

30 Sep '03

Good Morning, We've been noticing a very high load on our fileserver that holds our mailman spool. This morning when I ran 2 archivers at once because of a backlog to a list, I noticed that the stat request volume sky rocketed. From strace'ing a second archiver running, I see that the system doesn't even a tenth of a second in what I assume is the time.sleep call. [...] 11:15:10.712914 gettimeofday({1064938510, 712937}, NULL) = 0 11:15:10.712976 gettimeofday({1064938510, 712998}, NULL) = 0 <stat> 11:15:11.512977 gettimeofday({1064938511, 512995}, NULL) = 0 11:15:11.513035 gettimeofday({1064938511, 513051}, NULL) = 0 <stat> [...] Now maybe I'm not understanding what the following code is supposed to do: def __sleep(self): interval = random.random() * 2.0 + 0.01 time.sleep(interval) but I read it to be sleeping for between 0.01 and 2.01 seconds. During my traces, I never saw it sleep even 0.01 seconds, let alone anything ressembling something reasonably random. Am I just missing something? This was noticed on: Mailman version: 2.1.2 Python version: 2.2.2 OS version: Solaris 2.8 and RedHat 7.3 Thanks for any help! NOTE: I'm not on the developers list, but I thought they might have an answer for this behavior. Ciao, -- Pug Bainter | AMD, Inc. System Engineer, MTS | Mail Stop 625 Pug.Bainter(a)amd.com | pug(a)pug.net | 5900 E. Ben White Blvd Phone: (512) 602-0364 | Fax: (512) 602-6970 | Austin, TX 78741 Note: The views may not reflect my employers, or even my own for that matter.

1 0

[java-help@gcc.gnu.org] ezmlm warning
by Greg Stark 30 Sep '03

30 Sep '03

As an example of what I mean this is the notice that ezmlm sends. It's really helpful too, it explains that the messages have been bouncing, provides an example of the bounce so the user can make a guess *why* they're bouncing, and provides a link to the archives (actually instructions on retrieving them by e-mail). It also explains that in order to drop you from the list the bounce notification message has to itself bounce. In fact it goes a step further and sends a followup message. It will only remove you if that message bounces too. I think mailman really needs something like this. I haven't been removed from any ezmlm lists even though I periodically get these notifications from lots of them. In fact I wouldn't even know *why* I was being dropped from mailman lists regularly if it weren't for these ezmlm notifications. That's really bad. Mailman drops people without any warning or notification of what it's doing or why. -- greg

1 0

"@" in mail **text** gets replaced in archives
by Martin Maechler 30 Sep '03

30 Sep '03

>>>>> "Roger" == Roger Bivand <Roger.Bivand(a)nhh.no> >>>>> on Fri, 26 Sep 2003 14:26:58 +0200 (CEST) writes: ..... Roger> The second thing, ...: my answer on r-help to a pixmap question Roger> was: Roger> ... Roger> xx@col[1:20] Roger> ... Roger> which in the archives is rendered: Roger> ... Roger> xx at col[1:20] Roger> ... aaarg! The mailman-builtin archiving engine (pipermail) is really not that great.... At the moment I have to stay with it. I can only switch back to keep e-mail addresses unaltered. which caters to the spammers address-collection robots... Of course that's a bug in mailman/pipermail ===> forwarding to the developers. Martin Maechler <maechler(a)stat.math.ethz.ch> http://stat.ethz.ch/~maechler/ Seminar fuer Statistik, ETH-Zentrum LEO C16 Leonhardstr. 27 ETH (Federal Inst. Technology) 8092 Zurich SWITZERLAND phone: x-41-1-632-3408 fax: ...-1228 <><

13 26

RELEASED Mailman 2.1.3
by barry＠python.org 29 Sep '03

29 Sep '03

I have released Mailman 2.1.3, a bug fix release which also contains support for four new languages: Ukrainian, Serbian, Euskara (Basque), and Danish. This release also contains a fix for a cross-site scripting vulnerability in the 'create' cgi script, as well as improved performance of the bounce and outgoing queue runners. I recommend all sites running versions of the 2.1.x line upgrade to the new version. The full source tarball has been made available from the usual sites (although the gnu.org sites have not yet been updated). Sorry, there is no patch available, but you should be able to install 2.1.3 over your existing 2.1.x installation. See http://sourceforge.net/project/showfiles.php?group_id=103 for links to download all the patches and the source tarballs. After installing, be sure you restart your Mailman daemon by doing a "mailmanctl restart". See also: http://www.gnu.org/software/mailman http://www.list.org (not yet updated) http://mailman.sf.net Cheers, -Barry -------------------- snip snip -------------------- 2.1.3 (28-Sep-2003) Performance, Reliability, Security - Closed a cross-site scripting exploit in the create cgi script. - Improvements in the performance of the bounce processor. Now, instead of processing each bounce immediately (which can cause severe lock contention), bounce events are queued. Every 15 minutes by default, the queued bounce events are processed en masse, on a list-per-list basis, so that each list only needs to be locked once. - When some or all of a message's recipients have temporary delivery failures, the message is moved to a "retry" queue. This queue wakes up occasionally and moves the file back to the outgoing queue for attempted redelivery. This should fix most observed OutgoingRunner 100% cpu consumption, especially for bounces to local recipients when using the Postfix MTA. - Optional support for fsync()'ing qfile data after writing. Under some catastrophic system failures (e.g. power lose), it would be possible to lose messages because the data wasn't sync'd to disk. By setting SYNC_AFTER_WRITE to True in Mailman/Queue/Switchboard.py, you can force Mailman to fsync() queue files after flushing them. The benefits are debatable for most operating environments, and you must ensure that your Python has the os.fsync() function defined before enabling this feature (it isn't, even on all Unix-like operating systems). Internationalization - New languages Ukrainian, Serbian, Danish, Euskara/Basque. - Fixes to template lookup. Lists with local overriding templates would find the wrong template. - .mo files (for internationalization) are now generated at build time instead of coming as part of the source distribution. Documentation - A first draft of member documentation by Terri Oda. There is also a Japanese translation of this manual by Ikeda Soji. Archiver / Pipermail - In the configuration variables PUBLIC_EXTERNAL_ARCHIVER, and PRIVATE_EXTERNAL_ARCHIVER, %(hostname)s has been added to the list of allowable substitution variables. - The timezone is now taken into account when figuring the posting date for an article. Scripts / Cron - Fixes to cron/disabled for NotAMemberError crashes. - New script bin/show_qfiles which prints the contents of .pck message files. New script bin/discard which can be used to mass discard held messages. - Fixes to cron/mailpasswds to account for old password-less subscriptions. - bin/list_members has grown two new options: --invalid/-i prints only the addresses in the member database that are invalid (which could have snuck in via old releases); --unicode/-u prints addresses which are stored as Unicode objects instead of as normal strings. Miscellaneous - Fixes to problems in some configurations where Python wouldn't be able to find its standard library. - Fixes to the digest which could cause MIME-losing missing newlines when parts are scrubbed via the content filters. - In the News/Mail gateway admin page, the configuration variable nntp_host can now be a name:port pair. - When messages are pulled from NNTP, the member moderation checks are short-circuited. - email 2.5.4 is included. This fixes an RFC 2231 bug, among possibly others. - Fixed some extra spaces that could appear in the List-ID header. - Fixes to ensure that invalid email addresses can't be invited. - WEB_LINK_COLOR in Defaults.py/mm_cfg.py should now work. - Fixes so that shunted message file names actually match those logged in log/errors. - An improved pending action cookie generation algorithm has been added. - Fixes to the DSN bounce detector. - The usual additional u/i, internationalization, unicode, and other miscellaneous fixes.

2 1

Mailman-Developers September 2003