Mailman 3 April 2002 - Mailman-Developers

Re: [Mailman-Developers] Yet another weird-a$$ potential attack problem...
by J C Lawrence April 4, 2002

April 4, 2002

On Thu, 04 Apr 2002 11:07:30 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > Oh, man. This is my week for weird stuff. Yea and verily. > The question I'm bringing up is, I guess, is this something mailman > needs to worry about? Should it require that the returned token come > from the address being subscribed? (or does it already? In this case, > it came from a generic mailbot address @ that domain. An address > which, fwiw, bounces if you mail to it. Grimace.). I realize that the > "reply to confirm" is easy for users, but does it leave us open to > abuse in other ways? Should we make some cahnge to the process that > requires a person to do something? Mailman is becoming ubiquitous enough that I expect we'd do better to paranoid in advance than retroactively. > I don't have a good answer for any of this. I'm not even sure we > should consider it a problem. But since I've identified it as a > possible security flaw, I want to throw it out and let everyone chew > on it. Majordomo attempts to work around this by embedding a confirm token in the body of the message. To confirm the subscription you have to send that line back, unedited, to the list server. Happily, MD is fairly generous about line wrapping, quote prefixes etc. > On the other hand, IS it their fault for building a stupid tool? Or is > that no excuse to not protect ourselves from stupidity? There's no simple blanket answer as regards blame here: It is their fault for both installing and running a stupid auto-responder. Mailman Death Penalties are too good for them. It is our fault for writing and advocating a list server that is so trivially abused by stupid systems. At some point you have to drop back and say enough is enough and give up trying to stop stupid people I think thi Subject: business comes before that point however. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

RELEASED Mailman 2.0.9
by barry＠zope.com April 4, 2002

April 4, 2002

This message is to announce the release of GNU Mailman 2.0.9 which fixes a race condition that can cause mail to occasionally be lost, among other useful fixes. I recommend anybody running any version of Mailman 2.0.x to upgrade to 2.0.9, the latest stable release. Details of what's changed is included below. Note that this problem does not affect the Mailman 2.1 betas. As usual, I've made both full source tarballs and patches available. See http://sourceforge.net/project/showfiles.php?group_id=103 for links to download all the patches and the source tarball. If you decide to install the patches, please do read the release notes first: http://sourceforge.net/project/shownotes.php?release_id=63042 See also: http://www.gnu.org/software/mailman http://www.list.org http://mailman.sf.net Cheers, -Barry 2.0.9 (02-Apr-2002) - Closed a race condition which could, under rare circumstances, cause the occasional message to get lost. - HTML escape message excerpts and headers on the admindb page so JavaScript and other evil tags can't mess up the display. - Some additional Python 2.2 compatibility fixes. - Unlink the footer logos so as not to bug the python.org and gnu.org maintainers as much. :( - Fix a crash in the DSN bounce detection module, which could cause some bounce messages to remain in the queue. - Add the RFC-2822 mandated Date: header on internally generated outgoing messages. Not all MTAs add this field if missing (read: Qmail).

1 0

MM 2.0.9 not announced?
by Jason R. Mastaler April 4, 2002

April 4, 2002

I noticed MM 2.0.9 available for download on sourceforge, but I never saw an announcement for it. Checking the mailman-announce archives confirms this. Was this intentional? -- (http://tmda.sourceforge.net/)

4 8

2.0.x qrunner bug (bad one)
by Chuq Von Rospach April 4, 2002

April 4, 2002

I've just identified a pretty bad bug in mailman 2.0.x qrunner. It can cause messages to get lost, so, I almost hate to say this, Barry, but it might be time for a quick 2.0.9 patch. Given the changes to queuing in 2.1, I think this bug isn't relevant to the 2.1 tree. If you're hit by the bug, you'll see occasional reports in the error log like: Apr 02 06:48:03 2002 qrunner(12168): Traceback (most recent call last): Apr 02 06:48:03 2002 qrunner(12168): File "/export/home/mailman/cron/qrunner", line 282, in ? Apr 02 06:48:03 2002 qrunner(12168): kids = main(lock) Apr 02 06:48:03 2002 qrunner(12168): File "/export/home/mailman/cron/qrunner", line 202, in main Apr 02 06:48:03 2002 qrunner(12168): os.unlink(root+'.db') Apr 02 06:48:03 2002 qrunner(12168): OSError : [Errno 2] No such file or direct ory: '/export/home/mailman/qfiles/74a651f8eba5fce7ca800968bcd105b0bddb0c96.db' Apr 02 06:48:24 2002 admin(12193): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Here's the failure scenario. It can happen to any mailman system, because it's a timing hole, but a busy system will be more susceptible because of the number of files being created in qfiles. Qrunner, as it processes, opens the qfiles directory inode and walks it until it runs out of files (or times out) and exits. It starts at the top and simply goes to the end. That's why 2.0.x isn't FIFO through the queue, processing is based on location within the directory inode. The problem is when "wrapper post" is writing into qfile at the same time qrunner is processing and deleting. If qrunner hits a message AS IT IS BEING WRITTEN by "wrapper post", bad things can start happening. Scripts/post writes the .db file, then the message file. IF it happens to write the .db file into qfiles, and then qrunner tries to process it, qrunner will see no .msg, declare it orphaned, and delete it. Post then writes the .msg file, closes the .db file, and since qrunner unlinked it, the .db file then gets removed. You then get left with an orphaned .msg file. Unfortunately, qrunner and post don't lock each other off files, and sort of by definition, you can't have one lock the other out of the directory during processing. But that leaves this tiny window where they can get really confused, and bad things happen. You end up with messages that are accepted by post, and then disappear forever. I'm seeing this about once a day on my big server now, simply because of the message volumes. There are a couple of ways to fix this in the source. The "right" way would be to put file locking into post and qrunner, so qrunner can tell that post has the file open and skip it. Given that the queueing is being completely redone in 2.1, I'd suggest a second fix. In cron/qrunner, about 200 where the check for the missing file is, put in a quick check of the creation time for the file. If it's less than 5 minutes old, simply leave it alone and don't unlink it. Because it'll go and check again next trip through, it'll unlink real orphans, but it protects you from this tiny window of oopsie. Normally, I'd say "on to 2.1", but since this is a fairly serious "silent data loss" bug AND the fix is trivial, I think it might make sense to patch this and roll 2.0.9. At the least, I think a patch needs to be approved by Barry and released and made visible on the lists.org website. Please don't ask me how I found this. I'd have to kill you. But this is one of the more obscure bugs I've ever found... (grin) The window of opportunity to trigger it is immensely small. You need two programs to be simultaneously updating the same directory inode, and processing the same slot IN that inode, at the exact same time. We're talking about a latency of, as far as I can tell, 5-15 milliseconds every time post writes a message into qfile, but only if qrunner is actively processing. It looks like Barry took care (from reading the source) to avoid this kind of situation -- but didn't quite lock the window closed. -- Chuq Von Rospach, Architech chuqui(a)plaidworks.com -- http://www.chuqui.com/ The first rule of holes: If you are in one, stop digging.

11 30

More on the zombie brigade:
by Chuq Von Rospach April 3, 2002

April 3, 2002

Here's what I'm seeing. After qrunner is running for a while, I see: mailman 7303 7269 0 0:00 <defunct> mailman 7672 7269 0 0:00 <defunct> mailman 8008 7269 0 0:00 <defunct> mailman 8285 7269 0 0:00 <defunct> mailman 8412 7269 0 0:00 <defunct> mailman 8746 7269 0 0:00 <defunct> mailman 8833 7269 0 0:00 <defunct> mailman 8992 7269 0 0:00 <defunct> mailman 9139 7269 0 0:00 <defunct> mailman 9223 7269 0 0:00 <defunct> mailman 9361 7269 0 0:00 <defunct> mailman 7269 7268 3 11:08:00 ? 1:22 /usr/local/bin/python -S /export/home/mailman/cron/qrunner So these are dead, unreaped children owned by qrunner, barry. I'm wondering if it is the mail -> news stuff. I think I see one per message posted to a list, and pretty much all of my lists are gatewayed on that machine. But some sub-process is exiting and not being reaped. -- Chuq Von Rospach, Architech chuqui(a)plaidworks.com -- http://www.chuqui.com/ He doesn't have ulcers, but he's a carrier.

2 3

Some data on 2.0.x bounce processing.
by Chuq Von Rospach April 3, 2002

April 3, 2002

For various reasons, I had to run an address probe on one of my lists today to find a misbehaving but well-closed address. One thing I wanted to track was what percentage of the list comes back in some way. Of the 2000ish on the list, I ended up getting 38 bounces back (so far). Of those bounces, about 1/3 were being processed by the bounce system but not terminated. The other 2/3 were unprocessable for various reasons. I was able to manually deal with all of them, FWIW. Overall, that means that about 2% of the mail list was bouncing but not removed from the list. To me, with a fully automated system, that's not bad. It indicates it probably makes sense to do subscription validation probing on a manual basiis once in a while, and why we need to look at VERP and processing password reminders and the like, but I think it shows the bounce system works pretty well. I'll be curious what happens when I get 2.1 live and in production... Just FYI. Chuq

3 2

Module search path in Python/Mailman
by Peter Bengtson April 2, 2002

April 2, 2002

What is the recommended way to extend Python's module search path when using Mailman? When compiling Mailman (MacOS X), the following warning appears: > warning: install: modules installed to '/Users/mailman/pythonlib/', > which is not in Python's module search path (sys.path) -- you'll have > to change the search path yourself And when Mailman is run, a posting to the moderated "Mailman" list results in a message which keeps coming back hundreds of times: > Traceback (most recent call last): > File "/Users/mailman/cron/qrunner", line 89, in ? > from Mailman.Handlers import HandlerAPI > File "../Mailman/Handlers/HandlerAPI.py", line 26, in ? > ImportError: No module named pythonlib.StringIO So: should i use $PYTHONPATH dynamically? Or should i set up a symbolic link in site-packages plus a .pth file? Does this file need to contain all the subdirectories (email + japanese and further down?) Do I need to recompile or is this a dynamic process? Should I add the /Users/mailman/pythonlib directory somewhere in the ./configure invocation. There is absolutely no documentation about this in the 2.1 Mailman beta distribution. Searching the Python site reveals several strategies, but it is difficult to know the optimal one to use with Mailman. Grateful for any help. / Peter Bengtson

1 0

admindb
by Ron Jarrell April 1, 2002

April 1, 2002

When selecting the "forward to" option (and leaving it otherwise deferred) I get this: Traceback (most recent call last): File "/home/mailman/scripts/driver", line 82, in run_main main() File "/home/mailman/Mailman/Cgi/admindb.py", line 159, in main process_form(mlist, doc, cgidata) File "/home/mailman/Mailman/Cgi/admindb.py", line 644, in process_form forward, forwardaddr) File "/home/mailman/Mailman/ListAdmin.py", line 175, in HandleRequest forward, addr) File "/home/mailman/Mailman/ListAdmin.py", line 311, in __handlepost copy, lang) File "/home/mailman/Mailman/Message.py", line 158, in __init__ self.set_payload(text, charset) File "/home/mailman/pythonlib/email/Message.py", line 160, in set_payload self.set_charset(charset) File "/home/mailman/pythonlib/email/Message.py", line 198, in set_charset cte(self) File "/home/mailman/pythonlib/email/Encoders.py", line 75, in encode_7or8bit orig.encode('ascii') AttributeError: Message instance has no attribute 'encode' This was a fairly large note with a mime attachment.

2 2

Changing URLs in 2.1?
by Bill Bradford April 1, 2002

April 1, 2002

Anybody know how to change a 2.1 setup so that all the URLs on the admin/user web pages point to, say, /mailman2/ instead of /mailman? I'm trying to run a 2.1 install alongside a 2.0.8 install (to do testing), and all the URLs keep pointing back to stuff that doesent *exist* under /mailman. I've got the web aliases setup to go to /mailman2... Bill -- Bill Bradford mrbill(a)mrbill.net Austin, TX

2 1