Thanks to Wes Morriston for giving me access to his Linux machine, I have diagnosed the problem he and Edward Marshall were having. I don't have a better solution than the one Wes came up with, so I'm posting this followup here to see if anybody has any ideas.

The problem in a nutshell: on some Linux boxes, the effective group id is not preserved across a popen() call.

Here's essentially the chain when a message is posted to a list:

1. The mail wrapper is invoked by the MDA.  Mailman's wrapper is a 
   setgid C program, that should change the egid to `mailman',
   then exec scripts/post.

2. scripts/post also has egid `mailman' and it goes about the
   business of checking that the message can be posted.
   Eventually, it is going to popen() scripts/deliver and start
   pumping it data.  scripts/deliver in turn popens
   scripts/contact_transport, but it doesn't matter because by
   then the damage is done.

Now, on both my Solaris 2.6 box and my RH5.1 box, scripts/deliver will inherit the egid of the invoking process, i.e. gid `mailman'. So by setting ~mailman/data to be g+w and group owned by mailman, everything's cool.

However on Wes' SuSE machine, scripts/deliver runs with its egid reset to the real gid. This breaks mailman because the real gid isn't `mailman' and thus the process does not have permission to write into the ~mailman/data directory. Because the process is owned by daemon, Wes' solution of chown'ing ~mailman/data to daemon got this working again.

I'm at a loss as to what the right solution is. Wes', while it works, doesn't seem ultimately right. But I have no idea how to force the egid of scripts/deliver to `mailman'. I don't believe you can have setgid scripts on Linux (I tried this, and no it did not work).

Another solution would be to wrap scripts/deliver and probably scripts/contact_transport in C setgid wrappers, but that seems like a PITA.

Is there some Linux option that we can exploit to allow this? Does anybody else have any better ideas? I am attaching a tarball of code that you can use to test whether your machine has the problem. To run this, compile wrapper.c, then chgrp it to some other group and chmod it to g+s. Also make sure wrapper-2.py is executable.

Now run wrapper. If you see that wrapper, wrapper-1 and wrapper-2 all have the same egid, then you're okay. If wrapper-2 reverts to the real gid, then you'll get nailed by this.

I'm guessing that because Stevens [APitUE] suggests that what we're doing may open up a security hole, some versions of Linux disallow this by reverting the egid on a popen().

-Barry