[Mailman-Users] What happened to my archive? Why isn't the archive process running?

[Mark Sapiro]

TGPlatt, WebMaster wrote:
>
>Could this be a group/owner-ship issue?


Yes. It took me a while, but I finally connected you with our exchanges
from last June/July :)


>It hadn't occurred to me to look for mailman's error log. Plus I had no idea
>where it was. I've seen it mentioned but I'd seen nothing in the docs that
>said where to find it. But with a bit of looking, I found it in
>/usr/local/mailman/logs/error
>
> 
>
>The error log that was saved when we made our final September 28 backup on
>the old server was last updated 9/25. The more I look the more this looks
>like an ownership issue to me. It may be I screwed up somewhere back in
>July. In our current mailman directory structure a smattering of files
>throughout the mailman directory tree seem to be owned by root / mailman
>now; whereas in the old backup everything seems to have been owned by
>mailman / mailman or www-data (Debian's default Apache user) / mailman. On
>9/28, the error log was owned by mailman / mailman. Indeed everything except
>mischief, subscribed and vet were owned by mailman / mailman back then.
>Those three files were different and were owned by "www-data" / mailman.
>Today in our running copy of mailman, all logs are owned by mailman/mailman
>except error which is owned by root / mailman and mischief, vet and
>subscribed which are owned by www-data / mailman.


There were undoubtedly problems on the old server because of SUExec
issues which I told you at the time was incompatible with Mailman's
security model since Mailman's CGI wrappers can't be SETGID under
SUExec and that's the whole point of the wrappers in the first place.

So ownership and permissions within Mailman have to be such that the
SUExec user can read and write.

But the qrunner processes also have to be able to read and write and
they will run as the mailman user:group.

Normally the owner doesn't matter because everything runs as group
mailman, but that may not be the case here.


>The reason I think this is an ownership issue is because when I look at the
>July - September error log I see lots of errors like this:
>
> 
>
>Jul 09 07:29:22 2008 (10592) Archive file access failure:
>
>        /usr/local/mailman/archives/private/ourlist.mbox/ourlist.mbox [Errno
>13] Permission denied:
>'/usr/local/mailman/archives/private/ourlist.mbox/ourlist.mbox'
>
>Jul 09 07:29:22 2008 (10592) Uncaught runner exception: [Errno 13]
>Permission denied: '/usr/local/mailman/archives/private/ourlist
>
>.mbox/ourlist.mbox'
>
> 
>
>.and when I check the files in that directory, they're owned by root as a
>member of the group mailman.


That should be OK because ArchRunner should be running as group mailman
and the files shoud be group writable.


<snip>
>
>Sep 25 21:09:31 2008 (16599) SHUNTING:
>1222394969.723012+6122e5f84bb97875a24698a159961936bed8802b
>
> 
>
>Sadly, when I look at the qfiles/shunt directory in the 9/28 backup, the
>oldest file there seems to be from 09-20-2008. So it looks to me like it was
>only keeping those shunt files 5 or 6 days before discarding them.


If you are running Mailman 2.1.11, there is a cron that runs daily and
by default it discards anything in qfiles/bad and qfiles/shunt older
than 7 days. From Defaults.py

# The length of time after which a qfiles/bad or qfiles/shunt file is
# considered to be stale.  Set to zero to disable culling of qfiles/bad
and
# qfiles/shunt entries.
BAD_SHUNT_STALE_AFTER = days(7)

# The pathname of a directory (searchable and writable by the Mailman
cron
# user) to which the culled qfiles/bad and qfiles/shunt entries will be
# moved.  Set to None to simply delete the culled entries.
BAD_SHUNT_ARCHIVE_DIRECTORY = None



>Here are the first and last error in the current error file:
>
> 
>
>Oct 18 10:43:54 2008 mailmanctl(25124): Site list is missing: mailman
>
>Oct 18 10:43:54 2008 (25124) Site list is missing: mailman
>
>Oct 18 18:38:42 2008 (14433) admin.py access for non-existent list: ourlist
>
>Oct 18 18:39:34 2008 (14458) admin.py access for non-existent list: ourlist
>
>Oct 18 20:34:08 2008 (19034) admin.py access for non-existent list: ourlist
>
>Oct 20 08:58:31 2008 (9822) admin.py access for non-existent list: ourlist 
>
>Oct 21 22:33:46 2008 (1431) Uncaught runner exception: [Errno 13] Permission
>den
>
>ied: '/usr/local/mailman/archives/private/ourlist/index.html'
>
>Oct 21 22:33:46 2008 (1431) Traceback (most recent call last):
>
>  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 120, in _oneloop
>
>    self._onefile(msg, msgdata)
>
> 
>
>...
>
> 
>
>Oct 28 10:09:30 2008 (2589) Uncaught runner exception: [Errno 13] Permission
>den
>
>ied: '/usr/local/mailman/archives/private/ourlist/index.html'
>
>Oct 28 10:09:30 2008 (2589) Traceback (most recent call last):
>
>  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 120, in _oneloop
>
>    self._onefile(msg, msgdata)
>
>  File "/usr/local/mailman/Mailman/Queue/Runner.py", line 191, in _onefile
>
>    keepqueued = self._dispose(mlist, msg, msgdata)
>
>  File "/usr/local/mailman/Mailman/Queue/ArchRunner.py", line 73, in
>_dispose
>
>    mlist.ArchiveMail(msg)
>
>  File "/usr/local/mailman/Mailman/Archiver/Archiver.py", line 217, in
>ArchiveMa
>
>il
>
>    h.close()
>
>  File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 324, in
>close
>
>    self.write_TOC()
>
>  File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1097, in
>write_T
>
>OC
>
>    toc = open(os.path.join(self.basedir, 'index.html'), 'w')
>
>IOError: [Errno 13] Permission denied:
>'/usr/local/mailman/archives/private/ourlist/index.html'
>
> 
>
>Oct 28 10:09:30 2008 (2589) SHUNTING:
>1225213769.6014299+1a6c02d7003773ec8b731f9
>
>23e4d1e0eb795d0b5


So you still have permissions issues.

You could start with bin/check_perms which should get them right except
if the web server is still SUExec, the web interface may not work.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan