Archiving problems
Hi folks,
I recently discovered that all my lists stopped archiving back in October. I have no clear idea as to what caused that, but I suspect that it may have occurred as part of some hardware and software upgrades I did around that time.
I'm still stuck back at Mailman 2.1.13 because the port system I use hasn't updated in ages and I haven't had time to tackle it myself.
I was able to bring most of the archives up to date manually, but one fairly active one with public archives is still only showing through October 2014. When I run that manually it shows that it's processing posts beyond the last one seen in the web-visible archives and updates the index for 1st quarter 2015, but still no visibility and the files in archive/public aren't updated. The files in archive/private *are* updated...
How do I make the public archives reflect what's in private?
Thanks.
On 01/25/2015 03:45 PM, Bill Christensen wrote:
I was able to bring most of the archives up to date manually, but one fairly active one with public archives is still only showing through October 2014. When I run that manually it shows that it's processing posts beyond the last one seen in the web-visible archives and updates the index for 1st quarter 2015, but still no visibility and the files in archive/public aren't updated. The files in archive/private *are* updated...
How do I make the public archives reflect what's in private?
At some point you moved or copied the archives and instead of copying the symlinks in archives/public as symlinks, you copied the target data which is never updated.
The ONLY thing that ever should be in archives/public/ is symlinks to the corresponding items in archives/private/.
The solution is simple:
rm -r archives/private/*
Mailman will automatically recreate the necessary archives/public/ symlinks for each publicly archived list the next time it does anything to the list.
On 01/25/2015 05:21 PM, Stephen J. Turnbull wrote:
Mark Sapiro writes:
The solution is simple:
rm -r archives/private/*
You mean "rm -r archives/public/*", don't you? As given it erases all the data, no?
Oh My yes! Absolutely! I hope nobody does the former. Thanks for catching this Steve.
On 1/25/15 7:27 PM, Mark Sapiro wrote:
On 01/25/2015 05:21 PM, Stephen J. Turnbull wrote:
Mark Sapiro writes:
The solution is simple:
rm -r archives/private/*
You mean "rm -r archives/public/*", don't you? As given it erases all the data, no?
Oh My yes! Absolutely! I hope nobody does the former. Thanks for catching this Steve.
Ok, I removed the public archives. I didn't rm -r, just copied them elsewhere in case this didn't work.
Our digest goes out at noon (and did). I assume that qualifies as "the next time it does anything to the list". Four new messages went through the list since then as well.
The public archive folder didn't rebuild.
Tail on the error log says:
mlist.Save()
File "/opt/local/share/mailman/Mailman/MailList.py", line 574, in Save
self.CheckHTMLArchiveDir()
File "/opt/local/share/mailman/Mailman/Archiver/Archiver.py", line 241, in CheckHTMLArchiveDir
makelink(privdir, pubdir)
File "/opt/local/share/mailman/Mailman/Archiver/Archiver.py", line 49, in makelink
os.symlink(old, new)
OSError: [Errno 2] No such file or directory
Your thoughts? FYI, this is on Mac 10.9.5, non-server version, if that makes any difference.
On 01/26/2015 06:00 PM, Bill Christensen wrote:
Ok, I removed the public archives. I didn't rm -r, just copied them elsewhere in case this didn't work.
...
The public archive folder didn't rebuild.
Tail on the error log says:
mlist.Save()
File "/opt/local/share/mailman/Mailman/MailList.py", line 574, in Save
self.CheckHTMLArchiveDir()File "/opt/local/share/mailman/Mailman/Archiver/Archiver.py", line 241, in CheckHTMLArchiveDir
makelink(privdir, pubdir)File "/opt/local/share/mailman/Mailman/Archiver/Archiver.py", line 49, in makelink
os.symlink(old, new)OSError: [Errno 2] No such file or directory
Your thoughts? FYI, this is on Mac 10.9.5, non-server version, if that makes any difference.
You weren't supposed to remove/move archives/private, just the subordinate listname directories.
On 01/26/2015 06:08 PM, Mark Sapiro wrote:
You weren't supposed to remove/move archives/private, just the subordinate listname directories.
And once again, I misspoke (mistyped). I again meant the subordinate listname directories under archives/public, not archives/private.
Mailman expects archives/public to exist, but the subordinates which are supposed to be symlinks into archives/private are checked and added/removed if necessary each time the list object is saved because the list's archive_private setting might have changed necessitating the addition or removal of symlinks.
On 1/26/15 9:32 PM, Mark Sapiro wrote:
On 01/26/2015 06:08 PM, Mark Sapiro wrote:
You weren't supposed to remove/move archives/private, just the subordinate listname directories.
And once again, I misspoke (mistyped). I again meant the subordinate listname directories under archives/public, not archives/private.
Mailman expects archives/public to exist, but the subordinates which are supposed to be symlinks into archives/private are checked and added/removed if necessary each time the list object is saved because the list's archive_private setting might have changed necessitating the addition or removal of symlinks.
Ok.
I put the archives/public folder back.
Toggling the archives setting on the publicly archived lists from public to private and back again created the symlinks. So far so good.
All the archives work when in private state. But I had to manually
change the owner of all the publicly available list archives to _www in
private in order for them to be readable. Otherwise they get a big
"Forbidden" message. Should *all* the lists be _www? Everything in
archives/private, or just the archive folders and not the mbox folders?
The archives/private folder itself is already _www.
Thanks for your continued tolerance and assistance.
On 01/26/2015 09:12 PM, Bill Christensen wrote:
All the archives work when in private state. But I had to manually change the owner of all the publicly available list archives to _www in private in order for them to be readable. Otherwise they get a big "Forbidden" message. Should *all* the lists be _www? Everything in archives/private, or just the archive folders and not the mbox folders? The archives/private folder itself is already _www.
Thanks for your continued tolerance and assistance.
For public archives to work, the web server needs to be able to search the archives/private directory that the archives/public/ symlinks point into.
Normally this only requires that the archives/private/ directory itself, not any subordinates, be either o+x or owned by the web server user. The subordinate LISTNAME and LISTNAME.mbox directories are normally rwxrwsr-x and Mailman's group so Mailman can read and write them and subordinates will be created in Mailman's group and the web server can read and search them.
It's only archives/private itself which is in Mailman's group and either rwxrws--x or if rwxrws---, owned by the web server user. Otherwise, ownership doesn't matter.
See http://www.list.org/mailman-install/node9.html.
On 1/26/15 11:27 PM, Mark Sapiro wrote:
On 01/26/2015 09:12 PM, Bill Christensen wrote:
All the archives work when in private state. But I had to manually change the owner of all the publicly available list archives to _www in private in order for them to be readable. Otherwise they get a big "Forbidden" message. Should *all* the lists be _www? Everything in archives/private, or just the archive folders and not the mbox folders? The archives/private folder itself is already _www.
Thanks for your continued tolerance and assistance.
For public archives to work, the web server needs to be able to search the archives/private directory that the archives/public/ symlinks point into.
Normally this only requires that the archives/private/ directory itself, not any subordinates, be either o+x or owned by the web server user. The subordinate LISTNAME and LISTNAME.mbox directories are normally rwxrwsr-x and Mailman's group so Mailman can read and write them and subordinates will be created in Mailman's group and the web server can read and search them.
It's only archives/private itself which is in Mailman's group and either rwxrws--x or if rwxrws---, owned by the web server user. Otherwise, ownership doesn't matter.
Well, I had it all working on Monday night.
I got a report today that someone was getting "Forbidden" again.
The owner of the list in question (and only that one list, not any of the other publicly archived lists - which have not seen any posts in the last two days) had changed back from _www to root. CHOWNing it back to _www again brings up the archive, but then it was only showing the last two days worth of archives (owner of those posts: _mailman, the rest were root). Rebuilding the archives with --wipe and running Check perms -f (which is already cron jobbed to run every night) made the rest of them visible again.
What do i need to do so that I don't have to jump these hoops daily?
Thanks yet again.
On 01/28/2015 07:15 PM, Bill Christensen wrote:
Well, I had it all working on Monday night.
I got a report today that someone was getting "Forbidden" again.
The owner of the list in question (and only that one list, not any of the other publicly archived lists - which have not seen any posts in the last two days) had changed back from _www to root. CHOWNing it back to _www again brings up the archive, but then it was only showing the last two days worth of archives (owner of those posts: _mailman, the rest were root). Rebuilding the archives with --wipe and running Check perms -f (which is already cron jobbed to run every night) made the rest of them visible again.
What do i need to do so that I don't have to jump these hoops daily?
Have you tried running Mailman's bin/check_perms?
Here's what you should have in the way of ownership and permissions. Group should be _mailman on everything. 'owner' doesn't matter except in the one case where I indicate _www. SETGID bits are important.
drwxrwsr-x owner _mailman /path/to/mailman
drwxrwsr-x owner _mailman /path/to/mailman/archives
drwxrwsr-x owner _mailman /path/to/mailman/archives/
drwxrwsr-x owner _mailman /path/to/mailman/archives/public and only symlinks in this directory
Either drwxrws--x owner _mailman /path/to/mailman/archives/private or drwxrws--- _www _mailman /path/to/mailman/archives/private
If you want to protect against access to private archives by local users of the machine, you want the latter. Otherwise the former is fine.
drwxrwsr-x owner _mailman /path/to/mailman/archives/private/*
And similarly for subordinate /path/to/mailman/archives/private/*/* directories.
-rw-rw-r-- owner _mailman for most files subordinate to /path/to/mailman/archives/private/*.
Again, SETGID bits are important.
participants (3)
-
Bill Christensen -
Mark Sapiro -
Stephen J. Turnbull