Phantom moderation pending requests & heldmsg files
A couple of weeks ago our Mailman 2.0 server crashed (a physical
problem, it wasn't Mailman's fault!) and we had to upgrade to 2.1 in a
real hurry. As a result, I had to copy the entire ~mailman/data
directory... which may not have been such a good idea:
[mailman@list ~/data]$ ls -al | wc -l 40054
There are a _lot_ of files prefixed with "heldmsg-" in the data
directory, as you can see. There are entries all the way back to 2005,
in fact! I realize that I'm going to have to deal with all those files
at some stage, but I currently have another problem. A user
administers a list called "ams-announce", and they are receiving
emails with the subject "The AMS-announce@list.bowdoin.edu mailing
list has 31 request(s) waiting for your consideration". However, when
they or I go to the admin interface, there are no pending requests.
The user has been getting the same message for more than a week now,
always with the same amount (31) of pending messages but there are
never actually any messages there for the admin to discard.
Oddly, there are no files prefixed with "heldmsg-ams-announce" in the /
home/mailman/data directory, however I did grep for the string "ams-
announce" in _all_ heldmsg files and there are many matches --
virtually all of it looks like spam. I suppose, then, that I have two
questions:
- how can I print how many moderation requests a list has pending, and
is there a way to flush the pending requests from the command-line?
(The FAQ entries deal with doing it via the web interface, before
2.1.5's feature of "discard all messages") - can I print a list of what lists have what files pending in the
format of heldmsg-$listname-$msgID, so that I can cross-reference the
files I have on the filesystem (many of which are really old) with
what the lists think need to be delievered?
This is with a 2.1.9 server.
Thanks,
Chris
Chris Waltham wrote:
A couple of weeks ago our Mailman 2.0 server crashed (a physical
problem, it wasn't Mailman's fault!) and we had to upgrade to 2.1 in a
real hurry. As a result, I had to copy the entire ~mailman/data
directory... which may not have been such a good idea:[mailman@list ~/data]$ ls -al | wc -l 40054
There are a _lot_ of files prefixed with "heldmsg-" in the data
directory, as you can see. There are entries all the way back to 2005,
in fact! I realize that I'm going to have to deal with all those files
at some stage, but I currently have another problem.
Depending on what else you did, these files may now be orphans, but see <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp>.
A user
administers a list called "ams-announce", and they are receiving
emails with the subject "The AMS-announce@list.bowdoin.edu mailing
list has 31 request(s) waiting for your consideration". However, when
they or I go to the admin interface, there are no pending requests.
The user has been getting the same message for more than a week now,
always with the same amount (31) of pending messages but there are
never actually any messages there for the admin to discard.Oddly, there are no files prefixed with "heldmsg-ams-announce" in the / home/mailman/data directory, however I did grep for the string "ams- announce" in _all_ heldmsg files and there are many matches --
virtually all of it looks like spam. I suppose, then, that I have two
questions:
- how can I print how many moderation requests a list has pending, and
is there a way to flush the pending requests from the command-line?
(The FAQ entries deal with doing it via the web interface, before
2.1.5's feature of "discard all messages")- can I print a list of what lists have what files pending in the
format of heldmsg-$listname-$msgID, so that I can cross-reference the
files I have on the filesystem (many of which are really old) with
what the lists think need to be delievered?
First of all, the bogus '31 ams-announce moderator request(s)' email is almost certainly not coming from your current Mailman 2.1.9 installation. I have never seen a report of a case like this where the email was not coming from some old/backup/test/whatever installation whose cron/checkdbs was still being run.
Second, the '31' has nothing to do with data/heldmsg-* files. These files have the messages themselves, but not the pending request information. The data for the 'moderator request(s)' email come from the lists/listname/request.pck file. If you want to see what's there, you can dump it with bin/dumpdb, and you can clear it by simply removing the file, although if you dump it first, I think you'll see it's devoid of requests just like the admindb interface says.
As far as getting a list of what messages are pending for each list in the form heldmsg-$listname-$msgID is concerned, there's nothing that will give you that in a nice list, but if you go to a specific list's request.pck file and dump it with bin/dumpdb, you'll see one entry like
'version': (0, 1)
and for each held message, entries like
id: (1, (time, sender, subject, reason, filename, msgdata))
and entries with 2 or 3 as the first item of the value which are held subscribe and unsubscribe requests.
In the held message entries, the key, id, is the number you call $msgID; time is the time in seconds that the message was held; sender, subject and reason are self explanitory; filename is the name of the held message file and msgdata is a dictionary of the message's metadata at the time it was held.
Note that if you have any request.db files, these are residue from your prior installation. I don't know if you did an actual upgrade of your 2.0 installation to 2.1.9, but if you did, the data from the request,db files should have been automatically migrated to the request.pck files by bin/update as part of the 2.1.9 install. If that isn't what happened, then I'm not sure what you're faced with in terms of recovering any held messages.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Apr 7, 2008, at 6:03 PM, Mark Sapiro wrote:
Chris Waltham wrote:
A couple of weeks ago our Mailman 2.0 server crashed (a physical problem, it wasn't Mailman's fault!) and we had to upgrade to 2.1
in a real hurry. As a result, I had to copy the entire ~mailman/data directory... which may not have been such a good idea:[mailman@list ~/data]$ ls -al | wc -l 40054
There are a _lot_ of files prefixed with "heldmsg-" in the data directory, as you can see. There are entries all the way back to
2005, in fact! I realize that I'm going to have to deal with all those
files at some stage, but I currently have another problem.Depending on what else you did, these files may now be orphans, but
see <http://www.python.org/cgi-bin/faqw-mm.py? req=show&file=faq04.074.htp>.
Thanks Mark, I'll check that out.
A user administers a list called "ams-announce", and they are receiving emails with the subject "The AMS-announce@list.bowdoin.edu mailing list has 31 request(s) waiting for your consideration". However, when they or I go to the admin interface, there are no pending requests. The user has been getting the same message for more than a week now, always with the same amount (31) of pending messages but there are never actually any messages there for the admin to discard.
Oddly, there are no files prefixed with "heldmsg-ams-announce" in
the / home/mailman/data directory, however I did grep for the string "ams- announce" in _all_ heldmsg files and there are many matches -- virtually all of it looks like spam. I suppose, then, that I have two questions:
- how can I print how many moderation requests a list has pending,
and is there a way to flush the pending requests from the command-line? (The FAQ entries deal with doing it via the web interface, before 2.1.5's feature of "discard all messages")- can I print a list of what lists have what files pending in the format of heldmsg-$listname-$msgID, so that I can cross-reference the files I have on the filesystem (many of which are really old) with what the lists think need to be delievered?
First of all, the bogus '31 ams-announce moderator request(s)' email
is almost certainly not coming from your current Mailman 2.1.9 installation. I have never seen a report of a case like this where the email was not coming from some old/backup/test/whatever installation whose cron/checkdbs was still being run.
Guess what? This report also fails your criteria... it was the old
server. :-( I thought I removed the NFS mount for mailman, but it was
in the automounter so the cron job could still successfully run!
That'll teach me for not just removing the user. And because the old
server had the same name as the new server and I wasn't a list admin,
I couldn't compare headers. Color me embarrassed...
Second, the '31' has nothing to do with data/heldmsg-* files. These files have the messages themselves, but not the pending request information. The data for the 'moderator request(s)' email come from the lists/listname/request.pck file. If you want to see what's there, you can dump it with bin/dumpdb, and you can clear it by simply removing the file, although if you dump it first, I think you'll see it's devoid of requests just like the admindb interface says.
Yep, I bet it is:
[mailman@list ~/lists/ams-announce]$ dumpdb request.pck [----- start pickle file -----] <----- start object 1 -----> {'version': (0, 1)} [----- end pickle file -----]
Whereas if I run the "dumpdb" command on a list I know has outstanding
requests, I see the request itself. Guess I know how this works, now!
As far as getting a list of what messages are pending for each list in the form heldmsg-$listname-$msgID is concerned, there's nothing that will give you that in a nice list, but if you go to a specific list's request.pck file and dump it with bin/dumpdb, you'll see one entry
like'version': (0, 1)
and for each held message, entries like
id: (1, (time, sender, subject, reason, filename, msgdata))
and entries with 2 or 3 as the first item of the value which are held subscribe and unsubscribe requests.
In the held message entries, the key, id, is the number you call $msgID; time is the time in seconds that the message was held; sender, subject and reason are self explanitory; filename is the name of the held message file and msgdata is a dictionary of the message's metadata at the time it was held.
Note that if you have any request.db files, these are residue from
your prior installation. I don't know if you did an actual upgrade of your 2.0 installation to 2.1.9, but if you did, the data from the request,db files should have been automatically migrated to the request.pck files by bin/update as part of the 2.1.9 install. If that isn't what happened, then I'm not sure what you're faced with in terms of recovering any held messages.
With the exception of 3-4 lists (out of 800+), I let the "make update"
command run so I presume that actually upgraded the lists. I think I
might just delete the holdmsg files en masse, I can't see why
(organizationally) I should need to keep them. I think I did remove
pending_requests.db (or whatever it's called) when I did the upgrade,
but I think it was empty anyway.
Chris
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Chris Waltham wrote:
With the exception of 3-4 lists (out of 800+), I let the "make update"
command run so I presume that actually upgraded the lists. I think I
might just delete the holdmsg files en masse, I can't see why
(organizationally) I should need to keep them.
When you delete these files, it is better to use bin/discard as discussed in <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp> rather than just rm. This is because bin/discard actually removes the held message from the requests.pck file (see below). However, if the heldmsg-* file is an orphan, it won't be removed by bin/discard, so you need to run bin/discard first and then rm any files left.
The problem with just removing the heldmsg-* file is if there are a large number of held messages for a single list, the admindb CGI can time out trying to create the held messages page, and just removing the heldmsg-* files without also removing the request.pck entries won't cure this time out.
I think I did remove
pending_requests.db (or whatever it's called) when I did the upgrade,
but I think it was empty anyway.
You may be thinking of the global data/pending.db file, but this is different. The items in that file if any were moved to the list specific pending.pck file, and they are the confirmation tokens and associated data for actions waiting confirmation, they are not the actions waiting moderator approval which were in requests.db, now requests.pck.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark,
Just to re-visit this...
On Apr 8, 2008, at 1:24 PM, Mark Sapiro wrote:
Chris Waltham wrote:
With the exception of 3-4 lists (out of 800+), I let the "make
update" command run so I presume that actually upgraded the lists. I think I might just delete the holdmsg files en masse, I can't see why (organizationally) I should need to keep them.When you delete these files, it is better to use bin/discard as discussed in <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp> rather than just rm. This is because bin/discard actually removes the held message from the requests.pck file (see below). However, if the heldmsg-* file is an orphan, it won't be removed by bin/discard, so you need to run bin/discard first and then rm any files left.
The problem with just removing the heldmsg-* file is if there are a large number of held messages for a single list, the admindb CGI can time out trying to create the held messages page, and just removing the heldmsg-* files without also removing the request.pck entries won't cure this time out.
I looked at my heldmsg files (all 40,000 of them :-)) and there are a
number patterns. Most heldmsg files are from a handful of lists (let's
call them baseball and football). If I do a dumpdb of the hockey
list's pending.pck file, this is the output:
[mailman@list ~/bin]$ ./dumpdb ../lists/hockey/pending.pck [----- start pickle file -----] <----- start object 1 -----> { '6183e43cf99b4a6850537d2a4837b26886bade2a': ('H', 5145), 'd22c46a0b5c704ec8be5c63a791e53adc23dc7fc': ('H', 5146), 'df7eae0e0a275c43b9c4bfb8d5f037df578e6cc2': ('H', 5147),
Now, the first line ends with the value "5145". Here is what I can
find on the filesystem:
[mailman@list ~/data]$ ls -al heldmsg-hockey* | grep 5145 -rw-rw-r-- 1 mailman mailman 3395 May 4 12:54 heldmsg- hockey-5145.pck
I rinsed & repeated that process with the other two entries in the
pending.pck file, i.e. 5146 and 5147. For both of those two, I can
find heldmsg-hockey-NNNN.pck (where NNNN equals the ID above).
However, I still have an awful lot of heldmsg-hockey files that
are .txt files, not .pck files:
[mailman@list ~/data]$ ls -al heldmsg-hockey*.txt | wc -l 3610
[mailman@list ~/data]$ ls -alt heldmsg-hockey*.txt | head -rw-rw-r-- 1 mailman mailman 2210 Mar 13 08:42 heldmsg- hockey-4920.txt -rw-rw-r-- 1 mailman mailman 3250 Mar 13 08:14 heldmsg- hockey-4919.txt -rw-rw-r-- 1 mailman mailman 3646 Mar 13 01:18 heldmsg- hockey-4918.txt -rw-rw-r-- 1 mailman mailman 2505 Mar 12 03:50 heldmsg- hockey-4917.txt -rw-rw-r-- 1 mailman mailman 2377 Mar 12 03:50 heldmsg- hockey-4916.txt -rw-rw-r-- 1 mailman mailman 2815 Mar 11 14:38 heldmsg- hockey-4915.txt -rw-rw-r-- 1 mailman mailman 2354 Mar 10 12:02 heldmsg- hockey-4914.txt -rw-rw-r-- 1 mailman mailman 2591 Mar 10 11:28 heldmsg- hockey-4913.txt -rw-rw-r-- 1 mailman mailman 2227 Mar 10 11:19 heldmsg- hockey-4912.txt -rw-rw-r-- 1 mailman mailman 2551 Mar 10 11:19 heldmsg- hockey-4911.txt
I think March 10 is around the time that I went from Mailman 2.0.12 to
2.1.9. Now my question is this: does Mailman 2.1.9 reference .txt
files for heldmessages, or are we only using .pck files now?
Thanks!
Chris
Chris Waltham wrote:
Hi Mark,
Just to re-visit this...
On Apr 8, 2008, at 1:24 PM, Mark Sapiro wrote:
Chris Waltham wrote:
With the exception of 3-4 lists (out of 800+), I let the "make
update" command run so I presume that actually upgraded the lists. I think I might just delete the holdmsg files en masse, I can't see why (organizationally) I should need to keep them.When you delete these files, it is better to use bin/discard as discussed in <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp> rather than just rm. This is because bin/discard actually removes the held message from the requests.pck file (see below). However, if the heldmsg-* file is an orphan, it won't be removed by bin/discard, so you need to run bin/discard first and then rm any files left.
The problem with just removing the heldmsg-* file is if there are a large number of held messages for a single list, the admindb CGI can time out trying to create the held messages page, and just removing the heldmsg-* files without also removing the request.pck entries won't cure this time out.
I looked at my heldmsg files (all 40,000 of them :-)) and there are a
number patterns. Most heldmsg files are from a handful of lists (let's
call them baseball and football). If I do a dumpdb of the hockey
list's pending.pck file, this is the output:[mailman@list ~/bin]$ ./dumpdb ../lists/hockey/pending.pck [----- start pickle file -----] <----- start object 1 -----> { '6183e43cf99b4a6850537d2a4837b26886bade2a': ('H', 5145), 'd22c46a0b5c704ec8be5c63a791e53adc23dc7fc': ('H', 5146), 'df7eae0e0a275c43b9c4bfb8d5f037df578e6cc2': ('H', 5147),
Note that the entries in lists/hockey/pending.pck are only the confirmation tokens and they expire after (default) 3 days, so there may normally be lots of held messages that are not in pending.pck.
The tokens in pending.pck are for the submitter of the held message to be able to cancel the post or for the admin/moderator to be able to cancel or approve the post by email. They have nothing to do with what's in the admindb interface which is controlled by what's in the request.pck file.
Now, the first line ends with the value "5145". Here is what I can
find on the filesystem:[mailman@list ~/data]$ ls -al heldmsg-hockey* | grep 5145 -rw-rw-r-- 1 mailman mailman 3395 May 4 12:54 heldmsg- hockey-5145.pck
I rinsed & repeated that process with the other two entries in the
pending.pck file, i.e. 5146 and 5147. For both of those two, I can
find heldmsg-hockey-NNNN.pck (where NNNN equals the ID above).
However, I still have an awful lot of heldmsg-hockey files that
are .txt files, not .pck files:
The fact that they are .txt files is not a problem per se. As long as there are entries in request.pck for them, they should appear in the admindb interface and can be approved if you wish, or rejected or discarded.
If they are not in request.pck and do not appear in the admindb interface, all you can do is remove the files or 'repost' the messages using bin/inject
[mailman@list ~/data]$ ls -al heldmsg-hockey*.txt | wc -l 3610
[mailman@list ~/data]$ ls -alt heldmsg-hockey*.txt | head -rw-rw-r-- 1 mailman mailman 2210 Mar 13 08:42 heldmsg- hockey-4920.txt -rw-rw-r-- 1 mailman mailman 3250 Mar 13 08:14 heldmsg- hockey-4919.txt -rw-rw-r-- 1 mailman mailman 3646 Mar 13 01:18 heldmsg- hockey-4918.txt -rw-rw-r-- 1 mailman mailman 2505 Mar 12 03:50 heldmsg- hockey-4917.txt -rw-rw-r-- 1 mailman mailman 2377 Mar 12 03:50 heldmsg- hockey-4916.txt -rw-rw-r-- 1 mailman mailman 2815 Mar 11 14:38 heldmsg- hockey-4915.txt -rw-rw-r-- 1 mailman mailman 2354 Mar 10 12:02 heldmsg- hockey-4914.txt -rw-rw-r-- 1 mailman mailman 2591 Mar 10 11:28 heldmsg- hockey-4913.txt -rw-rw-r-- 1 mailman mailman 2227 Mar 10 11:19 heldmsg- hockey-4912.txt -rw-rw-r-- 1 mailman mailman 2551 Mar 10 11:19 heldmsg- hockey-4911.txt
I think March 10 is around the time that I went from Mailman 2.0.12 to
2.1.9. Now my question is this: does Mailman 2.1.9 reference .txt
files for heldmessages, or are we only using .pck files now?
Theis is controlled by HOLD_MESSAGES_AS_PICKLES which defaults to Yes but can be set to No in mm_cfg.py.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On May 7, 2008, at 4:07 PM, Mark Sapiro wrote:
I looked at my heldmsg files (all 40,000 of them :-)) and there
are a number patterns. Most heldmsg files are from a handful of lists
(let's call them baseball and football). If I do a dumpdb of the hockey list's pending.pck file, this is the output:[mailman@list ~/bin]$ ./dumpdb ../lists/hockey/pending.pck [----- start pickle file -----] <----- start object 1 -----> { '6183e43cf99b4a6850537d2a4837b26886bade2a': ('H', 5145), 'd22c46a0b5c704ec8be5c63a791e53adc23dc7fc': ('H', 5146), 'df7eae0e0a275c43b9c4bfb8d5f037df578e6cc2': ('H', 5147),
Note that the entries in lists/hockey/pending.pck are only the confirmation tokens and they expire after (default) 3 days, so there may normally be lots of held messages that are not in pending.pck.
Okay.
The tokens in pending.pck are for the submitter of the held message to be able to cancel the post or for the admin/moderator to be able to cancel or approve the post by email. They have nothing to do with what's in the admindb interface which is controlled by what's in the request.pck file.
Okay. :-) That clears up things very well, thank you.
[mailman@list ~/lists/hockey]$ dumpdb request.pck | grep heldmsg | wc -l 3835
Now, the first line ends with the value "5145". Here is what I can find on the filesystem:
[mailman@list ~/data]$ ls -al heldmsg-hockey* | grep 5145 -rw-rw-r-- 1 mailman mailman 3395 May 4 12:54 heldmsg- hockey-5145.pck
I rinsed & repeated that process with the other two entries in the pending.pck file, i.e. 5146 and 5147. For both of those two, I can find heldmsg-hockey-NNNN.pck (where NNNN equals the ID above). However, I still have an awful lot of heldmsg-hockey files that are .txt files, not .pck files:
The fact that they are .txt files is not a problem per se. As long as there are entries in request.pck for them, they should appear in the admindb interface and can be approved if you wish, or rejected or discarded.
Now if I run bin/discard, that will discard all the messages in
request.pck, right? As per this FAQ entry: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp
Can you think of an easy way to discard all the messages in
request.pck _except those_ which still have tokens remaining in
pending.pck?
Chris
Chris Waltham wrote:
Can you think of an easy way to discard all the messages in
request.pck _except those_ which still have tokens remaining in
pending.pck?
If you're willing to accept that the only messages that have unexpired tokens left in pending.pck are those newer than PENDING_REQUEST_LIFE which defaults to 3 days, you can use the method suggested in the FAQ
$ cd ~mailman $ find data -name heldmsg-<listname>-\* -print | xargs bin/discard
but make it
$ cd ~mailman $ find data -name heldmsg-\* -mtime +3 -print | xargs bin/discard
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On May 7, 2008, at 4:59 PM, Mark Sapiro wrote:
Chris Waltham wrote:
Can you think of an easy way to discard all the messages in request.pck _except those_ which still have tokens remaining in pending.pck?
If you're willing to accept that the only messages that have unexpired tokens left in pending.pck are those newer than PENDING_REQUEST_LIFE which defaults to 3 days, you can use the method suggested in the FAQ
Just out of interest, how do I check and/or set that variable? I tried
looking through list configs for pending_request_life and didn't see
it; is it a global setting unless the list owner specifically changes
it? I looked in the FAQ and couldn't find the answer, either.
Thanks again,
Chris
On May 8, 2008, at 10:23 AM, Chris Waltham wrote:
On May 7, 2008, at 4:59 PM, Mark Sapiro wrote:
Chris Waltham wrote:
Can you think of an easy way to discard all the messages in request.pck _except those_ which still have tokens remaining in pending.pck?
If you're willing to accept that the only messages that have
unexpired tokens left in pending.pck are those newer than PENDING_REQUEST_LIFE which defaults to 3 days, you can use the method suggested in the FAQJust out of interest, how do I check and/or set that variable? I
tried looking through list configs for pending_request_life and
didn't see it; is it a global setting unless the list owner
specifically changes it? I looked in the FAQ and couldn't find the
answer, either.
Foolish me!
[mailman@list ~/Mailman]$ grep PENDING_REQUEST_LIFE Defaults.py PENDING_REQUEST_LIFE = days(3)
So sorry. :-)
Chris
participants (2)
-
Chris Waltham -
Mark Sapiro