How to delete large numbers of held messages
We had a mail loop on one of our lists and ended up with over 51k messages in pending (held due to post not form a list member)
Trying to delete them from the web UI will take forever, as the largest pagination you can view is 200 at a time.
Tried to delete them from the Python interactive shell, but it looks like the API times out after a while.
Is there another way to delete all these unwanted held posts?
The problem comes after executing:
<block> reqs = list(adminlist.held) </block>
After a couple of minutes it comes back with:
<block> Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.9/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.9/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 718, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.9/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.9/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 107, in call response = request( File "/usr/lib/python3/dist-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 123, in held response, content = self._connection.call( File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 135, in call raise MailmanConnectionError( mailmanclient.restbase.connection.MailmanConnectionError: ('Could not connect to Mailman API: ', "ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))") </block>
On 4/21/22 04:32, lists--- via Mailman-Users wrote:
We had a mail loop on one of our lists and ended up with over 51k messages in pending (held due to post not form a list member)
Trying to delete them from the web UI will take forever, as the largest pagination you can view is 200 at a time.
Tried to delete them from the Python interactive shell, but it looks like the API times out after a while.
Is there another way to delete all these unwanted held posts?
You could try
bin/discard data/heldmsg-LISTNAME-*
If that times out, a bash script like
for file in ls data/heldmsg-LISTNAME-*; do
bin/discard $file
done
should do it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks for the reply Mark, where do I get the discard script from?
We are using Mailman3 but the only "discard" I can find belongs to postfix: /usr/lib/postfix/sbin/discard
There is a bin/discard on our Mailman 2.1 server but not on our Mailman 3.1 server
Thanks Chris
Another way to do this, if you have root privilege, is cd /var/lib/mailman/data (or words to that effect, depending on where "mailman" is) "rm" held* (or /usr/bin/rm held*, wherever "rm" is)
On 04/21/22 20:46, lists--- via Mailman-Users wrote:
Thanks for the reply Mark, where do I get the discard script from?
We are using Mailman3 but the only "discard" I can find belongs to postfix: /usr/lib/postfix/sbin/discard
There is a bin/discard on our Mailman 2.1 server but not on our Mailman 3.1 server
Thanks Chris
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
-- Jonathan Baron, Professor of Psychology, University of Pennsylvania Home page: https://www.sas.upenn.edu/~baron Founding Editor: Judgment and Decision Making (http://journal.sjdm.org)
On Thu, Apr 21, 2022 at 08:46:29PM -0000, lists--- via Mailman-Users wrote:
Thanks for the reply Mark, where do I get the discard script from?
We are using Mailman3 but the only "discard" I can find belongs to postfix: /usr/lib/postfix/sbin/discard
There is a bin/discard on our Mailman 2.1 server but not on our Mailman 3.1 server
You posted your thread to the Mailman 2.1 mailing list:
mailman-users@python.orgYou therefore got a reply appropriate to Mailman 2.1.
The Mailman 3 mailing list is at
mailman-users@mailman3.org(I'm sorry, but I don't know how to discard with Mailman 3. Maybe someone on the Mailman 3 mailing list can help you.)
Sam
On 4/21/22 13:46, lists--- via Mailman-Users wrote:
Thanks for the reply Mark, where do I get the discard script from?
We are using Mailman3 but the only "discard" I can find belongs to postfix: /usr/lib/postfix/sbin/discard
There is a bin/discard on our Mailman 2.1 server but not on our Mailman 3.1 server
Yes, my answer was for MM 2.1 because this is a MM 2.1 list. The MM 3 list is https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/
There is no equivalent to MM 2.1's discard script in MM 3 and held messages are in the database, not files in the data/ directory.
I suggest something like
$ bin/mailman shell -l list.example.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the list.example.com mailing list
>>> from mailman.app.moderator import handle_message
>>> requestdb = IListRequests(m)
>>> for id, type in requestdb.held_requests:
... if type == RequestType.held_message:
... handle_message(m, id, Action.discard)
...
>>> commit()
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (4)
-
Jon Baron -
lists＠g1fef.co.uk -
Mark Sapiro -
Sam Kuper