Messages disappear after moderator approval
We are seeing messages get dropped after passing through moderation on every one of our mailing lists.
== Moderation fails when performed on a *separate* webserver == Our Mailman environment is split across two servers, front-end and back-end. The back-end server handles Postfix and the Mailman qrunners, while the front-end server hosts Apache and the Mailman CGI scripts for moderating lists. The two servers share an NFS mount between them that includes all the shared Mailman data. All normal mail flow is working correctly, but when a list moderator logs into the web frontend and approves a message, it disappears without a trace.
- Postfix smtpd receives the incoming message over SMTP, then
- Postfix smtpd delivers the message to /usr/lib/mailman/mail/mailman.
- Mailman marks writes to vette logfile (backend server) that message is held for approval.
- List moderator uses CGI web interface to mark the message as approved.
- Mailman writes an entry to vette logfile (on frontend server) saying held message approved.
- At this point, the .pck file related to the held message disappears, but nothing is delivered, and no further log entries are created.
== Moderation succeeds with web interface on the main Mailman server == Although we don't normally run the Mailman web interface on the back-end server (to reduce attack surface), I got it running for testing purposes. When we use the Mailman web interface on the backend server, the message gets delivered normally and we see these log entries as expected.
- smtp logfile updated with number of recipients and time for completion
- post logfile updated with list name, message ID, and "success".
== Background == The problem started after migrating the Mailman environment to new servers. It didn't crop up on it's own, it's most likely a result of some configuration error that we haven't caught yet. We're using:
- Scientific Linux 6.3 on both servers
- Python 2.6.6 on both servers
- Mailman 2.1.12 installed from OS packages on both servers
- selinux in Permissive mode on backend server
- selinux in Enforcing mode on frontend (web) server, but no log entries with type=AVC are being recorded. Furthermore, using setenforce 0 doesn't fix the problem.
I'm not sure where to look next. Is it supported to run the Mailman frontend on a different server? What are the common points of failure when running a Mailman setup this way?
-- Nic Waller System Administrator Phone 250-960-5919 Office ADM 3-2078B Information Technology Services University of Northern British Columbia
On 08/22/2013 03:46 PM, Nicholas Waller wrote:
We are seeing messages get dropped after passing through moderation on every one of our mailing lists.
== Moderation fails when performed on a *separate* webserver == Our Mailman environment is split across two servers, front-end and back-end. The back-end server handles Postfix and the Mailman qrunners, while the front-end server hosts Apache and the Mailman CGI scripts for moderating lists. The two servers share an NFS mount between them that includes all the shared Mailman data.
ALL? Exactly what directories are shared?
All normal mail flow is working correctly, but when a list moderator logs into the web frontend and approves a message, it disappears without a trace.
- Postfix smtpd receives the incoming message over SMTP, then 2. Postfix smtpd delivers the message to /usr/lib/mailman/mail/mailman.
- Mailman marks writes to vette logfile (backend server) that message is held for approval. 4. List moderator uses CGI web interface to mark the message as approved. 5. Mailman writes an entry to vette logfile (on frontend server) saying held message approved.
Implying the log files are not shared?
- At this point, the .pck file related to the held message disappears, but nothing is delivered, and no further log entries are created.
Is there anything in Mailman's 'error' log(s)?
Is the Mailman/ directory shared? If not, what is the value of VAR_PREFIX, QUEUE_DIR and INQUEUE_DIR in Mailman/Defaults.py or Mailman/mm_cfg.py on the frontend? I.e., are the queues NFS shared with the backend?
I'm guessing that the message gets put in the qfiles/in/ queue on the frontend and this is not the queue processed by IncomingRunner on the backend.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks Mark, your guess was correct. The messages were getting put in the qfiles/in/ queue on the frontend and never processed by IncomingRunner on the backend. My resolution was to change *QUEUE_DIR config lines on both servers to point to a shared NFS mount.
Now that you've clarified the problem, it seems like the misconfiguration should have been obvious. However, it wasn't obvious to me, and I had trouble finding guidance online about which directories should be shared. Is there already an FAQ addressing this issue, and if not, would it be reasonable to have one created?
For the sake of completeness, here are answers to your first questions.
- The only shared directories were DATA_DIR, LIST_DATA_DIR, VIRGINQUEUE_DIR, and both of the ARCHIVE_FILE_DIR's.
- Yes, implying the log files were not shared. Is it normal and safe to have multiple servers writing shared logfiles on an NFS mount?
- There were no entries in the "error" logs on either server.
- VAR_PREFIX = /var/lib/mailman
- QUEUE_DIR = /var/spool/mailman -- note this directory is not an NFS mount
- INQUEUE_DIR = /var/spool/mailman/in
-- Nic Waller System Administrator Phone 250-960-5919
-----Original Message----- From: Mailman-Users [mailto:mailman-users-bounces+nicholas.waller=unbc.ca@python.org] On Behalf Of Mark Sapiro Sent: Sunday, September 01, 2013 9:28 PM To: mailman-users@python.org Subject: Re: [Mailman-Users] Messages disappear after moderator approval
On 08/22/2013 03:46 PM, Nicholas Waller wrote:
We are seeing messages get dropped after passing through moderation on every one of our mailing lists.
== Moderation fails when performed on a *separate* webserver == Our Mailman environment is split across two servers, front-end and back-end. The back-end server handles Postfix and the Mailman qrunners, while the front-end server hosts Apache and the Mailman CGI scripts for moderating lists. The two servers share an NFS mount between them that includes all the shared Mailman data.
ALL? Exactly what directories are shared?
All normal mail flow is working correctly, but when a list moderator logs into the web frontend and approves a message, it disappears without a trace.
- Postfix smtpd receives the incoming message over SMTP, then 2. Postfix smtpd delivers the message to /usr/lib/mailman/mail/mailman.
- Mailman marks writes to vette logfile (backend server) that message is held for approval. 4. List moderator uses CGI web interface to mark the message as approved. 5. Mailman writes an entry to vette logfile (on frontend server) saying held message approved.
Implying the log files are not shared?
- At this point, the .pck file related to the held message disappears, but nothing is delivered, and no further log entries are created.
Is there anything in Mailman's 'error' log(s)?
Is the Mailman/ directory shared? If not, what is the value of VAR_PREFIX, QUEUE_DIR and INQUEUE_DIR in Mailman/Defaults.py or Mailman/mm_cfg.py on the frontend? I.e., are the queues NFS shared with the backend?
I'm guessing that the message gets put in the qfiles/in/ queue on the frontend and this is not the queue processed by IncomingRunner on the backend.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/nicholas.waller%40unbc. ca
On 09/03/2013 09:56 AM, Nicholas Waller wrote:
Now that you've clarified the problem, it seems like the misconfiguration should have been obvious. However, it wasn't obvious to me, and I had trouble finding guidance online about which directories should be shared. Is there already an FAQ addressing this issue, and if not, would it be reasonable to have one created?
My advice would be for standard GNU Mailman to share all the mutable data which is everything in var_prefix which is all of the directories archives/, data/, lists/, locks/, logs/, qfiles/ and spam/, however you have a Scientific Linux (Red Hat derivative) package, so see the FAQ at <http://wiki.list.org/x/KYCB> for how these map to your install.
There is a FAQ at <http://wiki.list.org/x/wgB0> that addresses this somewhat. It might need some additions. Do see all the links.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Nicholas Waller