Expected mailq behavior/maintenance commands
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Howdy,
Not-so-longtime watcher—first time poster. I’ll start by saying how much that I appreciate this Mailman resource, as a noob it has proved very valuable for me.
I inherited the service owner role of Mailman recently and have been impressed at how well it works. Have not had any major pings until a few days ago, our monitoring system alerts if the queue exceeds 400 messages and it spiked to more than 600. It appears that a mass mailer produced a large number of bounces.
One of my co-workers moved some of the messages out of the queue and it rapidly went down to below 300—and then vacillated over the next few days between 225-260 messages.
It is now below 200 messages. I have not monitored the queue this closely before and it has raised some questions. 😊
I’ve looked through the documentation and community posts on the GNU website and I have not found specific information relating to what the expected mailq behavior/healthy number of messages is. I expect that this would vary by the environment and number of messages sent but am curious as to what would considered to be a normal/healthy threshold of queued messages.
Are there purge/maintenance commands that can be leveraged for normal maintenance or in the case above—more of a kill-switch purge?
Thanks in advance for any assistance.
Happy Friday!
John Lake
Application Security Analyst University of Oregon
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 10/29/21 8:18 AM, John Lake wrote:
It is now below 200 messages. I have not monitored the queue this closely before and it has raised some questions. 😊
I’ve looked through the documentation and community posts on the GNU website and I have not found specific information relating to what the expected mailq behavior/healthy number of messages is. I expect that this would vary by the environment and number of messages sent but am curious as to what would considered to be a normal/healthy threshold of queued messages.
Are you talking about MTA queues or Mailman's out queue? If you are talking about messages queued in the MTA, There's not much Mailman can do about that. If you are talking about messages queued in Mailman's out queue, normally this queue should be empty and if it has lots of queued messages, the queue is backlogged due to slow delivery to the MTA.
What to do about this depends on the MTA and some Mailman settings. There are many posts on this in the archives of this list. See https://mail.python.org/archives/search?mlist=mailman-users%40python.org&q=backlogged+out+queue
In particular, see https://mail.python.org/archives/list/mailman-users@python.org/message/CAKZU... for what to look for in Mailman's smtp log. If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking. You may want to set up an alternate port for Mailman to deliver to.
Are there purge/maintenance commands that can be leveraged for normal maintenance or in the case above—more of a kill-switch purge?
All you can do is manually move *.pck files out of Mailman's qfiles/out/ directory, but then those messages won't be delivered until you put them back.
If you have reason to believe some messages are unwanted, you can use Mailman's bin/show_qfiles command to inspect them and then delete unwanted ones.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Hi Mark,
Thanks very much for your fast response--I have a lot to learn and I really appreciate the help.
The qfiles/out directory is clear, the queued messages are in var/spool/mqueue.
We use Sendmail as our MTA , I'll ping our Unix admin on the setting that you referenced below:
"If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking."
I'll keep doing some digging, thanks again for the links and helpful information!
John
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Friday, October 29, 2021 3:49 PM To: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
On 10/29/21 8:18 AM, John Lake wrote:
Are you talking about MTA queues or Mailman's out queue? If you are talking about messages queued in the MTA, There's not much Mailman can do about that. If you are talking about messages queued in Mailman's out queue, normally this queue should be empty and if it has lots of queued messages, the queue is backlogged due to slow delivery to the MTA.
What to do about this depends on the MTA and some Mailman settings. There are many posts on this in the archives of this list. See https://urldefense.com/v3/__https://mail.python.org/archives/search?mlist=mailman-users*40python.org&q=backlogged*out*queue__;JSsr!!C5qS4YX3!T8Jt2kqtKunmlhTuLnmR_AgyvFvPCI-M5Bf0_THJBd-eqJBhL6jxN6StoUlCbrw2DUI$
In particular, see https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us... for what to look for in Mailman's smtp log. If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking. You may want to set up an alternate port for Mailman to deliver to.
Are there purge/maintenance commands that can be leveraged for normal maintenance or in the case above—more of a kill-switch purge?
All you can do is manually move *.pck files out of Mailman's qfiles/out/ directory, but then those messages won't be delivered until you put them back.
If you have reason to believe some messages are unwanted, you can use Mailman's bin/show_qfiles command to inspect them and then delete unwanted ones.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!T8Jt2kq... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!T8Jt2kq... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 10/30/21 9:27 AM, John Lake wrote:
Since Mailman's out queue is clear, the issue is with Sendmail's delivery of mail to the end user's MXs. There's not much Mailman can do about this.
Mailman does have settings that affect how the mail is delivered to Sendmail. For example, if VERP_DELIVERY_INTERVAL is set to 1, Mailman will sent a separate message to Sendmail for each recipient, and sentmail will in turn send separate messages to the recipient's MXs.
On the other hand, if VERP_DELIVERY_INTERVAL is set to 0, Mailman will deliver messages to Sendmail in chunks with multiple recipients per message. This in turn may allow Sendmail to deliver messages to recipient MXs with multiple recipients instead of a separate message per recipient which may improve Sendmail's delivery for these messages. This depends on Sendmail's behavior when it receives a message with multiple recipients in the same domain, and I don't know what that is or if it is configurable.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/8da339f04438d3fcc438e898cfe73c47.jpg?s=120&d=mm&r=g)
Mark Sapiro writes:
Note that personalization also affects this. For example, if the footer contains the subscriber's subscription page URL, the message has to be submitted separately for each subscriber.
I think those two settings are everything that affects the ability of the MTA to send to multiple recipients in one SMTP transaction. Mark?
Steve
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
We had a pronounced spike again in the /mqueue dir earlier this morning --it went up to more than 1200 messages and then 30 minutes later it was back down to 110.
Thanks again,
John
-----Original Message----- From: Stephen J. Turnbull <stephenjturnbull@gmail.com> Sent: Monday, November 1, 2021 12:03 AM To: Mark Sapiro <mark@msapiro.net> Cc: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
Mark Sapiro writes:
On the other hand, if VERP_DELIVERY_INTERVAL is set to 0, Mailman will > deliver messages to Sendmail in chunks with multiple recipients per > message.
Note that personalization also affects this. For example, if the footer contains the subscriber's subscription page URL, the message has to be submitted separately for each subscriber.
I think those two settings are everything that affects the ability of the MTA to send to multiple recipients in one SMTP transaction. Mark?
Steve
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!W1wGbe3... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!W1wGbe3... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 11/1/21 7:54 AM, John Lake wrote:
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
If it is not set in mm_cfg.py, it assumes the default value from Defaults.py.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Thanks Mark!
I checked Defaults.py and it appears that the VERP_DELIVERY_INTERVAL value setting is "0".
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Monday, November 1, 2021 9:26 AM To: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
On 11/1/21 7:54 AM, John Lake wrote:
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
If it is not set in mm_cfg.py, it assumes the default value from Defaults.py.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!QMkubdz... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!QMkubdz... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 11/1/21 10:04 AM, John Lake wrote:
Thanks Mark!
I checked Defaults.py and it appears that the VERP_DELIVERY_INTERVAL value setting is "0".
So Mailman is already doing everything it can to help Sendmail. Anything else has to be done in Sendmail.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 10/29/21 8:18 AM, John Lake wrote:
It is now below 200 messages. I have not monitored the queue this closely before and it has raised some questions. 😊
I’ve looked through the documentation and community posts on the GNU website and I have not found specific information relating to what the expected mailq behavior/healthy number of messages is. I expect that this would vary by the environment and number of messages sent but am curious as to what would considered to be a normal/healthy threshold of queued messages.
Are you talking about MTA queues or Mailman's out queue? If you are talking about messages queued in the MTA, There's not much Mailman can do about that. If you are talking about messages queued in Mailman's out queue, normally this queue should be empty and if it has lots of queued messages, the queue is backlogged due to slow delivery to the MTA.
What to do about this depends on the MTA and some Mailman settings. There are many posts on this in the archives of this list. See https://mail.python.org/archives/search?mlist=mailman-users%40python.org&q=backlogged+out+queue
In particular, see https://mail.python.org/archives/list/mailman-users@python.org/message/CAKZU... for what to look for in Mailman's smtp log. If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking. You may want to set up an alternate port for Mailman to deliver to.
Are there purge/maintenance commands that can be leveraged for normal maintenance or in the case above—more of a kill-switch purge?
All you can do is manually move *.pck files out of Mailman's qfiles/out/ directory, but then those messages won't be delivered until you put them back.
If you have reason to believe some messages are unwanted, you can use Mailman's bin/show_qfiles command to inspect them and then delete unwanted ones.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Hi Mark,
Thanks very much for your fast response--I have a lot to learn and I really appreciate the help.
The qfiles/out directory is clear, the queued messages are in var/spool/mqueue.
We use Sendmail as our MTA , I'll ping our Unix admin on the setting that you referenced below:
"If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking."
I'll keep doing some digging, thanks again for the links and helpful information!
John
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Friday, October 29, 2021 3:49 PM To: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
On 10/29/21 8:18 AM, John Lake wrote:
Are you talking about MTA queues or Mailman's out queue? If you are talking about messages queued in the MTA, There's not much Mailman can do about that. If you are talking about messages queued in Mailman's out queue, normally this queue should be empty and if it has lots of queued messages, the queue is backlogged due to slow delivery to the MTA.
What to do about this depends on the MTA and some Mailman settings. There are many posts on this in the archives of this list. See https://urldefense.com/v3/__https://mail.python.org/archives/search?mlist=mailman-users*40python.org&q=backlogged*out*queue__;JSsr!!C5qS4YX3!T8Jt2kqtKunmlhTuLnmR_AgyvFvPCI-M5Bf0_THJBd-eqJBhL6jxN6StoUlCbrw2DUI$
In particular, see https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us... for what to look for in Mailman's smtp log. If delivery is too slow, you may be able to address this in the MTA. You want the MTA to do no address verification at incoming SMTP time and minimal other checking. You may want to set up an alternate port for Mailman to deliver to.
Are there purge/maintenance commands that can be leveraged for normal maintenance or in the case above—more of a kill-switch purge?
All you can do is manually move *.pck files out of Mailman's qfiles/out/ directory, but then those messages won't be delivered until you put them back.
If you have reason to believe some messages are unwanted, you can use Mailman's bin/show_qfiles command to inspect them and then delete unwanted ones.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!T8Jt2kq... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!T8Jt2kq... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 10/30/21 9:27 AM, John Lake wrote:
Since Mailman's out queue is clear, the issue is with Sendmail's delivery of mail to the end user's MXs. There's not much Mailman can do about this.
Mailman does have settings that affect how the mail is delivered to Sendmail. For example, if VERP_DELIVERY_INTERVAL is set to 1, Mailman will sent a separate message to Sendmail for each recipient, and sentmail will in turn send separate messages to the recipient's MXs.
On the other hand, if VERP_DELIVERY_INTERVAL is set to 0, Mailman will deliver messages to Sendmail in chunks with multiple recipients per message. This in turn may allow Sendmail to deliver messages to recipient MXs with multiple recipients instead of a separate message per recipient which may improve Sendmail's delivery for these messages. This depends on Sendmail's behavior when it receives a message with multiple recipients in the same domain, and I don't know what that is or if it is configurable.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/8da339f04438d3fcc438e898cfe73c47.jpg?s=120&d=mm&r=g)
Mark Sapiro writes:
Note that personalization also affects this. For example, if the footer contains the subscriber's subscription page URL, the message has to be submitted separately for each subscriber.
I think those two settings are everything that affects the ability of the MTA to send to multiple recipients in one SMTP transaction. Mark?
Steve
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
We had a pronounced spike again in the /mqueue dir earlier this morning --it went up to more than 1200 messages and then 30 minutes later it was back down to 110.
Thanks again,
John
-----Original Message----- From: Stephen J. Turnbull <stephenjturnbull@gmail.com> Sent: Monday, November 1, 2021 12:03 AM To: Mark Sapiro <mark@msapiro.net> Cc: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
Mark Sapiro writes:
On the other hand, if VERP_DELIVERY_INTERVAL is set to 0, Mailman will > deliver messages to Sendmail in chunks with multiple recipients per > message.
Note that personalization also affects this. For example, if the footer contains the subscriber's subscription page URL, the message has to be submitted separately for each subscriber.
I think those two settings are everything that affects the ability of the MTA to send to multiple recipients in one SMTP transaction. Mark?
Steve
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!W1wGbe3... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!W1wGbe3... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 11/1/21 7:54 AM, John Lake wrote:
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
If it is not set in mm_cfg.py, it assumes the default value from Defaults.py.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/9acc110746bb8ca851798064d7c07321.jpg?s=120&d=mm&r=g)
Thanks Mark!
I checked Defaults.py and it appears that the VERP_DELIVERY_INTERVAL value setting is "0".
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Monday, November 1, 2021 9:26 AM To: mailman-users@python.org Subject: [Mailman-Users] Re: Expected mailq behavior/maintenance commands
On 11/1/21 7:54 AM, John Lake wrote:
Thanks Steve and Mark! Which file has the VERP_DELIVERY_INTERVAL setting? I did not see it in mm_cfg or the .py files in the MTA directory.
If it is not set in mm_cfg.py, it assumes the default value from Defaults.py.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/mailman-u... Mailman FAQ: https://urldefense.com/v3/__http://wiki.list.org/x/AgA3__;!!C5qS4YX3!QMkubdz... Security Policy: https://urldefense.com/v3/__http://wiki.list.org/x/QIA9__;!!C5qS4YX3!QMkubdz... Searchable Archives: https://urldefense.com/v3/__https://www.mail-archive.com/mailman-users@pytho... https://urldefense.com/v3/__https://mail.python.org/archives/list/mailman-us...
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 11/1/21 10:04 AM, John Lake wrote:
Thanks Mark!
I checked Defaults.py and it appears that the VERP_DELIVERY_INTERVAL value setting is "0".
So Mailman is already doing everything it can to help Sendmail. Anything else has to be done in Sendmail.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
John Lake
-
Mark Sapiro
-
Stephen J. Turnbull