Cron job to reset member bounce value
Can anyone tell me where the actual member bounce value is stored? I want to setup a cron job to reset the member bounce value to 0 for certain members whose mailboxes are only active when they have a satellite overhead to give them internet connectivity (about 8-10 hours each day). They keep getting disabled because their bounce value breaks the threshold value but this cannot be avoided due to their location and lack of internet. Thank you for your time and knowledge.
On 12/16/19 9:55 AM, Ethan Rudnitsky wrote:
Can anyone tell me where the actual member bounce value is stored? I want to setup a cron job to reset the member bounce value to 0 for certain members whose mailboxes are only active when they have a satellite overhead to give them internet connectivity (about 8-10 hours each day). They keep getting disabled because their bounce value breaks the threshold value but this cannot be avoided due to their location and lack of internet. Thank you for your time and knowledge.
The score is in the member's _BounceInfo object defined in Mailman/Bouncer.py. This object is retrieved and set by the list's getBounceInfo() and setBounceInfo() methods defined in the MemberAdaptor (Mailman/OldStyleMemberships.py by default).
However, See the script at https://www.msapiro.net/scripts/clear_bounce_info.py (mirrored at https://fog.ccsf.edu/~msapiro/scripts/clear_bounce_info.py) which will do what you want.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/16/2019 9:55 AM, Ethan Rudnitsky wrote:
Can anyone tell me where the actual member bounce value is stored? I want to setup a cron job to reset the member bounce value to 0 for certain members whose mailboxes are only active when they have a satellite overhead to give them internet connectivity (about 8-10 hours each day). They keep getting disabled because their bounce value breaks the threshold value but this cannot be avoided due to their location and lack of internet.
Sounds like a work-around to the real problem- that one of the MTA in the delivery chain is not properly retrying the message. In general, most MTAs are configured to retry for at least 24 hours and sometimes as long as 5 days before returning a fatal bounce, so if a destination server is off-line, the sending one will try again in an hour or so.
Which error code(s) are you seeing? Some 400 codes are designed to be retried later whereas 500's are immediately fatal.
Later,
z!
Mark and Carl,
Thank you both for your replies. So there are actually 2 issues causing bounces here... the first is the mailbox being offline at times, and the second is that it is getting messages bounced because of message size being over the limit. A few of these same accounts have strict 50KB message limits because they get their connectivity via Iridium modems, so when someone sends a message greater than 50KB it is bouncing. I have just utilized one of Mark's scripts and created a cron job in the crontab.in file at /usr/lib/mailman/cron that is formatted as follows: 0 8 * * * mailman /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
I then restarted the mailman service. Does this to be properly formatted for targeting only the 3 specific member accounts I want?
Thank you both for your time!
-Ethan
On Tue, Dec 17, 2019 at 10:39 AM Carl Zwanzig cpz@tuunq.com wrote:
Can anyone tell me where the actual member bounce value is stored? I want to setup a cron job to reset the member bounce value to 0 for certain members whose mailboxes are only active when they have a satellite overhead to give them internet connectivity (about 8-10 hours each day). They keep getting disabled because their bounce value breaks the
On 12/16/2019 9:55 AM, Ethan Rudnitsky wrote: threshold
value but this cannot be avoided due to their location and lack of internet.
Sounds like a work-around to the real problem- that one of the MTA in the delivery chain is not properly retrying the message. In general, most MTAs are configured to retry for at least 24 hours and sometimes as long as 5 days before returning a fatal bounce, so if a destination server is off-line, the sending one will try again in an hour or so.
Which error code(s) are you seeing? Some 400 codes are designed to be retried later whereas 500's are immediately fatal.
Later,
z!
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/erudnitsky%40gmail.com
On 12/17/2019 9:45 AM, Ethan Rudnitsky wrote:
So there are actually 2 issues causing bounces here... the first is the mailbox being offline at times, and the second is that it is getting messages bounced because of message size being over the limit.
Sounds like the -server- (aka 'MTA') is offline, and those messages should be retried; if they're not being retried, something is generally wrong and resetting the bounce counts is only masking that problem.
A few of these same accounts have strict 50KB message limits because they get their connectivity via Iridium modems, so when someone sends a message greater than 50KB it is bouncing.
Another option is to create a size-limited list and add that to the main list. Or just add a size limit to the existing list.... unless you're sending pictures or large files, most things can be said in under 50kb :) (and IMHO there are better ways of distributing larger files than email).
Later,
z!
On 12/17/19 9:45 AM, Ethan Rudnitsky wrote:
Mark and Carl,
Thank you both for your replies. So there are actually 2 issues causing bounces here... the first is the mailbox being offline at times, and the second is that it is getting messages bounced because of message size being over the limit. A few of these same accounts have strict 50KB message limits because they get their connectivity via Iridium modems, so when someone sends a message greater than 50KB it is bouncing. I have just utilized one of Mark's scripts and created a cron job in the crontab.in file at /usr/lib/mailman/cron that is formatted as follows: 0 8 * * * mailman /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
I then restarted the mailman service. Does this to be properly formatted for targeting only the 3 specific member accounts I want?
Carl raises some good points, but to answer your specific question, I don't know what if anything your distro's Mailman package does with /usr/lib/mailman/cron/crontab.in, but if it copies it automagically to /etc/cron.d/mailman, then putting the command there is OK.
The bottom line is if your Mailman cron jobs are in a system crontab such as /etc/cron.d/mailman, you want the line
0 8 * * * mailman /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
in that file. If your Mailman cron jobs are in the Mailman user's crontab /var/spool/cron/crontabs/mailman, you need to omit the username as in
0 8 * * * /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
You also need to have copied the script to /usr/lib/mailman/bin/reset_bounce.py
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark and Carl,
Thanks again for the replies.
Carl, the issue with the server/MTA bring offline is a known and unavoidable issue because of lack of satellite availability overhead for most of the day unfortunately. Regarding the mail size limit, those messages are ones that are really quite small (talking like 500kb) but still go over the 50kb limit of the mailbox. It is definitely a system that we are looking to replace but the geographic location and limited satellite coverage leave us with few options.
Mark, I have confirmed that the crontab.in does automatically copy /etc/cron.d/mailman, and /var/spool/cron/ is empty. I did also initially copy the reset_bounce.py to the directory you mentioned, so thank you for your diligence in checking, sir!
I guess we'll see how it behaves and how it goes. Thanks again; I really appreciate it.
-Ethan
On Tue, Dec 17, 2019 at 11:14 AM Mark Sapiro mark@msapiro.net wrote:
On 12/17/19 9:45 AM, Ethan Rudnitsky wrote:
Mark and Carl,
Thank you both for your replies. So there are actually 2 issues causing bounces here... the first is the mailbox being offline at times, and the second is that it is getting messages bounced because of message size being over the limit. A few of these same accounts have strict 50KB message limits because they get their connectivity via Iridium modems, so when someone sends a message greater than 50KB it is bouncing. I have just utilized one of Mark's scripts and created a cron job in the crontab.in file at /usr/lib/mailman/cron that is formatted as follows: 0 8 * * * mailman /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
I then restarted the mailman service. Does this to be properly formatted for targeting only the 3 specific member accounts I want?
Carl raises some good points, but to answer your specific question, I don't know what if anything your distro's Mailman package does with /usr/lib/mailman/cron/crontab.in, but if it copies it automagically to /etc/cron.d/mailman, then putting the command there is OK.
The bottom line is if your Mailman cron jobs are in a system crontab such as /etc/cron.d/mailman, you want the line
0 8 * * * mailman /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
in that file. If your Mailman cron jobs are in the Mailman user's crontab /var/spool/cron/crontabs/mailman, you need to omit the username as in
0 8 * * * /usr/lib/mailman/bin/withlist -a -r reset_bounce -- -u manager@xxxxxxxx.xxx.xxx -u comms@xxxxxxxx.xxx.xxx -u cargo@xxxxxxxx.xxx.xxx
You also need to have copied the script to /usr/lib/mailman/bin/reset_bounce.py
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/erudnitsky%40gmail.com
On 12/17/2019 10:24 AM, Ethan Rudnitsky wrote:
the issue with the server/MTA bring offline is a known and unavoidable issue because of lack of satellite availability overhead for most of the day unfortunately.
That is an entirely avoidable issue. When a destination MTA is offline/busy (which happens all the time and is accounted-for in SMTP), the sending MTA usually queues the messages and retries in 15/30/120 minutes, and usually keeps that up for at least 24 hours. If the sending MTA is giving up immediately and returning a failure, IMHO it's misconfigured (knowing the exact error code returned is important, here).
That said, the operator(s) of that server may not be interested in fixing the problem.
Do the satellite-served list members receive every message eventually or are they content to only receive ones sent during their connectivity window? If the -do- get all the messages eventually then they are being retried and the temporary failure is being counted as a bounce.
Later,
z!
Carl, my apologies as I see what you are saying now. The server is indeed retrying instead of giving up right away, so they do receive the messages eventually but the retries are being counted as a bounce.
Mark, when I run your get_bounce_info script I do get an output of the current score for each member in each list, but when I manually run "./withlist -a -r reset_bounce --" it returns "Importing reset_bounce..." [newline] "Running reset_bounce.reset_bounce()... and it then lists all of the lists on each line as it goes through them and appends each with an (unlocked) and then says Finalizing. However, if I run the get_bounce_info again, nothing is reset to 0 bounces, it lists the same thing as before. So I guess I need clarification, will this only reset bounce counts for members who are already/currently locked, or is it supposed to reset the count back to 0 even if they are unlocked and below the threshold?
Thank you, sir.
On Tue, Dec 17, 2019 at 11:34 AM Carl Zwanzig cpz@tuunq.com wrote:
On 12/17/2019 10:24 AM, Ethan Rudnitsky wrote:
the issue with the server/MTA bring offline is a known and unavoidable issue because of lack of satellite availability overhead for most of the day unfortunately.
That is an entirely avoidable issue. When a destination MTA is offline/busy (which happens all the time and is accounted-for in SMTP), the sending MTA usually queues the messages and retries in 15/30/120 minutes, and usually keeps that up for at least 24 hours. If the sending MTA is giving up immediately and returning a failure, IMHO it's misconfigured (knowing the exact error code returned is important, here).
That said, the operator(s) of that server may not be interested in fixing the problem.
Do the satellite-served list members receive every message eventually or are they content to only receive ones sent during their connectivity window? If the -do- get all the messages eventually then they are being retried and the temporary failure is being counted as a bounce.
Later,
z!
On 12/17/19 1:28 PM, Ethan Rudnitsky wrote:
Carl, my apologies as I see what you are saying now. The server is indeed retrying instead of giving up right away, so they do receive the messages eventually but the retries are being counted as a bounce.
If the message delayed DSNs or whatever they are are being scored as bounces by Mailman, this is a bug. Please send me one of the bounce messages you receive that increments the score or disables the member. You get these by setting bounce_notify_owner_on_disable and if your Mailman is new enough (>=2.1.19) bounce_notify_owner_on_bounce_increment on the Bounce Processing page in the web admin UI.
Send the full, raw message as attached to the notice or the full, raw notice, and I will try to update the recognizer to recognize it as non-fatal.
Mark, when I run your get_bounce_info script I do get an output of the current score for each member in each list, but when I manually run "./withlist -a -r reset_bounce --" it returns "Importing reset_bounce..." [newline] "Running reset_bounce.reset_bounce()... and it then lists all of the lists on each line as it goes through them and appends each with an (unlocked) and then says Finalizing. However, if I run the get_bounce_info again, nothing is reset to 0 bounces, it lists the same thing as before. So I guess I need clarification, will this only reset bounce counts for members who are already/currently locked, or is it supposed to reset the count back to 0 even if they are unlocked and below the threshold?
You seem to have picked the wrong script, and I didn't notice in my follow-up. The reset_bounce script will re-enable delivery for all members with delivery already disabled by bounce. It will not reset scores for members not yet disabled. This is probably not what you want. The script you want is one that resets the current bounce info and score for members with current bounce info but not yet disabled. That script is https://www.msapiro.net/scripts/clear_bounce_info.py (mirrored at https://fog.ccsf.edu/~msapiro/scripts/clear_bounce_info.py.
All of the
Importing reset_bounce... Running reset_bounce.reset_bounce()... and it then lists all of the lists on each line as it goes through them and appends each with an (unlocked) and then says Finalizing
output comes from withlist as it imports the script and runs it against the lists.
The scripts themselves produce no output unless you specify the -v option (after the --) in which case they just print a count of the number of users affected.
The -v output for clear_bounce info is
List <listname>: Cleared bounce info for <nn> members.
and for reset_bounce it is
List <listname>: Reset <nn> bouncing members.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Carl Zwanzig -
Ethan Rudnitsky -
Mark Sapiro