Adding backup to multi-server config
I have been running mailman 2.1 from three servers with NFS sharing for the past couple years, but due to a recent issue I did some digging and found the suggestions for clustering at http://mail.python.org/pipermail/mailman-users/2008-March/060753.html (the post by Mark Sapiro detailing how to set up qrunner slices). It turns out the problem I was having was user error, so apparently mailman *can* run on a low traffic site without issue... however since I now have the proper changes made for qrunner, I have a new question.
My network consists of three primary servers running postfix and mailman. These are each essentially redundant machines, and I am in the process of setting up a fourth machine in a VM for processing SMTP. What I am wondering about is if there would be a way to configure qrunner on this VM so that it only processes messages which have been in the queue for more than X minutes... basically a backup machine so that if the server processes one of the slices goes offline for an extended period, this backup server would make sure the message gets sent out.
Does this sound plausible? Unfortunately I know nothing about python, nor the specifics of mailman's code, so I'm not sure if this would be an easy change that someone could give me the code for... but any help is appreciated.
On 12/18/2012 11:15 AM, Jeff Taylor wrote:
Yes.
Assuming you have something like
QRUNNER_MESSAGE_IS_OLD_DELAY = minutes(2)
in mm_cfg.py for the backup machine and either no mention or something like
QRUNNER_MESSAGE_IS_OLD_DELAY = None
in mm_cfg.py/Defaults.py for the other machines, you could then patch Mailman/Queue/Switchboard.py as follows:
Find the section in the definition of the files() method that looks like
if ext <> extension:
continue
when, digest = filebase.split('+')
# Throw out any files which don't match our bitrange. BAW: test
# performance and end-cases of this algorithm. MAS: both
# comparisons need to be <= to get complete range.
if lower is None or (lower <= long(digest, 16) <= upper):
key = float(when)
while times.has_key(key):
key += DELTA
times[key] = filebaseand add the following
now = time.time()
age = now - float(when)
# Only process defined 'old' entries.
if not (
hasattr(mm_cfg, 'QRUNNER_MESSAGE_IS_OLD_DELAY') and
mm_cfg.QRUNNER_MESSAGE_IS_OLD_DELAY and
age > mm_cfg.QRUNNER_MESSAGE_IS_OLD_DELAY):
continuebetween
when, digest = filebase.split('+')and
# Throw out any files which don't match our bitrange. BAW: testAlso, Assuming you have patched mailmanctl as indicated at <http://mail.python.org/pipermail/mailman-users/2008-March/060753.html>, the mm_cfg.py QRUNNERS entries for the backup machine should look like
QRUNNERS = [ ('ArchRunner', 1,0,1), # messages for the archiver ... ]
i.e., the backup machine should be set up to be machine 0 of 1 machines. The actual number of slices for a given runner on this machine (the first '1' in '1,0,1' above) could be greater than 1, but you probably don't want that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Stephen J. Turnbull wrote:
No, it's just lack of careful typing and proof-reading on my part. (Also, a clue that what I wrote was inadequately tested at best).
As Stephen notes (thank you), this code block should be indented an additional four spaces.
So that the first 4 lines line up with the 'between' lines below and the remaining lines are also appropriately indented.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/18/2012 11:15 AM, Jeff Taylor wrote:
Yes.
Assuming you have something like
QRUNNER_MESSAGE_IS_OLD_DELAY = minutes(2)
in mm_cfg.py for the backup machine and either no mention or something like
QRUNNER_MESSAGE_IS_OLD_DELAY = None
in mm_cfg.py/Defaults.py for the other machines, you could then patch Mailman/Queue/Switchboard.py as follows:
Find the section in the definition of the files() method that looks like
if ext <> extension:
continue
when, digest = filebase.split('+')
# Throw out any files which don't match our bitrange. BAW: test
# performance and end-cases of this algorithm. MAS: both
# comparisons need to be <= to get complete range.
if lower is None or (lower <= long(digest, 16) <= upper):
key = float(when)
while times.has_key(key):
key += DELTA
times[key] = filebaseand add the following
now = time.time()
age = now - float(when)
# Only process defined 'old' entries.
if not (
hasattr(mm_cfg, 'QRUNNER_MESSAGE_IS_OLD_DELAY') and
mm_cfg.QRUNNER_MESSAGE_IS_OLD_DELAY and
age > mm_cfg.QRUNNER_MESSAGE_IS_OLD_DELAY):
continuebetween
when, digest = filebase.split('+')and
# Throw out any files which don't match our bitrange. BAW: testAlso, Assuming you have patched mailmanctl as indicated at <http://mail.python.org/pipermail/mailman-users/2008-March/060753.html>, the mm_cfg.py QRUNNERS entries for the backup machine should look like
QRUNNERS = [ ('ArchRunner', 1,0,1), # messages for the archiver ... ]
i.e., the backup machine should be set up to be machine 0 of 1 machines. The actual number of slices for a given runner on this machine (the first '1' in '1,0,1' above) could be greater than 1, but you probably don't want that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Stephen J. Turnbull wrote:
No, it's just lack of careful typing and proof-reading on my part. (Also, a clue that what I wrote was inadequately tested at best).
As Stephen notes (thank you), this code block should be indented an additional four spaces.
So that the first 4 lines line up with the 'between' lines below and the remaining lines are also appropriately indented.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Jeff Taylor -
Mark Sapiro -
Stephen J. Turnbull