On Fri, 29 Mar 2002 11:15:59 +0100 fil <fil@rezo.net> wrote:
From what I see it needs about 5-6 seconds (on a 50000 subscribers' list) to process and register one bounce. So it really shouldn't process more than 4 in a row (ie in a minute) if we want the lockfile to be released before 30 secs. However that would mean processing 4 bounces per minute. If you have 50 000 bounces, that's 12 500 minutes... not good. I don't know what to suggest, except maybe to make a BounceRunner preprocessor that doesn't read or write the list files, but just extracts the bounce information and gets rid fast of the bounce files. Then for the "real" BounceRunner to process the bounce information already computed would be very fast.
Bounce processing really needs to be abstracted into a two pass process. Due to its current ad-hoc nature load and lookup expense for processing the membership roster is dominating the equation.
Suggest:
One pass does nothing but parse the incoming bounces for addresses and then adds them to a local/shared DB. An infrequently run cronjob (suggest no more than a couple times a day), then picks up this DB and handles the removals in a single fast locked process.
--
J C Lawrence
---------(*) Satan, oscillate my metallic sonatas.
claw@kanga.nu He lived as a devil, eh?
http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.