[Mailman-Developers] traceback (still due to the BounceRunner)

[J C Lawrence]

On Fri, 29 Mar 2002 11:15:59 +0100 
fil  <fil@rezo.net> wrote:

> From what I see it needs about 5-6 seconds (on a 50000 subscribers'
> list) to process and register one bounce. So it really shouldn't
> process more than 4 in a row (ie in a minute) if we want the lockfile
> to be released before 30 secs. However that would mean processing 4
> bounces per minute. If you have 50 000 bounces, that's 12 500
> minutes... not good. I don't know what to suggest, except maybe to
> make a BounceRunner preprocessor that doesn't read or write the list
> files, but just extracts the bounce information and gets rid fast of
> the bounce files. Then for the "real" BounceRunner to process the
> bounce information already computed would be very fast.

Bounce processing really needs to be abstracted into a two pass process.
Due to its current ad-hoc nature load and lookup expense for processing
the membership roster is dominating the equation.  

Suggest: 

  One pass does nothing but parse the incoming bounces for addresses and
  then adds them to a local/shared DB.  An infrequently run cronjob
  (suggest no more than a couple times a day), then picks up this DB and
  handles the removals in a single fast locked process.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
claw@kanga.nu               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.