[Mailman-Developers] BounceRunner eating up alot of memory

[Barry A. Warsaw]

>>>>> "F" == Fil  <fil@rezo.net> writes:

    F> after a few more tests, I am confronted to a problem: the
    F> BounceRunner seems to load the list in its memory when it
    F> receives a bounce (in order to process that bounce). But it
    F> does this: 1- regardless of whether it's going to need the list
    F> or not 2- without releasing the memory afterwards

    F> I can live with 1-, but 2- can be a problem if you have several
    F> big lists.

For 1) It's pretty much going to need the list object, but perhaps you
mean the list lock?  I'm not going to worry about that.

For 2) That's a valid point.  Please try the following patch
(untested).  I'll try to test it and if it looks good, I'll commit
this change tomorrow.
    
    F> All in all, I'm really starting to wonder why the list
    F> settings' data and the list members' data are mixed in one
    F> single pck file, and not in two (or more) separate files. When
    F> you need only to check the list settings, it would be so much
    F> faster to load a 1k file than to process a 77Mb file? In many
    F> situations it seems that would make Mailman much faster and
    F> have a smaller mem footprint!

I know it's bad, but at least the situation isn't any worse than it
was for MM2.0.  Fixing this is off the table for MM2.1, because it's
long overdue and a change like this would be too disruptive.  Fixing
this is (a large part of) what MM3.0 is all about.

-Barry

Index: BounceRunner.py
===================================================================
RCS file: /cvsroot/mailman/mailman/Mailman/Queue/BounceRunner.py,v
retrieving revision 2.9
diff -u -r2.9 BounceRunner.py
--- BounceRunner.py	7 Mar 2002 22:30:04 -0000	2.9
+++ BounceRunner.py	11 Mar 2002 04:51:36 -0000
@@ -31,6 +31,7 @@
 
 class BounceRunner(Runner):
     QDIR = mm_cfg.BOUNCEQUEUE_DIR
+    CACHELISTS = 0
 
     def _dispose(self, mlist, msg, msgdata):
         outq = get_switchboard(mm_cfg.OUTQUEUE_DIR)
Index: Runner.py
===================================================================
RCS file: /cvsroot/mailman/mailman/Mailman/Queue/Runner.py,v
retrieving revision 2.17
diff -u -r2.17 Runner.py
--- Runner.py	11 Mar 2002 04:21:33 -0000	2.17
+++ Runner.py	11 Mar 2002 04:51:37 -0000
@@ -34,9 +34,11 @@
 
 
 class Runner:
-    def __init__(self, slice=None, numslices=1, cachelists=1):
+    CACHELISTS = 1
+
+    def __init__(self, slice=None, numslices=1):
         self._kids = {}
-        self._cachelists = cachelists
+        self._cachelists = self.CACHELISTS
         # Create our own switchboard.  Don't use the switchboard cache because
         # we want to provide slice and numslice arguments.
         self._switchboard = Switchboard(self.QDIR, slice, numslices)