[Mailman-Users] Mailman chewing up resources

[Mark Sapiro]

Geoff Shang wrote:
>
>The problem we're seeing is that if the list gets any way busy, the memory 
>usage of the python process that runs Mailman skyrockets, dragging the 
>system to a crawl.


Which Python process? mailmanctl should not be affected. Beyond that,
there are 8 qrunner processes. Are they all affected, or just one or
two?


>We're a little clueless as to how to debug this further, so any help would 
>be appreciated.


There is an issue that affects memory usage in the qrunners. They keep
a cache of list objects in memory to reduce disk IO. The cache is
supposed to free the space used by a list object when there are no
more references to that object, but it turns out there is a
self-reference in the list objects, so the cache simply grows until it
holds a copy of each list.

There are other issues in that large messages can cause the runners
that handle it to grow, and Python's memory management is such that
Python itself never gives freed memory back to the OS. Memory can be
freed within Python and it will be available for reuse within that
process, but it is not given back to the OS.

I recommend disabling the list cache within the qrunners. This was done
for the now defunct 2.2 branch, but has not been done on the 2.1
branch.

The attached Runner.patch.txt file contains a patch to do this. I
suggest you try the patch and see if that helps.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Runner.patch.txt
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20100312/24957034/attachment.txt>