Fletcher Cocquyt wrote:
Pmap shows its the heap
god@irt-smtp-02:in 8:08pm 64 # pmap 24167 24167: /bin/python /opt/mailman-2.1.9/bin/qrunner --runner=IncomingRunner:5:8 08038000 64K rwx-- [ stack ] 08050000 940K r-x-- /usr/local/stow/Python-2.5.2/bin/python 0814A000 172K rwx-- /usr/local/stow/Python-2.5.2/bin/python 08175000 312388K rwx-- [ heap ] CF210000 64K rwx-- [ anon ] <--many small libs --> total 318300K
Whether its a leak or not - we need to understand why the heap is growing and put a limit on its growth to avoid exausting memory and swapping into oblivion...
At this point, I don't think it's a leak.
Your runners start out at about 9.5 MB. Most of your working runners grow to about the 20-40 MB range which I don't think is unusual for a site with some config.pck files approaching 1.4 MB.
Only your IncomingRunners seem to grow really big, and I think that is because you are seeing occasional, very large messages, or perhaps it has something to do with your custom spam filtering interface.
Does your MTA limit incoming message size?
In any case, I know you're reluctant to just let it run, but I think if you did let it run for a couple of days that the IncomingRunners wouldn't get any bigger than the 310 +- MB that you're already seeing after 3 hours, and the rest of the runners would remain in the 10 - 50 MB range.
I don't think you'll see a lot of paging activity of that 300+MB because I suspect that most of the time nothing is going on in most of that memory
None of the lists seem too big: god@irt-smtp-02:lists 8:24pm 73 # du -sk */*pck | sort -nr | head | awk '{print $1}' 1392 1240 1152 1096 912 720 464 168 136 112
Researching python heap alloaction....
You may also be interested in the FAQ article at <http://wiki.list.org/x/94A9>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan