[Mailman-Users] Python process size grows 30x in 8 hours (memory
mark at msapiro.net
Wed Jul 2 06:22:30 CEST 2008
Fletcher Cocquyt wrote:
>Pmap shows its the heap
>god at irt-smtp-02:in 8:08pm 64 # pmap 24167
>24167: /bin/python /opt/mailman-2.1.9/bin/qrunner
>08038000 64K rwx-- [ stack ]
>08050000 940K r-x-- /usr/local/stow/Python-2.5.2/bin/python
>0814A000 172K rwx-- /usr/local/stow/Python-2.5.2/bin/python
>08175000 312388K rwx-- [ heap ]
>CF210000 64K rwx-- [ anon ]
><--many small libs -->
> total 318300K
>Whether its a leak or not - we need to understand why the heap is growing
>and put a limit on its growth to avoid exausting memory and swapping into
At this point, I don't think it's a leak.
Your runners start out at about 9.5 MB. Most of your working runners
grow to about the 20-40 MB range which I don't think is unusual for a
site with some config.pck files approaching 1.4 MB.
Only your IncomingRunners seem to grow really big, and I think that is
because you are seeing occasional, very large messages, or perhaps it
has something to do with your custom spam filtering interface.
Does your MTA limit incoming message size?
In any case, I know you're reluctant to just let it run, but I think if
you did let it run for a couple of days that the IncomingRunners
wouldn't get any bigger than the 310 +- MB that you're already seeing
after 3 hours, and the rest of the runners would remain in the 10 - 50
I don't think you'll see a lot of paging activity of that 300+MB
because I suspect that most of the time nothing is going on in most of
>None of the lists seem too big:
>god at irt-smtp-02:lists 8:24pm 73 # du -sk */*pck | sort -nr | head | awk
>Researching python heap alloaction....
You may also be interested in the FAQ article at
Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
More information about the Mailman-Users