[Mailman-Users] Python process size grows 30x in 8 hours (memory

[Mark Sapiro]

Fletcher Cocquyt wrote:

>Not finding a "leak" ref - save a irrelevant (for this runner issue) admindb


Nothing has been done in Mailman to fix any memory leaks. As far as I
know, nothing has been done to create any either.

If there is a leak, it is most likely in the underlying Python and not
a Mailman issue per se.

I am curious. You say this problem was exacerbated when you went from
one IncomingRunner to eight (sliced) IncomingRunners. The
IncomingRunner instances themselves should be processing fewer
messages each, and I would expect them to leak less. The other runners
are doing the same as before so I would expect them to be the same
unless by solving your 'in' queue backlog, you're just handling a
whole lot more messages.

Also, in an 8 hour period, I would expect that RetryRunner and
CommandRunner and, unless you are doing a lot of mail -> news
gatewaying, NewsRunner to have done virtually nothing.

In this snapshot

   PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
 10123 mailman    1  59    0  314M  311M sleep    1:57  0.02% python
 10131 mailman    1  59    0  310M  307M sleep    1:35  0.01% python
 10124 mailman    1  59    0  309M   78M sleep    0:45  0.10% python
 10134 mailman    1  59    0  307M   81M sleep    1:27  0.01% python
 10125 mailman    1  59    0  307M   79M sleep    0:42  0.01% python
 10133 mailman    1  59    0   44M   41M sleep    0:14  0.01% python
 10122 mailman    1  59    0   34M   30M sleep    0:43  0.39% python
 10127 mailman    1  59    0   31M   27M sleep    0:40  0.26% python
 10130 mailman    1  59    0   30M   26M sleep    0:15  0.03% python
 10129 mailman    1  59    0   28M   24M sleep    0:19  0.10% python
 10126 mailman    1  59    0   28M   25M sleep    1:07  0.59% python
 10132 mailman    1  59    0   27M   24M sleep    1:00  0.46% python
 10128 mailman    1  59    0   27M   24M sleep    0:16  0.01% python
 10151 mailman    1  59    0 9516K 3852K sleep    0:05  0.01% python
 10150 mailman    1  59    0 9500K 3764K sleep    0:00  0.00% python

Which processes correspond to which runners. And why are the two
processes that have apparently done the least the ones that have grown
the most.

In fact, why are none of these 15 PIDs the same as the ones from 8
hours earlier, or was that snapshot actually from after the above were
restarted?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan