[Mailman-Users] Python process size grows 30x in 8 hours (memory

[Fletcher Cocquyt]

Interesting, the growth per python is limited to 50M by ulimit -v 50000, but
I'm seeing each one gradually take up that limit - then what ? (stay tuned!
- mailman fails to malloc?)

load averages:  0.14,  0.11,  0.11
10:14:43
120 processes: 119 sleeping, 1 on cpu
CPU states: 93.9% idle,  3.7% user,  2.4% kernel,  0.0% iowait,  0.0% swap
Memory: 1640M real, 646M free, 858M swap in use, 2436M swap free

   PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
  7566 mailman    1  59    0   49M   46M sleep    0:08  0.01% python
  7565 mailman    1  59    0   49M   46M sleep    0:09  0.02% python
  7557 mailman    1  59    0   48M   46M sleep    0:08  0.01% python
  7569 mailman    1  59    0   48M   45M sleep    0:09  0.01% python
  7552 mailman    1  59    0   47M   45M sleep    0:08  0.01% python
  7561 mailman    1  59    0   47M   45M sleep    0:08  0.06% python
  7570 mailman    1  59    0   47M   44M sleep    0:20  0.01% python
  7571 mailman    1  59    0   35M   31M sleep    0:05  0.01% python
  7551 mailman    1  59    0   35M   32M sleep    0:31  0.16% python
  7554 mailman    1  59    0   31M   27M sleep    0:37  0.35% python
  7568 mailman    1  59    0   31M   28M sleep    0:05  0.01% python
  7574 mailman    1  59    0   30M   28M sleep    0:05  0.37% python
  7560 mailman    1  59    0   30M   27M sleep    0:20  0.02% python
  7562 mailman    1  59    0   27M   25M sleep    0:06  0.03% python
  7558 mailman    1  59    0   26M   24M sleep    0:33  0.35% python


On 7/2/08 9:01 AM, "Fletcher Cocquyt" <fcocquyt at stanford.edu> wrote:

> Last night I added
> ulimit -v 50000 
> 
> To the /etc/init.d/mailman script and restarted
> 
> And I am seeing the processes stop growing at 49M - and so far no adverse
> affects.
> 
> I view this as a workaround until the underlying cause is determined...
> But it also allows me to bump my incoming runners from 8 to 16 and
> manage/enforce the overall memory footprint - I like it (so far)
> 
> 
> last pid:  8371;  load averages:  0.07,  0.11,  0.16
> 08:57:41
> 91 processes:  90 sleeping, 1 on cpu
> CPU states: 98.2% idle,  0.5% user,  1.3% kernel,  0.0% iowait,  0.0% swap
> Memory: 1640M real, 792M free, 697M swap in use, 2602M swap free
> 
>    PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
>   7565 mailman    1  59    0   49M   46M sleep    0:07  0.01% python
>   7571 mailman    1  59    0   35M   31M sleep    0:04  0.01% python
>   7551 mailman    1  59    0   35M   32M sleep    0:18  0.02% python
>   7554 mailman    1  59    0   31M   27M sleep    0:22  0.02% python
>   7568 mailman    1  59    0   31M   28M sleep    0:03  0.01% python
>   7574 mailman    1  59    0   30M   28M sleep    0:03  0.01% python
>   7570 mailman    1  59    0   30M   27M sleep    0:14  0.01% python
>   7560 mailman    1  59    0   30M   27M sleep    0:12  0.15% python
>   7562 mailman    1  59    0   27M   25M sleep    0:03  0.02% python
>   7566 mailman    1  59    0   27M   25M sleep    0:03  0.01% python
>   7569 mailman    1  59    0   27M   25M sleep    0:04  0.01% python
>   7561 mailman    1  59    0   27M   24M sleep    0:03  0.01% python
>   7558 mailman    1  59    0   26M   24M sleep    0:19  0.02% python
>   7572 mailman    1  59    0   26M   23M sleep    0:03  0.01% python
>   7567 mailman    1  59    0   26M   23M sleep    0:03  0.01% python
> 
> 
> On 7/1/08 11:43 PM, "Fletcher Cocquyt" <fcocquyt at stanford.edu> wrote:
> 
>> Hey thanks, I really appreciated the responsiveness, and helpful analysis by
>> the folks on this list
>> 
>> BTW I am experimenting with ulimit -v to limit the growth of mailman procs
>> (added it to the init script)
>> Will see what the growth looks like overnight and report back - thanks
>> 
>> 
>> 
>> On 7/1/08 10:21 PM, "Brad Knowles" <brad at shub-internet.org> wrote:
>> 
>>> On 7/1/08, Fletcher Cocquyt wrote:
>>> 
>>>>  Fletcher Cocquyt
>>>>  Senior Systems Administrator
>>>>  Information Resources and Technology (IRT)
>>>>  Stanford University School of Medicine
>>> 
>>> BTW, in case it hasn't come through yet -- I am very sensitive to
>>> your issues.  In my "real" life, I am currently employed as a Sr.
>>> System Administrator at the University of Texas at Austin, with about
>>> ~50,000 students and ~20,000 faculty and staff, and one of my jobs is
>>> helping out with both the mail system administration and the mailing
>>> list system administration.
>>> 
>>> So, just because I post messages quoting the current statistics we're
>>> seeing on python.org, that doesn't mean I'm not sensitive to the
>>> problems you're seeing.  All I'm saying is that we're not currently
>>> seeing them on python.org, so it may be a bit more difficult for us
>>> to directly answer your questions, although we'll certainly do
>>> everything we can do help.

-- 
Fletcher Cocquyt
Senior Systems Administrator
Information Resources and Technology (IRT)
Stanford University School of Medicine

Email: fcocquyt at stanford.edu
Phone: (650) 724-7485