[Python-bugs-list] [ python-Bugs-507713 ] mem leak in imaplib

[noreply@sourceforge.net]

Bugs item #507713, was opened at 2002-01-23 13:28
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=507713&group_id=5470

Category: Python Library
Group: Python 2.1.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Scott Blomquist (scottdb)
>Assigned to: Piers Lauder (pierslauder)
Summary: mem leak in imaplib

Initial Comment:
When run in a multithreaded environment, the imaplib 
will leak memory if not run with the -O option.

Long running, multithreaded programs that we have 
which use the imaplib will run fine for a undefined 
period of time, then suddenly start to grow in size 
until they take as much mem as the system will give to 
them. Once they start to grow, they continue to grow 
at a pretty consistent rate.

Specifically:
If the -O option is not used, in the _log method 
starting on line 1024 in the imaplib class, the 
imaplib keeps the last 10 commands that are sent.

def _log(line):
    # Keep log of last `_cmd_log_len' interactions for 
debugging.
    if len(_cmd_log) == _cmd_log_len:
        del _cmd_log[0]
    _cmd_log.append((time.time(), line))

Unfortunately, in a multithreaded environment, 
eventually the len of the list will become larger than 
the _cmd_log_len, and since the test is for equality, 
rather than greater-than-equal-to, once the len of the 
_cmd_log gets larger than _cmd_log_len, nothing will 
ever be removed from the _cmd_log, and the list will 
grow without bound.

We added the following to test this hypothesis, we 
created a basic test which creates 40 threads. These 
threads sit in while 1 loops and create an imaplib and 
then issue the logout command. We also added the 
following debug to the method above:

    if len(_cmd_log) > 10:
        print 'command log len is:', len(_cmd_log)

We started the test, which ran fine, without leaking, 
for about 10 minutes, and without printing anything 
out. Somewhere around ten minutes, the process started 
to grow in size rapidly, and at the same time, the 
debug started printing out, and the size of the 
_cmd_log list did indeed grow very large, very fast. 
We repeated the test and the same symptoms occured, 
this time after only 5 minutes.


----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2002-02-14 22:09

Message:
Logged In: YES 
user_id=3066

It looks like the problem still exists in Python 2.1.2, 2.2,
and CVS.

I've attached a patch that I think solves this problem, but
this isn't easy for me to test.  Please check this.

Assigning to Piers Lauder since he knows more about this
module than I do.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=507713&group_id=5470