[ python-Bugs-1376775 ] Memory leak in the email package

[SourceForge.net]

Bugs item #1376775, was opened at 2005-12-09 02:03
Message generated for change (Comment added) made by effbot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1376775&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: ken668 (ken668)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: Memory leak in the email package

Initial Comment:
memory leak in email.message_from_string. This is what
I did to create a leak.

You used the attached file, memleak.eml.

f = open("memleak.eml")
buffer = f.read()
f.close()

# now buffer has the email string
msg = email.message_from_string(buffer)
msg = None # this should free the memory but it doesn't

# The memory that is used in msg is not completely free


----------------------------------------------------------------------

>Comment By: Fredrik Lundh (effbot)
Date: 2005-12-20 20:03

Message:
Logged In: YES 
user_id=38376

Nope.

If you do msg=None, you'll release a reference to the
message object, and all direct, non-cyclic members.
Any cyclic references held by the message object (or
created by the email parser) will be removed at a later
time, by the garbage collector.

And even if you add an explicit gc.collect() call to
your program, that won't remove the contents of the
*buffer* variable.

(If you run the program in a loop *without* an explicit
gc.collect() call, the process will appear to grow for
a while, since you're creating objects faster than the
GC can remove them.  Add gc.collect() to the loop, and
the memory use will be rock solid.)

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-20 19:37

Message:
Logged In: YES 
user_id=1400763

May be you did not look at my test program more closely.
Please look at the statement msg=None. The should removed
all references of the object. The memory will definely grow
if you repeat the operation.  Why don't you run the
email.message_from_string in a loop and see whether the
memory grows. Thanks.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2005-12-20 17:44

Message:
Logged In: YES 
user_id=38376

This is not a leak (a leak means that the process will
grow if you *repeat* the operation, not that things are
unexpectedly left in the object memory).

And it's not a bug, either; the "copy" of the message
that's returned by get_objects() is the contents of the
buffer variable in the test program.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-15 07:44

Message:
Logged In: YES 
user_id=1400763

Yes, it is a bug. My mistake was when I gave the details, I
should have mentioned gc.get_objects(), not gc.garbage.
gc.get_objects() showed the leaks not gc.garbage.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-12-15 06:56

Message:
Logged In: YES 
user_id=33168

Do you still believe there is a problem or can this report
be closed?

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 17:00

Message:
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 16:59

Message:
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 16:53

Message:
Logged In: YES 
user_id=1400763

I added these three lines after the line "msg=None"

import gc
print "gc.collect()\n\n", gc.collect()
print "gc.garbage\n\n", gc.garbage

If you pipe the output of gc.garbage to a file, you will see
the email message you just sent are still be in the memory.

Everytime I call email.message_from_string, a copy of the
message will be kept in the memory even after I set the
returned value to None.

I tried the same thing with email package email-2.5.tar.gz,
that memory was freed right after I set the variable "msg"
to None.

Thanks

Ken



----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-12-11 21:13

Message:
Logged In: YES 
user_id=33168

What causes you to believe this is a memory leak?  I ran
this under valgrind and it doesn't report any leaks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1376775&group_id=5470