[Python-bugs-list] [ python-Bugs-485139 ] mem leak in interpreter from syntax err
noreply@sourceforge.net
noreply@sourceforge.net
Fri, 07 Dec 2001 21:51:00 -0800
Bugs item #485139, was opened at 2001-11-24 10:45
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470
Category: Python Interpreter Core
Group: Python 2.2
Status: Open
Resolution: Wont Fix
>Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Tim Peters (tim_one)
Summary: mem leak in interpreter from syntax err
Initial Comment:
when a syntax error occurs in the interpreter, the
interpreter leaks
see the attached file for more info
----------------------------------------------------------------------
>Comment By: Tim Peters (tim_one)
Date: 2001-12-07 21:51
Message:
Logged In: YES
user_id=31435
Well, this just gets more mysterious the longer I stare at
it. I added a routine to compute the sum of all the
refcounts in the objects in the refchain, and fiddled
PyRun_InteractiveLoopFlags to print that sum after printing
_Py_RefTotal. That's the first mystery:
C:\Code\python\PCbuild>python_d
Adding parser accelerators ...
Done.
Python 2.2b2+ (#26, Dec 7 2001, 19:25:48) [MSC 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more
information.
>>>
[7004 refs] # _Py_RefTotal
[6540 sum] # sum of ob_refcnt across refchain objs
That is, right off the bat, _Py_RefTotal is 536 larger than
the sum of all refcounts in all known objects. I hoped
they'd be equal.
At least these stay in synch across lots of input
expressions:
>>> 1
1
[7006 refs]
[6542 sum]
Both went up by 2 there.
>>> []
[]
[7011 refs]
[6547 sum]
And both up by 5.
>>> ()
()
[7011 refs]
[6547 sum]
No change.
>>> {}
{}
[7011 refs]
[6547 sum]
No change.
>>> 2.3
2.2999999999999998
[7011 refs]
[6547 sum]
No change. Now comes another Mystery:
>>> def f(): pass
...
[7028 refs]
[6562 sum]
That is, _Py_RefTotal went up by 17, but the sum of all
refcounts only went up by 15. What's up with that?
Then both "leak" 1 for each repetition of a function defn:
>>> def f(): pass
...
[7029 refs]
[6563 sum]
>>> def f(): pass
...
[7030 refs]
[6564 sum]
>>> def f(): pass
...
[7031 refs]
[6565 sum]
>>>
That makes three mysteries. There's no other evidence of
an actual leak, though! In particular,
>>> import sys
[7036 refs]
[6570 sum]
>>> for i in range(10):
... def f(): pass
... print sys.gettotalrefcount()
...
7075
7075
7075
7075
7075
7075
7075
7075
7075
7075
[7037 refs]
[6571 sum]
>>>
That is, _PyRef_Total is *not* going up by 1 on each "def f
(): pass" if the def stmt is inside a loop.
Reducing the priority since this is such a pit.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-12-07 09:13
Message:
Logged In: YES
user_id=31435
Reopened and assigned to me. I don't care about a leak on
an uncaught syntax error, but something Neal said reminded
me of a problem that's repeatedly wasted my time: "enter
something at a debug-mode interactive prompt repeatedly"
often exhibits "and lose a refcount each time" behavior.
Like so:
>>> 1
1
[7026 refs]
>>> class C: pass
...
[7036 refs]
>>> class C: pass
...
[7037 refs]
>>> class C: pass
...
[7038 refs]
>>> class C: pass
...
[7039 refs]
>>>
Several times this has fooled me into looking for leaks
that don't exist.
>>> def f(): pass
...
[7056 refs]
>>> def f(): pass
...
[7057 refs]
>>> def f(): pass
...
[7058 refs]
>>>
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-07 08:46
Message:
Logged In: YES
user_id=6380
Neal writes:
---
I tried:
for i in range(1000):
try:
import t2
except SyntaxError:
print
But that doesn't leak. I think the reason it doesn't leak
is because
of the try/except. I opened the interpreter and
interactively
created syntax errors (=<return>, =<return>, ...) and it
leaked
for each error, not just once 874 bytes, instead of 38 for
1.
If there is a way to raise a syntax error without exitting
the interpreterr
I think it could blow up faster.
-----
My comment: if it only leaks when you don't catch it, that
doesn't bother me much, since that's the end of the process.
:-) So I'm closing this.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-06 19:11
Message:
Logged In: YES
user_id=6380
Neil, can you provide more evidence? I can't see this leak
in a loop, like this:
| while 1:
| try: compile("/", "", "exec")
| except SyntaxError: pass
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-11-27 12:05
Message:
Logged In: YES
user_id=31435
Assigned to Barry.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470