[Python-Dev] nested classes leaking in compiler

Amaury Forgeot d'Arc amauryfa at gmail.com
Sat Mar 29 21:27:03 CET 2008


Christian Heimes wrote:
> Georg Brandl schrieb:
>
> > Somehow I knew you'd be the one to find the problem :)
>
>  Agreed! :)
>
>  Amaury is really good in finding loose ends in the parser and AST code.
>  I still can't wrap my brain around the AST code but it can see the light
>  at the end of the code.

Actually the ast compiler is the part I know least in CPython code.

The approach I used is a bit brute-force, but it may work for other
reference leaks:
- First write a big leaking script:
    for x in xrange(100000): leaking_function()
- Since memory usage does not increase dramatically, it's only a
refcount problem - no new object every time.
- Modify the Py_INCREF macro so that it generates a breakpoint in the
debugger when the refcount is large:
  On Windows, it is something like:
     if (ob->refcount > 10000) { __asm int 3; }
  (with gdb I think you may signal SIGUSR1)
- carefully inspect the code each time the debugger stops.
  Fortunately, many functions can be skipped: PyDict_SetItem and other
bullet-proof parts of the code.
  The 20th break was the good one: a lack of symmetry between
Py_INCREF and Py_DECREF.

-- 
Amaury Forgeot d'Arc


More information about the Python-Dev mailing list