[IronPython] IronPython with CherryPy through WSGI Memory issue

[Can Gencer]

On Fri, Apr 2, 2010 at 12:49 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 01/04/2010 23:46, Dino Viehland wrote:
>>>
>>> On Thu, Apr 1, 2010 at 4:17 PM, Can Gencer<cgencer at gmail.com>  wrote:
>>>
>>>>
>>>> On Thu, Apr 1, 2010 at 7:27 PM, Dino Viehland<dinov at microsoft.com>
>>>>  wrote:
>>>> I tried this, ExceptionHelpers.DynamicStackFrames.Count seems to be
>>>> constant at 1.. I'm not very familiar with windbg but I can certainly
>>>> learn more about it.. What should I be looking for?
>>>>
>>>
>>> You could try running it under CLRProfiler:
>>>
>>>
>>> http://www.microsoft.com/downloads/details.aspx?FamilyID=a362781c-3870-43be-
>>> 8926-862b40aa0cd0&DisplayLang=en
>>>
>>> http://msdn.microsoft.com/en-us/library/ms979205.aspx
>>>
>>> With it you can see which objects stick around after a GC, what's
>>> holding those objects, and where they're allocated. I'm not too
>>> familiar with its use yet, but I plan on doing that tomorrow.
>>>
>>
>> Yeah CLRProfiler is a little more friendly :)  I like windbg because it
>> gives
>> reasonable way of tracking who's keeping what alive and I find it a little
>> more
>> reliable than CLRProfiler.  What I usually do is attach to the process
>> after
>> letting it leak for a while and then:
>>
>>
>
> Kamil Dworakowski wrote an interesting blog entry on tracing memory leaks in
> IronPython apps with windbg:
>
> http://blog.kamil.dworakowski.name/2008/02/debugging-memory-problems-in-ironpython.html
>
> All the best,
>
> Michael Foord
>
>> .loadby sos mscorwks
>>        Or
>> .loadbys sos clr
>>
>> For CLR v2 and CLR v4 respectively.  Then you can do:
>>
>> !DumpHeap -stat
>>
>> Run for a while, repeat that, and look at what types of objects are
>> growing.
>> They should be at the end of the list because after running for a while
>> they're
>> growing the most.  Then you can do:
>>
>> !DumpHeap -mt<method_table>
>>
>> Where<method_table>  is the address for the type that !DumpHeap -stat gave
>> you.
>>
>> Then you can start picking objects at random and do:
>>
>> !GCRoot<object_addr>
>>
>> And you see who's keeping it alive.
>>
>> If the leaks big the places to look at pretty obvious.  If it's leaking
>> Python
>> objects defined in classes though you'll see those as "Unloaded Type".
>>
>>

I did some more digging around this, I tried using the wsgi.py found
in FePy instead (and modifying it to fit with my webserver) and the
results seem to be better, although not perfect.

There seems to be a whole bunch of objects that are referenced from this root:

ESP:550f3e8:Root:015c0150(Microsoft.Scripting.Ast.Block2)->
015c00fc(Microsoft.Scripting.Ast.ScopeN)->
015c00ec(System.Runtime.CompilerServices.TrueReadOnlyCollection`1[[Microsoft.Scripting.Ast.Expression,
Microsoft.Scripting.Core]])->
015c002c(System.Object[])->

What is this exactly?

Also I have a question about ScriptScopes. Do they get garbage
collected when a ScriptScope goes out of scope?