[IronPython] Please vote up this DLR memory leak bug
Dody Gunawinata
empirebuilder at gmail.com
Thu Feb 26 09:07:03 CET 2009
Microsoft.Scripting2.Core is a namespace copy of the IP source code -
based on IP 2.0 Beta 3 - just before one version that introduced
incompatibility with .Net 3.5
We are using ASP.Net with IronPython support based on July CTP 2007
(which I think ran on IP 2.0 Alpha X). The latest drop of IP for
ASP.Net was based on IP 2.0 Beta 4 which contained the .Net 3.5
incompatibility which makes in unusable for our application.
So my application has 2 copies of IP - one that runs the asp.net
dynamic support and the other that runs the scripting support (with 2
suffix in the namespace). It's a stupid situation but we rely a IP for
ASP.Net and that piece of technology has not been brought up to date
to a working state for .Net 3.5 framework- and our web productions
people rely on the software so it's full speed ahead regardless of the
technology limitation.
This memory leak might be gone if we have an up to date IP for ASP.Net
but one never know so here's the last 25 lines of the heap dump. I
hope this is useful.
79104368 3939 94536 System.Collections.ArrayList
7911f030 1708 95648 System.Reflection.Emit.DynamicMethod
7911f400 2999 95968 System.Reflection.Emit.SignatureHelper
109cbc9c 2516 100640 System.Linq.Expressions2.ConditionalExpression
1085c930 3181 101792 System.Scripting.SourceSpan
109c4594 3244 103808 System.Linq.Expressions2.ParameterExpression
10f63bcc 1237 110188
System.Collections.Generic.Dictionary`2+Entry[[System.Linq.Expressions2.Expression,
Microsoft.Scripting2.Core],[System.Linq.Expressions2.CompilerScope+Storage,
Microsoft.Scripting2.Core]][]
79135014 3 114968
System.Collections.Generic.Dictionary`2+Entry[[System.Int32,
mscorlib],[System.String, mscorlib]][]
10f60a2c 2260 117520
System.Collections.Generic.Dictionary`2[[System.Linq.Expressions2.Expression,
Microsoft.Scripting2.Core],[System.Object, mscorlib]]
109cfb1c 2736 120384 System.Linq.Expressions2.BinaryExpression
10f6145c 1743 125700
System.Collections.Generic.Dictionary`2+Entry[[System.Linq.Expressions2.Expression,
Microsoft.Scripting2.Core],[System.Object, mscorlib]][]
109c4e4c 3164 126560 System.Linq.Expressions2.ScopeExpression
109c4274 4142 132544 System.Linq.Expressions2.VariableExpression
109caf7c 3735 134460 System.Linq.Expressions2.UnaryExpression
10f5cd24 3762 135432
System.Linq.Expressions2.StackSpiller+ChildRewriter
106fa09c 40 138784
System.Collections.Generic.Dictionary`2+Entry[[Microsoft.Scripting.SymbolId,
Microsoft.Scripting],[Microsoft.Scripting.DynamicMixin+SlotInfo,
Microsoft.Scripting]][]
79111038 2585 144760 System.Reflection.RuntimePropertyInfo
79110680 3086 148128 System.Signature
106f63bc 4183 150588 Microsoft.Scripting.BuiltinFunction
109cbb3c 4327 155772 System.Linq.Expressions2.MemberExpression
791336d8 37 162960
System.Collections.Generic.Dictionary`2+Entry[[System.String,
mscorlib],[System.Int32, mscorlib]][]
109c320c 4726 170136 System.Linq.Expressions2.AssignmentExpression
7912f348 7171 172104
System.Collections.Generic.List`1[[System.Reflection.MethodInfo,
mscorlib]]
7911f538 1708 177632 System.Reflection.Emit.DynamicILGenerator
7910ea30 15685 188220 System.RuntimeMethodHandle
109c76cc 6304 201728 System.Linq.Expressions2.Block
7912d9bc 889 204576 System.Collections.Hashtable+bucket[]
790ff734 10742 214840 System.RuntimeType
79130510 11613 223528 System.RuntimeTypeHandle[]
791208b4 16407 262512 System.Reflection.Emit.GenericFieldInfo
10f552cc 18594 297504
System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Linq.Expressions2.Expression,
Microsoft.Scripting2.Core]]
109c4f1c 9671 386840 System.Linq.Expressions2.MethodCallExpression
7912dd40 646 432936 System.Char[]
109c7634 16161 517152 System.Linq.Expressions2.ConstantExpression
79109778 12835 718760 System.Reflection.RuntimeMethodInfo
79135538 1494 1167624 System.Reflection.Emit.__FixupData[]
7910dd38 124487 1493844 System.RuntimeTypeHandle
7912d7c0 17031 1846316 System.Int32[]
7912dae8 11218 1877012 System.Byte[]
790fd8c4 46924 2987964 System.String
7912d8f8 134460 5074708 System.Object[]
On Wed, Feb 25, 2009 at 10:51 PM, Dino Viehland <dinov at microsoft.com> wrote:
> Your approach should be perfectly fine (it's basically what CompiledCode was designed for) and it's not clear to me what would be leaking in this scenario.
>
> One suggestion I'd have for you is to download and install windbg (http://www.microsoft.com/whdc/DevTools/Debugging/default.mspx).
>
> Then attach to your process which is using all the memory and type in:
>
> .loadby sos mscorwks
> !DumpHeap -stat
>
> And send back the about the last 25 lines of that output. That'll give us the most common objects which are using memory in the GC heap and we might be able to track it down from there. If that's not sufficient maybe we can iterate and figure out what's going on.
>
>> -----Original Message-----
>> From: users-bounces at lists.ironpython.com [mailto:users-
>> bounces at lists.ironpython.com] On Behalf Of Dody Gunawinata
>> Sent: Wednesday, February 25, 2009 8:38 AM
>> To: Dino Viehland
>> Cc: Discussion of IronPython
>> Subject: Re: [IronPython] Please vote up this DLR memory leak bug
>>
>> What is the impact of caching CompiledCode into this "collectability"
>> issue?
>>
>> I am working on a CMS that expose functions implemented in IronPython
>> to a template engine. So I read a directory of 40 or 50 xx.py files,
>> load it up and compiled them. This is off course a costly operation to
>> do it in every single HTTP request (and SLOW) so I compiled the source
>> code to CompiledCode and cache them.
>>
>> Then for every HTTP request, I retrieve these CompiledCode's from the
>> Cache and call Execute(scope) to make the frozen code to be a real
>> function. So with X HTTP request, there will be X scope and (X * 40)
>> functions being generated at runtime.
>>
>> Now our memory consumption is huge and is getting bigger until IIS
>> gave up and recycle the process.
>>
>> Dody G.
>>
>>
>>
>>
>> > Just some comments on this as there's a few issues intertwined here:
>> >
>> > 1. Optimized code - this is currently the default and part of
>> the
>> > original problem. If you execute code in "optimized" mode we
>> generate an
>> > uncollectible type. In 2.6 I'm switching the default to be non-
>> optimized
>> > and I also am trying out a change that should reduce the amount of
>> leakage
>> > when we do this. That will make the out of the box experience much
>> better.
>> >
>>
>>
>> > 2. Multiple script runtimes - there's a bug in the DLR when you
>> > repeatedly create script runtimes that the rules aren't getting
>> collected.
>> > This is supposed to be fixed in 2.6 but I'm still seeing some leaks
>> in this
>> > scenario that I'll look into and fix. Back-porting this to 2.0 would
>> be
>> > quite difficult as a lot of things have changed but we could
>> certainly look
>> > at backporting a different fix to 2.0.
>> >
>>
>> > 3. There's a repro that defines "TestScriptOldClass" + n which
>> leaks.
>> > All class / function / member names in code get interned and this is
>> where
>> > this leak is coming from. If you remove the + n then everything
>> works. We
>> > may get rid of this interning at some point in the future but it's a
>> fairly
>> > significant change. I'm inclined to believe this one isn't as much
>> of an
>> > issue as usually you aren't compiling code with new names every
>> single
>> > type.
>> >
>>
>>
>>
>>
>> >
>> >
>> > From: users-bounces at lists.ironpython.com
>> > [mailto:users-bounces at lists.ironpython.com] On Behalf Of Dody
>> Gunawinata
>> > Sent: Sunday, February 22, 2009 2:10 PM
>> > To: Discussion of IronPython
>> > Subject: [IronPython] Please vote up this DLR memory leak bug
>> >
>> >
>> >
>> > There's only 5 votes so far.
>> >
>> >
>> >
>> >
>> http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=20399
>> >
>> >
>> >
>> > --
>> > nomadlife.org
>>
>>
>>
>> --
>> nomadlife.org
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
--
nomadlife.org
More information about the Ironpython-users
mailing list