<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
text-align:right;
direction:rtl;
unicode-bidi:embed;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
text-align:right;
direction:rtl;
unicode-bidi:embed;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:.5in;
margin-bottom:0in;
margin-left:0in;
margin-bottom:.0001pt;
text-align:right;
direction:rtl;
unicode-bidi:embed;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle23
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle25
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle28
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle29
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle30
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle31
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:54201393;
mso-list-type:hybrid;
mso-list-template-ids:1777755276 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:82531172;
mso-list-type:hybrid;
mso-list-template-ids:763034224 659980682 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:Arial;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style></head><body lang="EN-US" link="blue" vlink="purple"><div class="WordSection1"><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">I tried most of what you suggested but it didn’t work for me (I am probably doing something wrong).</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">The only thing I can say for sure is that the number jitted methods(in the perfmon) is raising constantly. This is something that is reproduced even when I run a fraction of the system on my local machine.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Is there any way to find what these methods are? I thought about dumping the VTable size of each type and see who is getting bigger, should that work? Could you tell me on what module should the Jitted methods appear so I can narrow my search?</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Thanks</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p>
<div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Dino Viehland<br>
<b>Sent:</b> Thursday, October 14, 2010 8:31 PM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">So from an IronPython/DLR perspective the process should stabilize over time. That could take a while – as various Python functions get used repeatedly we’ll switch from interpreting them to compiling them. We’ll also potentially produce new call site rules which are compiled. That could account for the increase of 16k->18k dynamic methods. That’s a 12% increase and could be a reasonable amount of it remained steady from then on. Likewise the # of function codes seems stable but the _codeCount is rising. That might mean that you’re defining new functions (via exec/execfile/compile) and they’re getting collected - but we still have their (dead) weak references in the code list. Eventually that list should get cleaned out when we hit context._nextCodeCleanup (which should be greater than context._codeCount).</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">I would expect if you were to walk the entire PythonContext._allCodes linked list that you’d see about half of the lists having a dead weak reference. If your windbg-foo is up for writing the script to do this that’d be great but mine is rusty enough I’d need to look it up in the documentation.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">As for jitted code – all dynamic methods are collectible and any normal RefEmit code is not collectible (there’s an option to make assemblies collectible in .NET 4.0, but we don’t use it as we generally don’t generate that many types). Oh, we do also generate new types for subclasses but you should see that NewTypeMaker._newTypes has a stable count over time because we share these between types w/ common bases. </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Closures, callbacks, generators should all be fine. </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">If you do .symfix, then .reload, in Windbg does “dt HostCodeHeap” show you the fields of the HostCodeHeap structure? Does !eeheap –loader give you the address of the HostCodeHeap? It might be useful to know what m_allocationCount is on the heap overtime as if that’s relatively stable the heap could be getting fragmented. Looking at the CLR code I’m becoming more and more convinced HostCodeHeap is only used for dynamic methods so if that’s growing then I’m thinking we would appear to be leaking dynamic methods. </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">The only way I can think of figuring out where the allocations are coming from is to put a breakpoint on mscorwks!HostCodeHeap::AllocMemory_NoThrow (hopefully this will show up in the public symbols). That may be difficult if you can’t attach the debugger to the server but if the issue is blocking the server you could have a breakpoint here which does a stack trace and continues execution so you could inspect the stack trace later. I’d include both “kb” and “!ClrStack” from sos in the stack trace.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">2.7 shouldn’t really change this – whether .NET 4.0 would or not would depend on if the CLR changed anything here. But I’m not sure – I would assume it wouldn’t.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Idan Zaltzberg<br>
<b>Sent:</b> Thursday, October 14, 2010 6:44 AM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Hi,</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">I've been looking on this for some time, and I'm still don’t understand some things. Maybe I should begin by explaining our usage of Ipy (2.6.1 .NET 2.0) a bit better.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">We have a long running (stateful) application that we cannot simulate or run with a debugger open (so no breakpoints). </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">However, since the application is run on a VM, we can take snapshots of it and then open WinDbg instance and break in the middle of the application.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">We do this a few hours after the application restarted and again after two days so we can see the difference.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">This way we saw that Jit Code Heap is increasing by a few hundred MB per day, and the number of HostCodeHeap objects is increasing.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">We also compared the performance counters for the two snapshots, and saw the Jitted Code Bytes increased from 100MB to 862MB and the number of methods jitted increased from 700K to 6.3M.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">In the WinDbg we saw that the Jitted Code Heap size increases from 126MB to 424MB.</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<span style="color:#1F497D">On the other hand, the object types you mentioned stay relatively the same:</span></p><p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l1 level1 lfo2;direction:ltr;unicode-bidi:embed">
<span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D">DynamicMethods count went from 16K to 18K</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l1 level1 lfo2;direction:ltr;unicode-bidi:embed">
<span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D">FunctionCode count went from 4013 to 4025</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l1 level1 lfo2;direction:ltr;unicode-bidi:embed">
<span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D">The _<i>code</i>Count field in the PythonContext went from 4447 to 7800</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Here is what we don't understand:</span></p><p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">Is it normal for the application to keep jitting code and methods forever? Should is stabilize?</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">From the numbers I guess that some of the jitted code IS collected. Which types are collectable and which are not? How can I tell which ones I am using?</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">Are there any specific patterns I should avoid to decrease uncollectable code (or jitting in general). I am using a lot of closures, callbacks and generators.</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">What datatypes that are visible in WinDbg can I use to understand if and why IronPython is generating uncollectable code?</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">5.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">Is there a way to trace back the HostCodeHeap objects to my code (or IronPython specific features)?</span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;text-align:left;text-indent:-.25in;mso-list:l0 level1 lfo4;direction:ltr;unicode-bidi:embed">
<span style="mso-list:Ignore">6.<span style="font:7.0pt "Times New Roman""> </span></span><span dir="LTR"></span><span style="color:#1F497D">Can I expect an improvement in these issues by moving to Ipy 2.7 and/or .Net 4.0?</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Thanks</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Dino Viehland<br>
<b>Sent:</b> Wednesday, October 13, 2010 10:01 PM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">If you build from source you could set some breakpoints in AssemblyGen.cs in the DefineType method. You can also set one in DelegateUtils.cs <b>and</b> DelegateHelpers.cs in DefineDelegateType. I think those are all the places where we are creating uncollectible types. If we’re continuously hitting those breakpoints after you believe your app has reached steady state then something is going wrong.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">This code <a href="http://www.koders.com/cpp/fid5CC8EACFCC85496B49B8CF83BD05AB36DE691E90.aspx">http://www.koders.com/cpp/fid5CC8EACFCC85496B49B8CF83BD05AB36DE691E90.aspx</a> leads me to believe the HostCodeHeap might also be used for DynamicMethods. If that is the case then the other place to look would be if FunctionCode objects are being re-created repeatedly. That will happen if there’s exec/eval/compile calls which are happening and if those objects are being kept alive then we could be growing the heap over time. </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">There’s also some complicated code which deals with keeping a list of all code that is alive. We do cleanup this list, and the list is a list of weak references so it shouldn’t actually keep the code alive, but you could put some breakpoints at FunctionCode.RegisterFunctionCode and FunctionCode.CodeCleanup to see if that list is growing boundlessly (which it would be if something was keeping code objects alive after an exec/eval/compile). </span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Another place where code generation could be occurring would be w/ regexes. If you are dynamically generating reg-exes, or executing a huge different variety of them over time, and they’re compiled, then the compiled regexes could be staying in memory. There is a regex cache and you can clear it by calling re.purge(). But it should only cache up to 100 regexes.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">A final possible thing to investigate might be what happens if you throw away the entire ScriptEngine instance. Here you could try re-cycling the ScriptEngine say every 6 hours and see if the problem goes away. If that fixes the problem then it’s likely that it is one of the things I mentioned (or some other cache that’s per-runtime). At least that would start to narrow it down vs. some potentially global state (like the subtype list which is shared across ScriptEngines).</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">That’s a bunch of different things to look at – hopefully it’ll give some insight into what’s going on and help track down the issue.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Idan Zaltzberg<br>
<b>Sent:</b> Wednesday, October 13, 2010 12:12 AM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">I tried what you suggested (changed </span><span style="font-size:10.0pt;font-family:Consolas">setup.DebugMode = <span style="color:blue">false</span>;</span>)</p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">But still I get the same behavior:</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<span style="color:#1F497D">The "Jit Code Heap" increases from about 17MB to 230MB in 2 days.</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Is there a way to verify from the IronPython code that DebugMode is off?</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Is there anything else I can do (other startup settings?) to decrease/understand the increase in HostCodeHeap objects?</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Thanks.</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p>
<div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Dino Viehland<br>
<b>Sent:</b> Tuesday, October 05, 2010 6:59 PM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Yep, DebugMode is the same as –X:Debug. In general I’d suggest making this configurable somehow and only turn it on if you’re actually debugging. It’s unfortunate that we can’t offer both debugging & collectability but right now that’s simply a limitation of the CLR and/or our lack of a separate VS debug engine which can debug Python code.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Idan Zaltzberg<br>
<b>Sent:</b> Tuesday, October 05, 2010 9:09 AM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">Im running using the engine from a hosting app.</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<span style="color:#1F497D">We have these lines in the startup:</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="font-size:10.0pt;font-family:Consolas;color:#2B91AF">ScriptRuntimeSetup</span><span style="font-size:10.0pt;font-family:Consolas"> setup = <span style="color:blue">new</span> <span style="color:#2B91AF">ScriptRuntimeSetup</span>();<br>
setup.DebugMode = <span style="color:blue">true</span>;</span></p><pre style="margin-bottom:12.0pt"><span style="font-family:Consolas;color:#2B91AF">ScriptRuntime</span><span style="font-family:Consolas"> runtime = <span style="color:#2B91AF">Python</span>.CreateRuntime(setup.Options);</span></pre>
<pre><span style="font-family:Consolas">engine = runtime.GetEngine(<span style="color:#A31515">"py"</span>);</span></pre><pre><span style="font-family:Consolas"> </span></pre><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<span style="color:#1F497D">Is this is the same like –X:Debug?</span></p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">You reckon this could be the cause?</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Dino Viehland<br>
<b>Sent:</b> Tuesday, October 05, 2010 5:53 PM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> Re: [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D">My guess is that’s code in the JIT heap that’s building up but I’m not 100% certain. How is your code being executed? Do you have the debug option (-D or –X:Debug) enabled? To support debug mode we need to produce uncollectible code which could be building up.</span></p>
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><span style="color:#1F497D"> </span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com">users-bounces@lists.ironpython.com</a>] <b>On Behalf Of </b>Idan Zaltzberg<br>
<b>Sent:</b> Tuesday, October 05, 2010 2:26 AM<br><b>To:</b> Discussion of IronPython<br><b>Subject:</b> [IronPython] HostCodeHeap leakage?</span></p></div></div><p class="MsoNormal" style="text-align:left;direction:ltr;unicode-bidi:embed">
</p><p>I am trying to find a memory/"performance" leak in an Ipy application.</p><p>Using WINDBG (!eeheap -loader), we noticed the that the LoaderHeap is getting bigger (150MB increase per day). From the !eeheap output it seems that the increase is due to HostCodeHeap (objects?).</p>
<p>As I understand these objects might be created by Ipy infra, is that right?</p><p>Is there anyway I can get more info on their content, or prevent them from growing?</p><p>Thanks</p><p class="MsoNormal" dir="RTL"><span dir="LTR"> </span></p>
</div></div></div></div></div></body></html>