[IronPython] [python] Re: Default recursion limit

Michael Foord fuzzyman at voidspace.org.uk
Wed Oct 3 23:58:12 CEST 2007


Dino Viehland wrote:
> -X:StaticMethods causes us to never use DynamicMethod's.  It's really more of a debugging tool than anything else.  The problem w/ DynamicMethods is that they are completely undebuggable.  You don't get line number information from them, you can't see the IL of them, and you can't verify the IL by saving it to disk.  So we can switch into StaticMethods mode and get all of this dumped out and see what's going wrong in the code.
>
> The downside is that we can't generate just a method - we have to generate an entire type.  And that entire type is then not collectible by the GC so we leak memory.
>
> Yep, so on my machine I see around a ~15% degrade on Pystone.  It probably is a function of how many function's you're calling and how much work they do, but there's definitely a cost associated with it.  And that cost will rise if you go above ~20 threads or so when the one optimization to make this faster stops working on the higher #'d threads.
>   

Ok - thanks Dino.

Michael

> -----Original Message-----
> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
> Sent: Wednesday, October 03, 2007 2:15 PM
> To: Discussion of IronPython
> Subject: Re: [IronPython] [python] Re: Default recursion limit
>
> Dino Viehland wrote:
>   
>> Ahh, stack overflow is fun...  The special value of maxint actually means that we stop enforcing recursion altogether.
>>
>> Why did we pick that default?  It's a combination of performance and our general thinking of stack overflow.  We typically think of SO as programmer error and therefore choose to make the trade-off in terms of better performance vs. better stack overflow handling.  Unfortunately it's hard to enforce a recursion limit and do so in a fast manner (it typically involves reading and writing a thread static which isn't that fast in .NET - we actually have a work-around for this which will work good for apps w/ a smaller number of threads, but it's still a significant slowdown).  What would be ideal would be if .NET had good stack overflow handling but it's such a hard issue to get right that .NET only enables recovering from SO for advanced hosts (and even then it comes at the cost of losing your app domain).
>>
>> Finally if you don't like our decision it is only one command (or one command-line option) away from being changed.  If people don't like this default we could consider changing it but there hasn't been much issue with this to date.
>>
>> The recursion issue is interesting.  Somehow this is related to DynamicMethod's - if I run w/ -X:SaveAssemblies -X:StaticMethods this speeds up considerably (on my machine we go from ~3-4 seconds for 5000 frames to .08 or .03 seconds depending on -X:NoTraceback).
>>
>>
>>     
>
> Interesting (it takes about 10 seconds on my machine at the interactive
> interpreter).
>
> What does '-X:StaticMethods' do?
>
> So setting a recursionlimit will generally reduce performance? (Our
> performance tests still pass - so it doesn't look like this is a major
> issue for us).
>
> Thanks Dino
>
> Michael
> http://www.manning.com/foord
>
>   
>> I expected this to be related to traceback support - that forces us to emit a try/catch instead of a try/fault in a DynamicMethod - which means we catch & rethrow exceptions all the way up the stack and that's going to be horrible for performance.  But enabling -X:NoTraceback no DynamicMethod's actually seems to make this go slower!
>>
>> So there's definitely something funky going on here.  It might be an issue w/ throwing exceptions & DynamicMethod's inside of the CLR, or it might be something we're doing.  I'll need to investigate it a little more.  I've opened a low-priority bug to investigate this (http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=13080).  If you think that's the wrong priority let me know :)
>>
>>
>>
>>
>> -----Original Message-----
>> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
>> Sent: Tuesday, October 02, 2007 4:53 PM
>> To: Discussion of IronPython
>> Subject: [IronPython] Default recursion limit
>>
>> Hello all,
>>
>> For IronPython the default recursion limit is maxint: 2147483647
>>
>> This is too deep for .NET - and infinite recursion causes the process to
>> be terminated with a stack overflow exception.
>>
>> You can fix this by calling "sys.setrecursionlimit(something)" with
>> something sensible - but an unfeasibly large value as the default seems
>> incorrect (?).
>>
>> There is a further performance related issue. Generally IronPython is
>> much faster for recursion due to lower overhead in function calls. For
>> recursion depths of about 5000 though, IronPython takes *much* longer to
>> raise an error than CPython (about 10 seconds as opposed to half a
>> second or less).
>>
>> Michael
>> http://www.manning.com/foord
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>
>>
>>     
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>   




More information about the Ironpython-users mailing list