[IronPython] [python] Re: Default recursion limit

Michael Foord fuzzyman at voidspace.org.uk
Wed Oct 3 23:14:46 CEST 2007


Dino Viehland wrote:
> Ahh, stack overflow is fun...  The special value of maxint actually means that we stop enforcing recursion altogether.
>
> Why did we pick that default?  It's a combination of performance and our general thinking of stack overflow.  We typically think of SO as programmer error and therefore choose to make the trade-off in terms of better performance vs. better stack overflow handling.  Unfortunately it's hard to enforce a recursion limit and do so in a fast manner (it typically involves reading and writing a thread static which isn't that fast in .NET - we actually have a work-around for this which will work good for apps w/ a smaller number of threads, but it's still a significant slowdown).  What would be ideal would be if .NET had good stack overflow handling but it's such a hard issue to get right that .NET only enables recovering from SO for advanced hosts (and even then it comes at the cost of losing your app domain).
>
> Finally if you don't like our decision it is only one command (or one command-line option) away from being changed.  If people don't like this default we could consider changing it but there hasn't been much issue with this to date.
>
> The recursion issue is interesting.  Somehow this is related to DynamicMethod's - if I run w/ -X:SaveAssemblies -X:StaticMethods this speeds up considerably (on my machine we go from ~3-4 seconds for 5000 frames to .08 or .03 seconds depending on -X:NoTraceback).
>
>   

Interesting (it takes about 10 seconds on my machine at the interactive 
interpreter).

What does '-X:StaticMethods' do?

So setting a recursionlimit will generally reduce performance? (Our 
performance tests still pass - so it doesn't look like this is a major 
issue for us).

Thanks Dino

Michael
http://www.manning.com/foord

> I expected this to be related to traceback support - that forces us to emit a try/catch instead of a try/fault in a DynamicMethod - which means we catch & rethrow exceptions all the way up the stack and that's going to be horrible for performance.  But enabling -X:NoTraceback no DynamicMethod's actually seems to make this go slower!
>
> So there's definitely something funky going on here.  It might be an issue w/ throwing exceptions & DynamicMethod's inside of the CLR, or it might be something we're doing.  I'll need to investigate it a little more.  I've opened a low-priority bug to investigate this (http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=13080).  If you think that's the wrong priority let me know :)
>
>
>
>
> -----Original Message-----
> From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
> Sent: Tuesday, October 02, 2007 4:53 PM
> To: Discussion of IronPython
> Subject: [IronPython] Default recursion limit
>
> Hello all,
>
> For IronPython the default recursion limit is maxint: 2147483647
>
> This is too deep for .NET - and infinite recursion causes the process to
> be terminated with a stack overflow exception.
>
> You can fix this by calling "sys.setrecursionlimit(something)" with
> something sensible - but an unfeasibly large value as the default seems
> incorrect (?).
>
> There is a further performance related issue. Generally IronPython is
> much faster for recursion due to lower overhead in function calls. For
> recursion depths of about 5000 though, IronPython takes *much* longer to
> raise an error than CPython (about 10 seconds as opposed to half a
> second or less).
>
> Michael
> http://www.manning.com/foord
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>   




More information about the Ironpython-users mailing list