[IronPython] Default recursion limit

Dino Viehland dinov at exchange.microsoft.com
Wed Oct 3 19:35:30 CEST 2007


Ahh, stack overflow is fun...  The special value of maxint actually means that we stop enforcing recursion altogether.

Why did we pick that default?  It's a combination of performance and our general thinking of stack overflow.  We typically think of SO as programmer error and therefore choose to make the trade-off in terms of better performance vs. better stack overflow handling.  Unfortunately it's hard to enforce a recursion limit and do so in a fast manner (it typically involves reading and writing a thread static which isn't that fast in .NET - we actually have a work-around for this which will work good for apps w/ a smaller number of threads, but it's still a significant slowdown).  What would be ideal would be if .NET had good stack overflow handling but it's such a hard issue to get right that .NET only enables recovering from SO for advanced hosts (and even then it comes at the cost of losing your app domain).

Finally if you don't like our decision it is only one command (or one command-line option) away from being changed.  If people don't like this default we could consider changing it but there hasn't been much issue with this to date.

The recursion issue is interesting.  Somehow this is related to DynamicMethod's - if I run w/ -X:SaveAssemblies -X:StaticMethods this speeds up considerably (on my machine we go from ~3-4 seconds for 5000 frames to .08 or .03 seconds depending on -X:NoTraceback).

I expected this to be related to traceback support - that forces us to emit a try/catch instead of a try/fault in a DynamicMethod - which means we catch & rethrow exceptions all the way up the stack and that's going to be horrible for performance.  But enabling -X:NoTraceback no DynamicMethod's actually seems to make this go slower!

So there's definitely something funky going on here.  It might be an issue w/ throwing exceptions & DynamicMethod's inside of the CLR, or it might be something we're doing.  I'll need to investigate it a little more.  I've opened a low-priority bug to investigate this (http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=13080).  If you think that's the wrong priority let me know :)




-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
Sent: Tuesday, October 02, 2007 4:53 PM
To: Discussion of IronPython
Subject: [IronPython] Default recursion limit

Hello all,

For IronPython the default recursion limit is maxint: 2147483647

This is too deep for .NET - and infinite recursion causes the process to
be terminated with a stack overflow exception.

You can fix this by calling "sys.setrecursionlimit(something)" with
something sensible - but an unfeasibly large value as the default seems
incorrect (?).

There is a further performance related issue. Generally IronPython is
much faster for recursion due to lower overhead in function calls. For
recursion depths of about 5000 though, IronPython takes *much* longer to
raise an error than CPython (about 10 seconds as opposed to half a
second or less).

Michael
http://www.manning.com/foord
_______________________________________________
Users mailing list
Users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



More information about the Ironpython-users mailing list