<div dir="ltr">The" .NET people" might say they already have an answer which involves precompiling the IL with NGEN. And in fact, using DLR precompilation and NGEN with IronPython results in some pretty nice performance improvements.<div>
<br></div><div>But using IronPython directly against .py files incurs several expenses that aren't part of normal .NET startup. We have to parse the .py file, build DLR-level expression trees from the language tree and build IL from the expression tree. Only then are we at the point where a C# starts, and can JIT the IL.</div>
<div><br></div><div>It's worth noting that CPython has to do similar work when it first loads a .py file. It, too, has to parse into a tree structure and generate byte codes for the tree. And even though the CPython byte code is at a higher level than the IL generated by the DLR, CPython still has a performance issue -- which is addressed by storing the byte codes as a .pyc file.</div>
<div><br></div><div>I'm hoping that over the next year, we'll come up with a convention (like .pyc) that allows subsequent runs of a program to benefit from the work done in previous runs -- without having to go through the manual step of DLR precompilation.</div>
<div><br><div class="gmail_quote">On Mon, Oct 13, 2008 at 9:57 AM, Vernon Cole <span dir="ltr"><<a href="mailto:vernondcole@gmail.com">vernondcole@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div dir="ltr">Help me out here. I have seen many references to schemes to optimize import speeds on Iron Python. Isn't this an attempt to make up for the snail-like startup speeds suffered by <b>all</b> .net applications? It seems to me that when Iron Python finally starts running, it imports and performs very well. The slowdown is in getting the .net engine started. Try comparing startup times for "Hello World" in C Python and Iron.<br>
It is my experience that I can identify .net programs in general by the fact that I have double-clicked its desktop icon three times (thinking I had missed it somehow) before the first instance displays its splash screen, and then the next two display an "already running" dialog. <br>
Any chance of getting the .net people to help out with this?<br>--<br><font color="#888888">Vernon Cole</font><div><div></div><div class="Wj3C7c"><br><br><div class="gmail_quote">On Mon, Oct 13, 2008 at 10:00 AM, Dino Viehland <span dir="ltr"><<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex">This is all still on 1.x, right? It looks like #1 is fixed in 2.0 (we are locking but on the wrong object in 1.x).<br>
<br>
#2 is still broken in 2.x though as well.<br>
<div><div></div><div><br>
-----Original Message-----<br>
From: <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>] On Behalf Of Kamil Dworakowski<br>
Sent: Monday, October 13, 2008 4:45 AM<br>
To: <a href="mailto:users@lists.ironpython.com" target="_blank">users@lists.ironpython.com</a><br>
Subject: [IronPython] parallel importing<br>
<br>
Importing time is such a pain that I wanted to do it in parallel on<br>
many threads. I figured that if the modules where imported in such a<br>
way that:<br>
<br>
no two threads import the same module at the same time<br>
<br>
everything would be fine. To ensure that condition, it is enough to<br>
build a dependency graph and import based on that.<br>
<br>
I did it: Resolver One start up time improved by 20% on a two-core<br>
machine. But it crashes, surprisingly on single core machines it is<br>
more often (6 crashes on 200 starts).<br>
<br>
So far I have identified two causes for crashes:<br>
<br>
1. One thread imports a module with class B inside while another is<br>
importing a module with class C inside. If B and C are subclasses of<br>
A, it can result in IndexOutOfRangeException being raised, when, under<br>
the hood, IronPython.Runtime.Types.DynamicType.AddSubclass is being<br>
executed.<br>
<br>
2. Attributes on .NET modules are loaded lazily, so importing<br>
namespaces only is not enough. Attribute getting from reflected<br>
packages is not thread safe. Looks like I would have to import every<br>
class explicitly (would that be enough?).<br>
<br>
Second cause would be pretty easy to address, but I'm not so sure<br>
about the first one. Are there any more potential points of problems?<br>
I am beginning to think I was to optimistic about all of this<br>
importing on multiple cores, but if these are the only ones it could<br>
probably be still fixed.<br>
<br>
If anyone is interested the code for it is on github:<br>
<a href="http://github.com/luntain/ipy-parallel-import" target="_blank">http://github.com/luntain/ipy-parallel-import</a>.<br>
<br>
--<br>
Kamil Dworakowski<br>
Resolver Systems Ltd.<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
</div></div></blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
<br></blockquote></div><br></div></div>