[IronPython] Informal Poll: measuring IronPython performance under x64 OSes

[Dave Fugate]

>From what I've seen on the mailing list and several emails sent directly to me there seems to be about a 50-50 split between x86/x64 interest.

Anyone interested in x86 vs x64 IronPython performance might want to check out the following data points taken from two identical Vista machines that differ only in terms of OS bitted-ness:

*         MAY 2008  - IronPython startup performance is 25% slower on x64 than x86.

*         JUNE 2007 - IronPython startup performance was %32 slower on x64 than x86.



*         MAY 2008 -  PyBench (Calls) is around 33% slower on x64 than x86.

*         JUNE 2007 - PyBench (Calls) was 34% slower on x64 than x86.



*         MAY 2008  - PyStone is 40% slower on x64 than x86.

*         JUNE 2007 - PyStone is %36 slower on x64 than x86.

As you can see, although IronPython performance in given areas may change over the course of a year ("PyBench Calls" was about 10% faster in May 2008), the delta between IronPython on x86 and x64 platforms pretty much stays the same for a given benchmark.  The principal reason behind this is IronPython has few (any?) x64 only code paths.  Any differences in performance should be due to differences between the x86 and x64 CLRs.  My disclaimer on this statement is that deltas can vary widely depending upon which benchmark we look at (e.g., the delta for "PyBench Lists" between x86/x64 is under 10%).

From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Dave Fugate
Sent: Monday, August 18, 2008 2:43 PM
To: Discussion of IronPython
Subject: [IronPython] Informal Poll: measuring IronPython performance under x64 OSes

Hi everyone, one of the things the IronPython test team does daily is run a set of standardized benchmarks against the latest internal build of IronPython.  We're in the process of reevaluating the platforms we measure these benchmarks on with the end goal of increasing the number of tests we can run.  For example, if we could cut x64 Vista from the list of platforms, the x64 Vista machine(s) could be reimaged to run new benchmarks we wouldn't have had the resources for before.  Alternatively, we might reimage the x64 Vista machine(s) as some other platform we don't currently test under.  Of course in such a scenario we'd still run other classifications of tests under x64 Vista.

I wanted to hear peoples' opinions on the importance of measuring IronPython performance under x64 Windows OSes before we make any decisions on this.  First off, how many people out there are actually running IronPython under a 64-bit OS?  Would it be sufficient for us to report news on IronPython performance in terms of 32-bit platforms?

Thanks,

The IronPython Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20080820/07d0e4da/attachment.html>