<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
@font-face
{font-family:Consolas}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Times New Roman","serif"}
span.emailstyle18
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle20
{font-family:"Calibri","sans-serif";
color:#1F497D}
.MsoChpDefault
{font-size:10.0pt}
@page Section1
{margin:1.0in 1.0in 1.0in 1.0in}
-->
</style><style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1" lang="EN-US" link="blue" vlink="purple">
<div style="direction: ltr; font-family: Tahoma; color: rgb(0, 0, 0); font-size: 13px;">
<div style="">In the case that affects me the code isn't global. The class looks like this:<br>
<br>
<span style="font-family: Consolas;">class EseDBCursor(object):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> ...</span><br>
<br>
<span style="font-family: Consolas;"> def __getitem__(self, key): </span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> with _EseTransaction(self._sesid):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> self._seekForKey(key)</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> return self._retrieveCurrentRecordValue()</span><br>
<br style="font-family: Consolas;">
<span style="font-family: Consolas;"> def _makeKey(self, key):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> """Construct a key for the given value."""</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> Api.MakeKey(self._sesid, self._tableid, str(key), Encoding.Unicode, MakeKeyGrbit.NewKey)</span><br style="font-family: Consolas;">
<br style="font-family: Consolas;">
<span style="font-family: Consolas;"> def _seekForKey(self, key):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> """Seek for the specified key. A KeyError exception is raised if the key isn't found."""</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> self._makeKey(key)</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> if not Api.TrySeek(self._sesid, self._tableid, SeekGrbit.SeekEQ):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> raise KeyError('key \'%s\' was not found' % key)</span><br>
<br>
I tried moving the call to Api.MakeKey into _seekForKey but that didn't improve things. It is definitely the call to Api.MakeKey that is slowing things down -- removing that one call speeds things up. The code that tests the performance looks like this:<br>
<br>
<span style="font-family: Consolas;">def insertRetrieveTest():<br>
...<br>
timer = Stopwatch.StartNew()</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> for i in xrange(n):</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> data = db[k]</span><br style="font-family: Consolas;">
<span style="font-family: Consolas;"> timer.Stop()<br>
...<br>
</span><br>
# Basic test first<br>
insertRetrieveTest()<br>
<br>
</div>
<div style="font-family: Times New Roman; color: rgb(0, 0, 0); font-size: 16px;">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF683459"><font color="#000000" face="Tahoma" size="2"><b>From:</b> users-bounces@lists.ironpython.com [users-bounces@lists.ironpython.com] on behalf of Dino Viehland [dinov@microsoft.com]<br>
<b>Sent:</b> Tuesday, April 20, 2010 12:51 PM<br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython]Bad performance calling .NET method<br>
</font><br>
</div>
<div></div>
<div>
<div class="Section1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">Is the perf problem only there when the code is global? In general we don’t try to optimize code which occurs at the top-level of a script – we
assume the most significant work will occur inside of function definitions. In this case the update to “i” through each loop iteration needs to update a global value in a dictionary rather than updating a local on the stack each time which is going to be
much more expensive. Also the reads from the globals instead of parameters will be much more expensive as well. And while I doubt this is much of the perf problem you’re also accessing the Unicode property on encoding each time through in the global case.
There’s also the chance depending on what version of IronPython you’re running on that we’re staying in the interpreter. I think on 2.6.1 we should compile the global loop eventually but it’ll still be slower than the compiled function.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">Is there a reason the top-level code needs to be efficient?</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"> </span></p>
<div style="border-width: medium medium medium 1.5pt; border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; padding: 0in 0in 0in 4pt;">
<div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif";">From:</span></b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif";"> users-bounces@lists.ironpython.com [mailto:users-bounces@lists.ironpython.com]
<b>On Behalf Of </b>Laurion Burchall<br>
<b>Sent:</b> Tuesday, April 20, 2010 11:57 AM<br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython] Bad performance calling .NET method</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;">You were right about the structs. By creating a method with the same type of signature (struct, struct, string, Encoding, enumeration).
I got the same slowdown in a trivial DLL. This is with the .NET 2.0 version of IP:<br>
<br>
** First, the C# code:<br>
<br>
</span><span style="font-size: 10pt; font-family: Consolas; color: black;">namespace IronPythonInteropPerf<br>
{<br>
using System.Text;<br>
<br>
public struct Struct1<br>
{<br>
internal int i;<br>
}<br>
<br>
public struct Struct2<br>
{<br>
internal int i;<br>
}<br>
<br>
public enum Enumeration<br>
{<br>
Foo,<br>
}<br>
<br>
public static class Interop<br>
{<br>
public static void A(Struct1 a, Struct2 b, string x, Encoding encoding, Enumeration e)<br>
{<br>
}<br>
}<br>
}<br>
</span><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;"><br>
** Now the IronPython test:<br>
<br>
</span><span style="font-size: 10pt; font-family: Consolas; color: black;">import System<br>
import System.Diagnostics<br>
import System.Text<br>
<br>
from System.Diagnostics import Stopwatch<br>
from System.Text import Encoding<br>
<br>
import clr<br>
clr.AddReferenceByPartialName('IronPythonInteropPerf')<br>
from IronPythonInteropPerf import *<br>
<br>
N = 1000000<br>
<br>
a = Struct1()<br>
b = Struct2()<br>
c = 'hello'<br>
d = Enumeration.Foo<br>
<br>
stopwatch = Stopwatch.StartNew()<br>
for i in xrange(N):<br>
Interop.A(a, b, c, Encoding.Unicode, d)<br>
stopwatch.Stop()<br>
print ' A: %s' % stopwatch.Elapsed<br>
<br>
def foo(a1, a2, a3, a4, a5):<br>
stopwatch = Stopwatch.StartNew()<br>
for j in xrange(N):<br>
Interop.A(a1, a2, a3, a4, a5)<br>
stopwatch.Stop()<br>
print 'foo.A: %s' % stopwatch.Elapsed<br>
<br>
foo(a, b, c, Encoding.Unicode, d)<br>
<br>
</span><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;">** Note that I am calling Interop.A twice. This gives very different results -- the call from inside of function foo() is fast (11M calls/second) but the call from the script
is slow (50K calls/second):<br>
<br>
A: 00:00:18.6063353<br>
foo.A: 00:00:00.0894888<br>
<br>
There isn't a caching effect at work either -- foo() is faster if I call it before the other code.</span></p>
</div>
<div>
<div class="MsoNormal" style="text-align: center;" align="center"><span style="color: black;">
<hr align="center" size="2" width="100%">
</span></div>
<div id="divRpF994385">
<p class="MsoNormal" style="margin-bottom: 12pt;"><b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;">From:</span></b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;"> users-bounces@lists.ironpython.com
[users-bounces@lists.ironpython.com] on behalf of Dino Viehland [dinov@microsoft.com]<br>
<b>Sent:</b> Tuesday, April 20, 2010 11:34 AM<br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython]Bad performance calling .NET method</span><span style="color: black;"></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">I assume something is going horribly wrong with our type checks. Can you attach the repro? Or at least are these just classes, or are any structs,
or maybe weird classes like delegates?</span><span style="color: black;"></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"> </span><span style="color: black;"></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">And is this on .NET 2.0 or .NET 4.0?</span><span style="color: black;"></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"> </span><span style="color: black;"></span></p>
<div style="border-width: medium medium medium 1.5pt; border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color windowtext; padding: 0in 0in 0in 4pt;">
<div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: windowtext -moz-use-text-color -moz-use-text-color; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;">From:</span></b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;"> users-bounces@lists.ironpython.com [mailto:users-bounces@lists.ironpython.com]
<b>On Behalf Of </b>Laurion Burchall<br>
<b>Sent:</b> Tuesday, April 20, 2010 2:11 AM<br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> [IronPython] Bad performance calling .NET method</span><span style="color: black;"></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color: black;"> </span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;">I am getting terrible performance invoking a C# method from IP. I have a static class called Api with this method:<br>
<br>
public static void MakeKey(JET_SESID sesid, JET_TABLEID tableid, string data, Encoding encoding, MakeKeyGrbit grbit)<br>
<br>
When I call it directly from C# I get about 3M calls/second. In IronPython I get only 50,000 calls/second -- a 60X slowdown!<br>
<br>
The method is overloaded. When I call these overloads I get good performance (~ 1M calls/second):<br>
public static void MakeKey(JET_SESID sesid, JET_TABLEID tableid, int data, MakeKeyGrbit grbit)<br>
public static void MakeKey(JET_SESID sesid, JET_TABLEID tableid, float data, MakeKeyGrbit grbit)<br>
public static void MakeKey(JET_SESID sesid, JET_TABLEID tableid, byte[] data, MakeKeyGrbit grbit)<br>
(for the last overload I passed in the string turned into a byte array with Encoding.GetBytes())<br>
<br>
Things I have tried that didn't help:<br>
- Changing the name of the method so it was unique.<br>
- Calling the method using Api.MakeKey.Overloads[...]<br>
- Calling other methods I have that take string arguments. They were fast.<br>
<br>
When I profile the code the time shows up in mscorwks.dll (56%) and mscorlib.ni.dll (17%). IronPython is only 8% and my code is 6%.<br>
<br>
Can anyone help me work out what is going wrong? I have a short, turn-key repro of this. MakeKey is a very commonly used method so having it be so slow is crippling.<br>
<br>
thanks,<br>
--Laurion<br>
<br>
</span><span style="color: black;"></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>