I set the ModuleBuiltins compiler option but the behavior is the same. The only difference from the command line is that I am calling a python function from the DLR, and on the command line I am just calling the method from __main__. Do you think could be an issue? The other thing is that the python script relies heavily on I/O, input file is in the range of 18Mb and the data is stored on another file. Does the DLR handles I/O differently than the command line?<div>
<br></div><div>regards,</div><div><br></div><div>wilfredo<br><br><div class="gmail_quote">On Wed, Nov 12, 2008 at 5:39 PM, Dino Viehland <span dir="ltr"><<a href="mailto:dinov@microsoft.com">dinov@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p><span style="font-size:11.0pt;color:#1F497D">The only thing that immediately pops out at me as being
different is we also set ModuleOptions.ModuleBuiltins in the command line case –
but I wouldn't expect it to make such a large difference. But try setting that
option as well and let's see what happens.</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt">
<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a> [mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>] <b>On
Behalf Of </b>Wilfredo Lugo<br>
<b>Sent:</b> Wednesday, November 12, 2008 12:16 PM<div><div></div><div class="Wj3C7c"><br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython] Performance Issue</div></div></span></p>
</div><div><div></div><div class="Wj3C7c">
<p> </p>
<p>Thanks. Performance improved, but still over two
minutes. Here is the latest code:</p>
<div>
<p> </p>
</div>
<div>
<div>
<p> static void Main(string[]
args)</p>
</div>
<div>
<p> {</p>
</div>
<div>
<p> ScriptRuntime
runtime = Python.CreateRuntime();</p>
</div>
<div>
<p> ScriptEngine
engine = Python.CreateEngine();</p>
</div>
<div>
<p> ScriptSource
source = engine.CreateScriptSourceFromFile("interpolate.py");</p>
</div>
<div>
<p>
PythonCompilerOptions options = (PythonCompilerOptions) engine.GetCompilerOptions();</p>
</div>
<div>
<p>
options.Module |= ModuleOptions.Optimized;</p>
</div>
<div>
<p>
Console.WriteLine(DateTime.Now);</p>
</div>
<div>
<p> CompiledCode
compiled = source.Compile(options);</p>
</div>
<div>
<p>
compiled.Execute();</p>
</div>
<div>
<p> ScriptScope
scope = compiled.DefaultScope;</p>
</div>
<div>
<p>
Microsoft.Func<string, string, string, string, int, int> func =
scope.GetVariable<Microsoft.Func<string, string, string, string, int,
int>>("interpolate_start");</p>
</div>
<div>
<p>
Console.WriteLine(func("3-day-data.txt", "3-day-output.txt",
"2008-01-01 00:00:21","2008-01-03
23:59:08",60).ToString());</p>
</div>
<div>
<p>
Console.WriteLine(DateTime.Now);</p>
</div>
<div>
<p>
Console.ReadLine();</p>
</div>
<div>
<p> }</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Output :</p>
</div>
<div>
<div>
<p>11/12/2008 4:10:10 PM</p>
</div>
<div>
<p>WARNING: desired starting time (Tue Jan 01 00:00:21 2008)
prior to time of first</p>
</div>
<div>
<p> data point (Tue Jan 01 00:00:32 2008), current line:
9411</p>
</div>
<div>
<p> series: 3D</p>
</div>
<div>
<p>force==False, exiting..</p>
</div>
<div>
<p>0</p>
</div>
<div>
<p>11/12/2008 4:12:45 PM</p>
</div>
</div>
<div>
<p> </p>
</div>
<div>
<p style="margin-bottom:12.0pt"> </p>
<div>
<p>On Wed, Nov 12, 2008 at 4:01 PM, Dino Viehland <<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>> wrote:</p>
<div>
<div>
<p><span style="font-size:11.0pt;color:#1F497D">Oh, sorry, I missed a
step… You'll need to call ScriptEngine.GetCompilerOptions(). Cast
that to a PythonCompilerOptions, and then do options.Module |=
ModuleOptions.Optimized; Finally do source.Compile(options) and then it
should give you the optimized code.</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>
[mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Wilfredo Lugo<br>
<b>Sent:</b> Wednesday, November 12, 2008 11:45 AM</span></p>
<div>
<div>
<p><span style="font-size:10.0pt"><br>
<b>To:</b> Discussion of IronPython<br>
<b>Subject:</b> Re: [IronPython] Performance Issue</span></p>
</div>
</div>
</div>
<div>
<div>
<p> </p>
<p>Thanks!.</p>
<div>
<p> </p>
</div>
<div>
<p>I was able to get the default scope from CompiledCode, but I am still
getting the same performance. Here is the latest code : </p>
</div>
<div>
<div>
<p> static void Main(string[] args)</p>
</div>
<div>
<p> {</p>
</div>
<div>
<p> ScriptRuntime runtime =
Python.CreateRuntime();</p>
</div>
<div>
<p> ScriptEngine engine =
Python.CreateEngine();</p>
</div>
<div>
<p> ScriptSource source =
engine.CreateScriptSourceFromFile("interpolate.py");</p>
</div>
<div>
<p>
Console.WriteLine(DateTime.Now);</p>
</div>
<div>
<p> CompiledCode compiled =
source.Compile();</p>
</div>
<div>
<p> compiled.Execute();</p>
</div>
<div>
<p> ScriptScope scope =
compiled.DefaultScope;</p>
</div>
<div>
<p> Microsoft.Func<string,
string, string, string, int, int> func =
scope.GetVariable<Microsoft.Func<string, string, string, string, int,
int>>("interpolate_start");</p>
</div>
<div>
<p>
Console.WriteLine(func("3-day-data.txt",
"3-day-output.txt", "2008-01-01 00:00:21","2008-01-03
23:59:08",60).ToString());</p>
</div>
<div>
<p> Console.WriteLine(DateTime.Now);</p>
</div>
<div>
<p> Console.ReadLine();</p>
</div>
<div>
<p> }</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>This is the latest output :</p>
</div>
<div>
<div>
<p> 11/12/2008 3:13:01 PM</p>
</div>
<div>
<p> WARNING: desired starting time (Tue Jan 01 00:00:21 2008) prior
to time of first</p>
</div>
<div>
<p> data point (Tue Jan 01 00:00:32 2008), current line: 9411</p>
</div>
<div>
<p> series: 3D</p>
</div>
<div>
<p> force==False, exiting..</p>
</div>
<div>
<p> 0</p>
</div>
<div>
<p>11/12/2008 3:16:41 PM</p>
</div>
</div>
<div>
<p> </p>
</div>
<div>
<p>Is there could be a problem on how I am calling the interpolation function?
I mean from python.exe I do something like this:</p>
</div>
<div>
<div>
<p> </p>
</div>
<div>
<p>if __name__ == '__main__':</p>
</div>
<div>
<p> Interpolate('3-day-data.txt', '3-day-output-original.txt',
'2008-01-01 00:00:</p>
</div>
<div>
<p>21', '2008-01-03 23:59:08', intvl=60, maxgap = 5, stacked = True, stackedCol
= 2</p>
</div>
<div>
<p>, tformat = '%Y-%m-%d %H:%M:%S', debug = False).start()</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>To be able to call the script from the DLR I have something like this:</p>
</div>
<div>
<p> </p>
</div>
<div>
<div>
<p>def interpolate_start(inFile, outFile, startDate, endDate, intvl):</p>
</div>
<div>
<p> Interpolate(inFile, outFile,startDate,endDate,intvl, maxgap =
5, stacked = Tr</p>
</div>
<div>
<p>ue, stackedCol = 2, tformat = '%Y-%m-%d %H:%M:%S', debug = False).start()</p>
</div>
<div>
<p> return 0</p>
</div>
</div>
</div>
<div>
<p> </p>
</div>
<div>
<p>That's the only difference I found on how the script is called from python
and DLR.</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Any comments?</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>regards,</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>wilfredo</p>
</div>
<p> </p>
<div>
<p>On Wed, Nov 12, 2008 at 2:56 PM, Dino Viehland <<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>> wrote:</p>
<div>
<div>
<p><span style="font-size:11.0pt;color:#1F497D">CompiledCode exposes the
default scope it executes in via the DefaultScope property. So hold onto
the result of source.Compile and after executing the code you can grab the
scope and use it to get your variable.</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>
[mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Wilfredo Lugo<br>
<b>Sent:</b> Wednesday, November 12, 2008 10:25 AM<br>
<b>To:</b> Discussion of IronPython</span></p>
<div>
<div>
<p><span style="font-size:10.0pt"><br>
<b>Subject:</b> Re: [IronPython] Performance Issue</span></p>
</div>
</div>
</div>
<div>
<div>
<p> </p>
<p>Thanks.</p>
<div>
<p> </p>
</div>
<div>
<p>But then If I don't use my defined scope, how I could fill my function
delegate?</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Right now I am using : </p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Microsoft.Func<string, string, string, string, int, int> func =
scope.GetVariable<Microsoft.Func<string, string, string, string, int,
int>>("interpolate_start");</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>If I use engine.GetVariables<Func<....>() I still need to pass a
ScriptScope to it. How do I get the default scope (from ScriptEngine or
ScriptSource) to be able to get the delegate?</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>regards,</p>
</div>
<div>
<p> </p>
</div>
<div>
<p style="margin-bottom:12.0pt">wilfredo</p>
<div>
<p>On Wed, Nov 12, 2008 at 2:01 PM, Dino Viehland <<a href="mailto:dinov@microsoft.com" target="_blank">dinov@microsoft.com</a>>
wrote:</p>
<div>
<div>
<p><span style="font-size:11.0pt;color:#1F497D">Instead of doing:</span></p>
<div>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p> ScriptScope scope =
engine.CreateScope();</p>
<p>
Console.WriteLine(DateTime.Now);</p>
<p> source.Execute(scope);</p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
</div>
<p><span style="font-size:11.0pt;color:#1F497D">do:</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p><span style="font-size:11.0pt;color:#1F497D">
source.Compile().Execute()</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p><span style="font-size:11.0pt;color:#1F497D">and your code should then run in
an optimized default scope for the code – just like it does at the command
line.</span></p>
<p><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> <a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>
[mailto:<a href="mailto:users-bounces@lists.ironpython.com" target="_blank">users-bounces@lists.ironpython.com</a>]
<b>On Behalf Of </b>Wilfredo Lugo<br>
<b>Sent:</b> Wednesday, November 12, 2008 9:29 AM<br>
<b>To:</b> <a href="mailto:users@lists.ironpython.com" target="_blank">users@lists.ironpython.com</a><br>
<b>Subject:</b> Re: [IronPython] Performance Issue</span></p>
</div>
<div>
<div>
<p> </p>
<p> </p>
<div>
<p>I run the script directly from ipy.exe and it basically behaves pretty
similar to python.exe (it always took one second more, but I could live with
that). Here is the output: </p>
</div>
<div>
<div>
<p> </p>
</div>
<div>
<p>$ date +%M:%S;./ipy.exe interpolate.py;date +%M:%S</p>
</div>
<div>
<p>17:36</p>
</div>
<div>
<p>WARNING: desired starting time (Tue Jan 01 00:00:21 2008) prior to time of
first</p>
</div>
<div>
<p> data point (Tue Jan 01 00:00:32 2008), current line: 9411</p>
</div>
<div>
<p> series: 3D</p>
</div>
<div>
<p>force==False, exiting..</p>
</div>
<div>
<p>17:41</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>So, the problem is on the DLR side. Any clues?</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>-------------------------------------------------------------------------</p>
</div>
<div>
<p>How does python.exe compare directly to ipy.exe? That is, try running
"date +%M:%S;ipy.exe interpolate.py;date +%M:%S" instead of using the
DLR hosting APIs.</p>
</div>
<div><pre>Thanks,</pre><pre> </pre><pre>Dave</pre><pre>From: <a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">users-bounces at lists.ironpython.com</a> [mailto:<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">users-bounces at lists.ironpython.com</a>] On Behalf Of Wilfredo Lugo</pre>
<pre>Sent: Wednesday, November 12, 2008 8:32 AM</pre><pre>To: <a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">users at lists.ironpython.com</a></pre><pre>Subject: [IronPython] Performance Issue</pre>
</div>
</div>
</div>
</div>
</div>
</div>
<p style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a></p>
</div>
<p> </p>
</div>
</div>
</div>
</div>
</div>
<p style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a></p>
</div>
<p> </p>
</div>
</div>
</div>
</div>
</div>
<p style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com" target="_blank">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a></p>
</div>
<p> </p>
</div>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.ironpython.com">Users@lists.ironpython.com</a><br>
<a href="http://lists.ironpython.com/listinfo.cgi/users-ironpython.com" target="_blank">http://lists.ironpython.com/listinfo.cgi/users-ironpython.com</a><br>
<br></blockquote></div><br></div>