[IronPython] Performance Issue
Wilfredo Lugo
wilfredo.lugo at gmail.com
Thu Nov 13 00:24:11 CET 2008
I set the ModuleBuiltins compiler option but the behavior is the same. The
only difference from the command line is that I am calling a python function
from the DLR, and on the command line I am just calling the method from
__main__. Do you think could be an issue? The other thing is that the
python script relies heavily on I/O, input file is in the range of 18Mb and
the data is stored on another file. Does the DLR handles I/O differently
than the command line?
regards,
wilfredo
On Wed, Nov 12, 2008 at 5:39 PM, Dino Viehland <dinov at microsoft.com> wrote:
> The only thing that immediately pops out at me as being different is we
> also set ModuleOptions.ModuleBuiltins in the command line case – but I
> wouldn't expect it to make such a large difference. But try setting that
> option as well and let's see what happens.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Wilfredo Lugo
> *Sent:* Wednesday, November 12, 2008 12:16 PM
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Performance Issue
>
>
>
> Thanks. Performance improved, but still over two minutes. Here is the
> latest code:
>
>
>
> static void Main(string[] args)
>
> {
>
> ScriptRuntime runtime = Python.CreateRuntime();
>
> ScriptEngine engine = Python.CreateEngine();
>
> ScriptSource source =
> engine.CreateScriptSourceFromFile("interpolate.py");
>
> PythonCompilerOptions options = (PythonCompilerOptions)
> engine.GetCompilerOptions();
>
> options.Module |= ModuleOptions.Optimized;
>
> Console.WriteLine(DateTime.Now);
>
> CompiledCode compiled = source.Compile(options);
>
> compiled.Execute();
>
> ScriptScope scope = compiled.DefaultScope;
>
> Microsoft.Func<string, string, string, string, int, int> func =
> scope.GetVariable<Microsoft.Func<string, string, string, string, int,
> int>>("interpolate_start");
>
> Console.WriteLine(func("3-day-data.txt", "3-day-output.txt",
> "2008-01-01 00:00:21","2008-01-03 23:59:08",60).ToString());
>
> Console.WriteLine(DateTime.Now);
>
> Console.ReadLine();
>
> }
>
>
>
> Output :
>
> 11/12/2008 4:10:10 PM
>
> WARNING: desired starting time (Tue Jan 01 00:00:21 2008) prior to time of
> first
>
> data point (Tue Jan 01 00:00:32 2008), current line: 9411
>
> series: 3D
>
> force==False, exiting..
>
> 0
>
> 11/12/2008 4:12:45 PM
>
>
>
>
>
> On Wed, Nov 12, 2008 at 4:01 PM, Dino Viehland <dinov at microsoft.com>
> wrote:
>
> Oh, sorry, I missed a step… You'll need to call
> ScriptEngine.GetCompilerOptions(). Cast that to a PythonCompilerOptions,
> and then do options.Module |= ModuleOptions.Optimized; Finally do
> source.Compile(options) and then it should give you the optimized code.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Wilfredo Lugo
> *Sent:* Wednesday, November 12, 2008 11:45 AM
>
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] Performance Issue
>
>
>
> Thanks!.
>
>
>
> I was able to get the default scope from CompiledCode, but I am still
> getting the same performance. Here is the latest code :
>
> static void Main(string[] args)
>
> {
>
> ScriptRuntime runtime = Python.CreateRuntime();
>
> ScriptEngine engine = Python.CreateEngine();
>
> ScriptSource source =
> engine.CreateScriptSourceFromFile("interpolate.py");
>
> Console.WriteLine(DateTime.Now);
>
> CompiledCode compiled = source.Compile();
>
> compiled.Execute();
>
> ScriptScope scope = compiled.DefaultScope;
>
> Microsoft.Func<string, string, string, string, int, int> func =
> scope.GetVariable<Microsoft.Func<string, string, string, string, int,
> int>>("interpolate_start");
>
> Console.WriteLine(func("3-day-data.txt", "3-day-output.txt",
> "2008-01-01 00:00:21","2008-01-03 23:59:08",60).ToString());
>
> Console.WriteLine(DateTime.Now);
>
> Console.ReadLine();
>
> }
>
>
>
> This is the latest output :
>
> 11/12/2008 3:13:01 PM
>
> WARNING: desired starting time (Tue Jan 01 00:00:21 2008) prior to time
> of first
>
> data point (Tue Jan 01 00:00:32 2008), current line: 9411
>
> series: 3D
>
> force==False, exiting..
>
> 0
>
> 11/12/2008 3:16:41 PM
>
>
>
> Is there could be a problem on how I am calling the interpolation function?
> I mean from python.exe I do something like this:
>
>
>
> if __name__ == '__main__':
>
> Interpolate('3-day-data.txt', '3-day-output-original.txt', '2008-01-01
> 00:00:
>
> 21', '2008-01-03 23:59:08', intvl=60, maxgap = 5, stacked = True,
> stackedCol = 2
>
> , tformat = '%Y-%m-%d %H:%M:%S', debug = False).start()
>
>
>
> To be able to call the script from the DLR I have something like this:
>
>
>
> def interpolate_start(inFile, outFile, startDate, endDate, intvl):
>
> Interpolate(inFile, outFile,startDate,endDate,intvl, maxgap = 5, stacked
> = Tr
>
> ue, stackedCol = 2, tformat = '%Y-%m-%d %H:%M:%S', debug = False).start()
>
> return 0
>
>
>
> That's the only difference I found on how the script is called from python
> and DLR.
>
>
>
> Any comments?
>
>
>
> regards,
>
>
>
> wilfredo
>
>
>
> On Wed, Nov 12, 2008 at 2:56 PM, Dino Viehland <dinov at microsoft.com>
> wrote:
>
> CompiledCode exposes the default scope it executes in via the DefaultScope
> property. So hold onto the result of source.Compile and after executing the
> code you can grab the scope and use it to get your variable.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Wilfredo Lugo
> *Sent:* Wednesday, November 12, 2008 10:25 AM
> *To:* Discussion of IronPython
>
>
> *Subject:* Re: [IronPython] Performance Issue
>
>
>
> Thanks.
>
>
>
> But then If I don't use my defined scope, how I could fill my function
> delegate?
>
>
>
> Right now I am using :
>
>
>
> Microsoft.Func<string, string, string, string, int, int> func =
> scope.GetVariable<Microsoft.Func<string, string, string, string, int,
> int>>("interpolate_start");
>
>
>
> If I use engine.GetVariables<Func<....>() I still need to pass a
> ScriptScope to it. How do I get the default scope (from ScriptEngine or
> ScriptSource) to be able to get the delegate?
>
>
>
> regards,
>
>
>
> wilfredo
>
> On Wed, Nov 12, 2008 at 2:01 PM, Dino Viehland <dinov at microsoft.com>
> wrote:
>
> Instead of doing:
>
>
>
> ScriptScope scope = engine.CreateScope();
>
> Console.WriteLine(DateTime.Now);
>
> source.Execute(scope);
>
>
>
> do:
>
>
>
> source.Compile().Execute()
>
>
>
> and your code should then run in an optimized default scope for the code –
> just like it does at the command line.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Wilfredo Lugo
> *Sent:* Wednesday, November 12, 2008 9:29 AM
> *To:* users at lists.ironpython.com
> *Subject:* Re: [IronPython] Performance Issue
>
>
>
>
>
> I run the script directly from ipy.exe and it basically behaves pretty
> similar to python.exe (it always took one second more, but I could live with
> that). Here is the output:
>
>
>
> $ date +%M:%S;./ipy.exe interpolate.py;date +%M:%S
>
> 17:36
>
> WARNING: desired starting time (Tue Jan 01 00:00:21 2008) prior to time of
> first
>
> data point (Tue Jan 01 00:00:32 2008), current line: 9411
>
> series: 3D
>
> force==False, exiting..
>
> 17:41
>
>
>
> So, the problem is on the DLR side. Any clues?
>
>
>
> -------------------------------------------------------------------------
>
> How does python.exe compare directly to ipy.exe? That is, try running
> "date +%M:%S;ipy.exe interpolate.py;date +%M:%S" instead of using the DLR
> hosting APIs.
>
> Thanks,
>
>
>
> Dave
>
> From: users-bounces at lists.ironpython.com <http://lists.ironpython.com/listinfo.cgi/users-ironpython.com> [mailto:users-bounces at lists.ironpython.com <http://lists.ironpython.com/listinfo.cgi/users-ironpython.com>] On Behalf Of Wilfredo Lugo
>
> Sent: Wednesday, November 12, 2008 8:32 AM
>
> To: users at lists.ironpython.com <http://lists.ironpython.com/listinfo.cgi/users-ironpython.com>
>
> Subject: [IronPython] Performance Issue
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20081112/51b68436/attachment.html>
More information about the Ironpython-users
mailing list