[IronPython] Debugging support PythonEngine

[Shri Borde]

I have opened this bug - http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPython&WorkItemId=2248 - to track the issue with stepping in. We are not emitting any line number information for IL offset 0.

From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Shri Borde
Sent: Wednesday, August 16, 2006 12:29 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Debugging support PythonEngine

We have started doing some work for improving debugging as shown by the VSIP sample (http://blogs.msdn.com/aaronmar/archive/2006/02/16/533273.aspx). However, it still needs more work for it to be production-quality. By tools support, I was referring more to VSIP sample. We don't have a concrete timeline for when we will be able to improve it further.

Btw, using your sample script below, in most cases, I am able to step from Python code into C# code in IronPython.dll if I hit F11 in VS. However, when IronPython.dll calls the method Add, we step into native code for a while before actually reaching the source code for Add. This looks like something we can fix by changing the debug information we generate. After doing this fix, it would be possible to mark IronPython.dll as library code (either using DebuggerNonUserCodeAttribute or through some other scheme), and be able to step from Python code into a Python method.

x = 1
y = 2

def Add(a, b):
    return a + b

z = Add(x, y) ß Stepping in here takes you to IronPython.dll source code, then native code, and then the source code for Add
print z


From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Kristof Wagemans
Sent: Saturday, August 12, 2006 1:38 AM
To: 'Discussion of IronPython'
Subject: Re: [IronPython] Debugging support PythonEngine

Thanks for the info. It's good to hear that there will be improvement possible in this area in the future. Although, the way I interpret your explanation, it's not going to be anytime soon (or even this year), because you need enhancements to the underlying platform to make this possible.

I've experimented a bit with System.Diagnostics.DebuggerNonUserCodeAttribute to see if it wasn't possible to skip stepping into IronPython code. Unfortunately, the point where I get stuck in the assembly instructions is in generated code. I don't understand this part of the code well enough to know if it's possible to make the debugger skip these lines. Maybe this is why you need the changes to the platform?
________________________________
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Shri Borde
Sent: Friday 11 August 2006 21:04
To: Discussion of IronPython
Subject: Re: [IronPython] Debugging support PythonEngine

Yes, we will definitely be working on improving the debugging support as it is a critical part of the development process.  However, full support will need work in all parts of the tool chain including VS.

Python local variables are implemented as normal MSIL variables (except in cases like closures). Hence, VS is able to display them.

Currently, the best way to debug Python functions in VS while using PythonEngine would be to enable EngineOptions.ClrDebuggingEnable, open the PY file in VS and put a breakpoint where you want. Stepping in and out of Python functions will step you through methods in IronPython.dll .

From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Kristof Wagemans
Sent: Friday, August 11, 2006 12:36 AM
To: 'Discussion of IronPython'
Subject: Re: [IronPython] Debugging support PythonEngine

Are there plans to improve the debugging experience in the future or are you at a point where it is "as good as it gets"? Being able to debug makes writing and using python scripts a lot easier.
Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary.
Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. I'm not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#.

________________________________
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Shri Borde
Sent: Friday 11 August 2006 0:30
To: Discussion of IronPython
Cc: Glenn Hackney
Subject: Re: [IronPython] Debugging support PythonEngine

If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about the dictionary. If you use PythonEngine.CreateOptimizedModule, most global variables are implemented using CLR statics, and so VS may be able to display them for you. Global variables added using an exec statement will still not be visible.

If EngineOptions.ClrDebuggingEnabled is false, we will use System.Reflection.Emit.DynamicMethod. This does not support debug information tracking at all, and you will not even be able to see function variables.

So If EngineOptions.ClrDebuggingEnabled is will improve your debugging experience, but it wont give you a perfect experience.

________
Do you want to help develop Dynamic languages on CLR?<http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038> (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038)
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Kristof Wagemans
Sent: Thursday, August 10, 2006 2:10 PM
To: Discussion of IronPython
Subject: [IronPython] Debugging support PythonEngine

I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems.

PythonEngine _pe;
EngineOptions options = new EngineOptions();
options.ClrDebuggingEnabled = true;
_pe = new PythonEngine(options);
_pe.ExecuteFile(@"  <script>  ");

Test script:

x = 1
y = 2

def Add(a, b):
    return a + b

z = Add(x, y)
print z


I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of the global variables.
When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times through the assembly instructions I can return to the original source code. Inside the function I can see the values of the function variables.
I have tried debugging ipy.exe with the script and there I can see the global variables, but I still have the problem with stepping into a function. In ipy.exe the script file is executed in a different way. Using the same method I can also see the global variables with my PythonEngine instance. I apparently don't need to set ClrDebuggingEnabled in this case.

PythonEngine _pe;
_pe = new PythonEngine();
OptimizedEngineModule engineModule = _pe.CreateOptimizedModule(@"  <script>  ", "__main__", true);
engineModule.Execute();

Are you required to use an OptimizedEngineModule to be able to debug completely? Am I forgetting some settings for debugging? Can I step directly into a function without getting into the assembly instructions?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20060817/a316c367/attachment.html>