[IronPython] IronPython 2.6.1 & ctypes, C library, and debugging with VS2008
TP
wingusr at gmail.com
Mon Jul 19 07:47:33 CEST 2010
On Sun, Jul 18, 2010 at 2:38 PM, Dino Viehland <dinov at microsoft.com> wrote:
> TP wrote:
>> I'm using IronPython 2.6.1 for Net 2.0 and VS2008 on Windows XP SP3.
>>
>> I have a python script that lets me access the Leptonica Image
>> Processing C library (http://leptonica.com/) using the ctypes module.
>>
>> Everything seems to work fine with cpython 2.6. I can correctly load
>> leptonlib.dll, create leptonica PIX images, manipulate them, and even
>> display them using PyQT. I can use VS2008 to put breakpoints on
>> leptonica C functions and step through the C code in the VS2008
>> debugger.
>>
>> I thought I'd give IronPython a try since it also has ctypes, but I
>> ran into a number of problems.
>>
>> The leptonlib.dll must be loading since I am able to correctly get
>> back the library's version string. That is the following works:
>>
>> _getLeptonlibVersion = _leptonlib.getLeptonlibVersion
>> _getLeptonlibVersion.restype = ctypes.c_void_p
>> addr = _getLeptonlibVersion()
>> version = ctypes.string_at(addr)
>>
>> However the following code doesn't seem to work under IronPython 2.6.1:
>>
>> pix = Pix(dimensions=(10,20,1))
>>
>> where essentially the following is called:
>>
>> _pix = ctypes.c_void_p()
>> _pix.value = _leptonlib.pixCreate(width, height, depth)
>> self._pix = _pix
>>
>> and I want to treat _pix as an opaque ptr I just hand back to
>> leptonica whenever it needs it. For example:
>>
>> def height(self):
>> """Get height of pix in pixels."""
>> return _leptonlib.pixGetHeight(self._pix)
>>
>> Which works with cpython but returns something other than 20 with
>> IronPython.
>>
>> By specifying "-X:Debug -X:FullFrames -X:Tracing leptonica.py" as the
>> arguments to C:\Program Files\IronPython 2.6\ipy.exe I can get
>> IronPython to correctly stop at places in my script where I put:
>>
>> import pdb
>> pdb.set_trace()
>>
>> when I debug leptonlibd.dll with VS2008. However, my breakpoints on
>> leptonica's C functions never seem to get hit. This works fine if I
>> instead use python26.exe as command to launch when debugging.
>
> You should only need -X:Debug to get debugging under VS. pdb uses a
> more Pythonic form of debugging but there's no support for it in VS.
Let me be clear here. I am using the VS2008 Solution that I use to
create leptonlib.dll. I am debugging that dll by right-clicking its
project and setting its Configuration Properties | Debugging tab to:
Command: C:\Program Files\IronPython 2.6\ipy.exe
Arguments: -X:Debug -X:FullFrames -X:Tracing -i leptonica.py
Working Directory: C:\leptonica\
If I only use -X:Debug as you suggest I get the following error:
Traceback (most recent call last):
File "leptonica.py", line 458, in <module>
File "leptonica.py", line 176, in __init__
File "C:\Program Files\IronPython 2.6\Lib\pdb.py", line 1220, in set_trace
AttributeError: 'module' object has no attribute '_getframe'>>>
Googling, I determined that I needed to add at least -X:FullFrames,
and by trial and error I found I also needed -X:Tracing. (I wasn't
able to find any documentation on ipy.exe's command line switches? I
just ran "ipy.exe -h" to dump out the short help description and took
a wild guess at which might be useful)
>>
>> Ideally I like to have the VS Debugger stop in leptonlibd.dll's
>> pixCreate() C function just before it returns its value so I can
>> compare that to what I get back on the IronPython side.
>>
>> So how's does one use VS2008 to step through C functions in DLLs that
>> are loaded by IronPython and the ctypes module?
>
> Do you have symbols (.PDB files) for leptonlibd.dll? You'll need symbols
> to be able to step through the C code. You can still probably set a breakpoint
> in the function because it's DLL exported - I think you can debug->new
> breakpoint and enter leptonlibd!pixCreate. Even w/o symbols you could
> step through the assembly.
leptonlibd.dll is created with the C7 Compatible (/Z7) compiler
switch. This embeds the debugging info directly in the DLL so no .pdb
file is produced. It also does NOT use pre-compiled headers.
I might also point out that the same exact .dll will hit breakpoints
if debugged with python26.exe rather than ipy.exe. Perhaps since
ipy.exe is built on top of .NET the VS2008 debugger handles any .dll's
loaded by it differently? I know for example that the VS2008 debugger
didn't like trying to run the IronPython for NET 4.0 version of
ipy.exe (I gather you have to use VS2010 if you want to do that).
Trying to set a breaking at leptonlibd!pixCreate didn't work.
>>
>> Secondly, am I using ctypes wrong, and does my code only work by
>> happenstance for cpython.
>
> I don't see anything particularly wrong on your side and this looks like
> a pretty simple call. I would assume the C functions are defined as:
>
> void* pixCreate(int width, int height, int depth);
> int pixGetHeight(void* pixel);
That's correct.
>
> I would hope we're getting all of this right as it seems like simple stuff
> that should be tested somewhere. My first guess would be maybe we're
> getting the calling convention wrong. Maybe it needs to be a WinDLL instead
> of a CDLL? Maybe there's a check in the Python code for sys.platform which
> is looking for win32 and uses WinDLL to open the DLL instead of CDLL on
> Windows?
More information on the problem:
I decided to explicitly set the restype even though it's the default
return type. I get the same exact behavior as my original code. Works
with cpython, doesn't work with IronPython 2.6.1.
If I have with the following python code:
_pix = ctypes.c_void_p()
_pixCreate = _leptonlib.pixCreate
_pixCreate.restype = ctypes.c_int
import pdb
pdb.set_trace()
_pix.value = _pixCreate(width, height, depth)
I can do the following in the Command Window after pdb breaks into the script:
> leptonica.py(180)__init__()
-> _pix.value = _pixCreate(width, height, depth)
(Pdb) n
> leptonica.py(181)__init__()
-> self._pix = _pix
(Pdb) _pix.value
*** AttributeError: 'cell' object has no attribute 'value'
This is a bit strange since I just assigned to _pix.value.
Investigating further:
(Pdb) _pix
<cell at 43: c_void_p object at 44>
So instead of assigning to _pix.value (that is changing an attribute
of _pix), IronPython is clobbering _pix?
If I instead separately assign the result of pixCreate() to a
temporary variable:
width, height, depth = dimensions
_pix = ctypes.c_void_p()
_pixCreate = _leptonlib.pixCreate
_pixCreate.restype = ctypes.c_int
import pdb
pdb.set_trace()
addr = _pixCreate(width, height, depth)
_pix.value = addr
Once pdb stops the script:
> leptonica.py(178)__init__()
-> addr = _pixCreate(width, height, depth)
(Pdb) n
> leptonica.py(179)__init__()
-> _pix.value = addr
(Pdb) addr
<cell at 43: int object at 44>
(Pdb) type(addr)
<type 'cell'>
(Pdb) dir(addr)
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__',
'__getattribute__', '__hash__', '__init__', '__ne__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', 'cell_contents']
(Pdb) type(addr.cell_contents)
<type 'int'>
(Pdb) addr.cell_contents
77491296
So instead of returning an int, ctypes under IronPython is returning a
"cell" in this case for some reason?
Compare this to what I get with cpython. If I change the leptonlib.dll
project Configuration Properties | Debugging tab to:
Command: c:\Python26\python26.exe
Arguments: -i leptonica.py
Working Directory: C:\leptonica\
Debugging by typing F5 (or choosing Debug > Start Debugging) without
changing leptonica.py or leptonlibd.dll in any way, I then hit all my
breakpoints set in leptonlibd.dll. I get warning messages about
python.exe having stopped for each breakpoint and I just click the
Continue button to close them. After I type F5 to get past all my C
function breakpoints, pdb stops my script with:
> leptonica.py(178)__init__()
-> addr = _pixCreate(width, height, depth)
(Pdb) n
> leptonica.py(179)__init__()
-> _pix.value = addr
(Pdb) addr
19492760
(Pdb) hex(addr)
'0x1296f98'
and 0x1296f98 matches what I see for the return value of pixCreate()
from the VS2008 debugger.
I also tried debugging my leptonica.py script by following the
directions in the IronPython tutorial. I made a new Solution with the
following Debugging properties:
Command: C:\Program Files\IronPython 2.6\ipy.exe
Arguments: -X:Debug -i leptonica.py
Working Directory: C:\leptonica\
I added my leptonlib.vcprog file to this solution by right-clicking it
and choosing Add > Existing Project. I again set a breakpoint at the C
function pixCreate().
I also set various breakpoints in my leptonica.py script.
Pressing F5 to Start Debugging, I now hit my leptonica.py breakpoints
but still don't hit any C function breakpoints.
>From the locals window I can see that addr is:
addr 0x00bf6ea0 object {int}
Looking in a Memory window at "addr" I see:
0x06AAB590 79332d70 00bf6ea0 00000000 793042f4 00000001 33b4c9bc
p-3y n¿.....ôB0y.....É´3
So there's something else at "addr", but the next int is 0x00bf6ea0 again.
The memory at 0x00bf6ea0 looks correct:
0x00BF6EA0 0a 00 00 00 14 00 00 00 01 00 00 00 01 00 00 00 01 00
00 00 00 00 ......................
0x00BF6EB6 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f8 6e bf 00 ..................øn¿.
0x00BF6ECC fd fd fd fd 10 00 0b 00 f2 01 0c 00 80 6e bf 00 00 00
00 00 00 00 ýýýý....ò...€n¿.......
0x00BF6EE2 00 00 00 00 00 00 50 00 00 00 01 00 00 00 4d 00 00 00
fd fd fd fd ......P.......M...ýýýý
At least it looks to me like those are indeed ints for width, height &
depth which is what a PIX starts out with.
After:
_pix.value = addr
I see this in the locals window:
_pix "ctypes.c_void_p instance" object
{IronPython.NewTypes.IronPython.Modules.SimpleCData_4$4}
In the Immediate window I get:
_pix
"ctypes.c_void_p instance"
base {IronPython.Modules.CTypes.SimpleCData}: "ctypes.c_void_p instance"
.class: PythonType: "c_void_p"
.dict: null
.slots_and_weakref: null
But I am unable to figure out how to tell what the "value" of _pix is?
More information about the Ironpython-users
mailing list