Hi! Nice idea. Prakhar Goel schrieb am 29.12.18 um 15:02:

I wanted to test the waters on this idea.

The idea is to allow debugging Cython programs with PDB. This relies on making call-backs to trigger the sys.trace functionality every now and then. It is very similar to how profiling is handled so I figured that a bunch of the infra for this is already present. It involves a lot of overhead so it would only be enabled for a special debug mode but could be very useful in that capacity since it would allow using identical tools for both the Python and Cython modules. Additionally other people have built debugging tools on top of sys.settrace so some of the more advanced debugging facilities would also become at least partially available for Cython modules.

It would be a fair bit of work. I'm not asking you to do this work of course! Just looking for some feedback here. The rough pieces as far as I can tell:

...

Adding in a flag for a special debug mode that calls Python's trace functions appropriately.

...

code-gen for all the trace calls.

Most of this should already be in place. I wouldn't want a special flag for it, just extend the existing tracing support (which works for coverage analysis, mostly).

...

Ideally we want some kind of wrapped call-frame that exports the current set of variables. Cython knows quite a bit about the variables currently in scope thanks to the cdef declarations.

Not only those, it actually knows all defined names in the current scope (except for the global scope, which is extensible at runtime).

...

Ideally we should export these over to the Python side (with appropriate wrappers perhaps for the raw-c structs?). This is probably the most work but I'm hoping that getting a MVP here wouldn't be too hard...

There is support for locals(), which does most of what you need here. Regarding frames, Cython's tracing code uses frames already. I think the important steps would be: 1) Make sure there is only one frame per function execution. Currently, there are cases where we use one per source code line, which is an ugly hack in lack of proper Cython line number reporting support. This can be achieved by finalising the lnotab support in PR-93, and then cleaning up the way we create code objects and frames for tracing. https://github.com/cython/cython/pull/93 2) Extend the meta-data that the code objects (which we already create, just mostly empty) provide for each of the functions, since that is where PDB gets its introspection details from. 3) Make the frame refer to the current "locals", Preferably in a way that only generates the dict on request, not for all frames. In the worst case, that could be something to enable with a C compile time define, as we do for tracing in general. 4) Trial and error fixing. :) So, yeah, there is a bit of work involved, but it seems doable and worth doing. Are you interested in giving this a try? Stefan