[Cython] Debugging Cython programs with PDB

[E. Madison Bray]

On Mon, Dec 31, 2018 at 10:55 AM Prakhar Goel <newt0311 at gmail.com> wrote:
>
> I wanted to test the waters on this idea.
>
> The idea is to allow debugging Cython programs with PDB. This relies
> on making call-backs to trigger the sys.trace functionality every now
> and then. It is very similar to how profiling is handled so I figured
> that a bunch of the infra for this is already present. It involves a
> lot of overhead so it would only be enabled for a special debug mode
> but could be very useful in that capacity since it would allow using
> identical tools for both the Python and Cython modules. Additionally
> other people have built debugging tools on top of sys.settrace so some
> of the more advanced debugging facilities would also become at least
> partially available for Cython modules.
>
> It would be a fair bit of work. I'm not asking you to do this work of
> course! Just looking for some feedback here. The rough pieces as far
> as I can tell:
>
> > Adding in a flag for a special debug mode that calls Python's trace functions appropriately.
>
> > code-gen for all the trace calls.
>
> > Ideally we want some kind of wrapped call-frame that exports the current set of variables. Cython knows quite a bit about the variables currently in scope thanks to the cdef declarations. Ideally we should export these over to the Python side (with appropriate wrappers perhaps for the raw-c structs?). This is probably the most work but I'm hoping that getting a MVP here wouldn't be too hard...
>
> Thoughts? Comments? Suggestions?

I have had the exact same idea before, and I believe it to be
possible, but as you say non-trivial.  But at least you're not the
only one to have the idea so it can't be completely crazy :)

If you can get the basics working, you might also want to look into
supporting it via Python 3.7's sys.breakpointhook:
https://www.python.org/dev/peps/pep-0553/