[Python-Dev] Interactive Debugging of Python

[Mark Hammond]

[Guido writes...]
> This topic sounds mostly unrelated to the stackless discussion -- in

Sure is - I just saw that as an excuse to try and hijack it <wink>

> Some issues:
>
>   - The slots containing local variables may be renumbered after

Generally, I think we could make something very useful even with a number
of limitations.  For example, I would find a first cut completely
acceptable and a great improvement on today if:

* Only the function at the top of the stack can be recompiled and have the
code reflected while executing.  This function also must be restarted after
such an edit.  If the function uses global variables or makes calls that
restarting will screw-up, then either a) make the code changes _before_
doing this stuff, or b) live with it for now, and help us remove the
limitation :-)

That may make the locals being renumbered easier to deal with, and also
remove some of the problems you discussed about editing functions below the
top.

> What kind of limitations do other systems that support modifying a
> "live" program being debugged impose?  Only allowing modification of

I can only speak for VC, and from experience at that - I havent attempted
to find documentation on it.

It accepts most changes while running.  The current line is fine.  If you
create or change the definition of globals (and possibly even the type of
locals?), the "incremental compilation" fails, and you are given the option
of continuing with the old code, or stopping the process and doing a full
build.

When the debug session terminates, some link process (and maybe even
compilation?) is done to bring the .exe on disk up to date with the
changes.

If you do wierd stuff like delete the line being executed, it usually gives
you some warning message before either restarting the function or trying to
pick a line somewhere near the line you deleted.  Either way, it can screw
up, moving the "current" line somewhere else - it doesnt crash the
debugger, but may not do exactly what you expected.  It is still a _huge_
win, and a great feature!

Ironically, I turn this feature _off_ for Python extensions.  Although
changing the C code is great, in 99% of the cases I also need to change
some .py code, and as existing instances are affected I need to restart the
app anyway - so I may as well do a normal build at that time.  ie, C now
lets me debug incrementally, but a far more dynamic language prevents this
feature being useful ;-)

> the function at the top of the stack might eliminate some problems,
> although there are still ways to mess up.  The value stack is not
> always empty even when we only stop at statement boundaries

If we forced a restart would this be better?  Can we reliably reset the
stack to the start of the current function?

> I've been thinking a bit about this.  Function objects now have
> mutable func_code attributes (and also func_defaults), I think we can
> use this.
>
> The hard part is to do the analysis needed to decide which functions
> to recompile!  Ideally, we would simply edit a file and tell the
> programming environment "recompile this".  The programming environment
> would compare the changed file with the old version that it had saved
> for this purpose, and notice (for example) that we changed two methods
> of class C.  It would then recompile those methods only and stuff the
> new code objects in the corresponding function objects.

If this would work for the few changed functions/methods, what would the
impact be of doing it for _every_ function (changed or not)?  Then the
analysis can drop to the module level which is much easier.  I dont think a
slight performace hit is a problem at all when doing this stuff.

> One option would be to actually change the semantics of the class and
> def statements so that they modify an existing class or function
> rather than using assignment.  Effectively, this proposal would change
> the semantics of
>
>     class A:
>         ...some code...
>
>     class A:
>         ...some more code...
>
> to be the same as
>
>     class A:
>         ...more code...
>         ...some more code...

Or extending this (didnt this come up at the latest IPC?)
# .\package\__init__.py
class BigMutha:
  pass

# .\package\something.py
class package.BigMutha:
  def some_category_of_methods():
    ...

# .\package\other.py
class package.BigMutha:
  def other_category_of_methods():
    ...
[Of course, this wont fly as it stands; just a conceptual possibility]

> So perhaps these new semantics should only be invoked when a special
> "reload-compile" is asked for...  Or perhaps the programming
> environment could do this through source parsing as I proposed
> before...