Utilities for easier debugging
Minimal strawman proposal. New keyword debug. debug EXPRESSION Executes EXPRESSION when in debug mode. debug context Prints all the variables of the enclosing closure and all the variable names accessed within that block. For example, if in foo you access the global variable spam, spam would be printed. The format would be: variableName: value variableTwo: value where "value" is the repr() of the variable. Separated by new lines. The exact output format would not be part of the spec. ?identifier would print "identifier: value." Repr as before. Using this in non-debug mode emits a warning. ?identifier.property.property is also valid. A new property descriptor on the global variable, “debugger.” This is an alias for importing PDB and causing the debugger to pause there. The behavior of this descriptor in non-debug mode is TBD. Debug mode may be specified per-module at interpreter launch.
James Lu wrote:
Wouldn't be dropping into PDB more efficient here? Anyway you could still use something like `inspect.stack()` and inspect the closure of the function. Together with a dedicated variable (from OS or via argv) you could distinguish between debug and non-debug mode.
This you can do in Python 3.8 via f-strings: [`f'{indentifier=}`](https://docs.python.org/3.8/whatsnew/3.8.html#f-strings-now-support-for-quic...)
This is possible via [`breakpoint`](https://docs.python.org/3/library/functions.html#breakpoint).
The behavior of this descriptor in non-debug mode is TBD. Debug mode may be specified per-module at interpreter launch.
On Jul 27, 2019, at 22:19, James Lu <jamtlu@gmail.com> wrote:
You really mean expression, not statement here? That doesn’t seem very useful. Sure, some expressions are used for side effects, but most are used for values, while most things you do want to execute for side effects are statements (and not just expression statements). Also, why is it not sufficient to use if DEBUG: expr (which already works, and can take a statement, even a compound statement)?
debug context
Prints all the variables of the enclosing closure and all the variable names accessed within that block. For example, if in foo you access the global variable spam, spam would be printed. The format would be:
What do you want this for? The previous statement seems like it would only be useful for non-interactive use, but this one seems like it would be useless there. If I’m trying to debug a daemon or a webservice or a GUI app or something, I probably want the debug output to go to my logs, or at least stderr, rather than stdout. An inspect function that gave a string I could print (or a stack of functions similar to the traceback module ones, where the lowest one gives a dict, but there are convenience helpers above that to give a pre-formatted string instead, or to just dump that string directly) seems like it would be more useful (although still, the top one seems like it should dump to stderr rather than stdout). How does the interpreter know which globals are accessed from the current frame? Looking at the frame’s code object’s names gets you about 70% of the way there, but that’s the names of all identifiers that aren’t local/cell/free vars, which includes more than global vars, and it doesn’t include globals looked up indirectly, and it includes globals that are only referenced in dead code, and so on. Plus, if you wanted that, you could write it today as a 3-line function using inspect; no need for a new statement. If you want the interpreter to keep track of things at runtime instead of relying on the statically available information, it seems like the only way to do that is to have debug mode do the equivalent of a per-opcode trace function or an all-variables watchpoint so it can maintain the information in case you ask for it. This would probably be pretty slow. Also, most of the globals used in most functions are the top-level functions and classes used in the function. Since these don’t usually change at runtime, and their reprs don’t tell you anything useful beyond their name, what’s the point in dumping them? Also, what if you have a variable named context? That’s hardly rare as a name, and your two statements seem to be ambiguous in that case. Also, except for this being a statement that needs a new keyword and a new context-sensitive keyword instead of a plain old function you stick in builtins named debug_context(), it seems like any feasible version of this could already be written and used in Python today. (The first one, using static frame info, should be about 3 lines of inspect calls.)
?identifier
would print "identifier: value." Repr as before. Using this in non-debug mode emits a warning.
This would break iPython’s improved interactive console, which already uses this syntax to provide a similar feature. And in non-interactive use, I can’t see any use for this. Especially since it would have to be wrapped in #if DEBUG, so the one-liner brevity doesn’t seem to buy you anything.
?identifier.property.property
is also valid.
Why not just allow anything that’s valid as a target, like identifier[expr]? Or even just any expression at all? Is there an advantage to defining and using a similar but more limited syntax here?
A new property descriptor on the global variable, “debugger.” This is an alias for importing PDB and causing the debugger to pause there.
You want this added to every module’s globals? Why not put it in builtins instead? And why isn’t the existing builtin breakpoint sufficient? I can only see two differences. First, you have to call breakpoint as a function, not just reference it, but I’m having trouble imagining when that’s a problem. Second, your magic property would always use pdb, even if the breakpoint hook has been set to another debugger, but that seems like it’s just always worse when it’s different.
Debug mode may be specified per-module at interpreter launch.
Wait, this new debug mode is separate from the existing debug mode? Then how do they interact? It might help to show us how all this stuff would actually be used in debugging some realistic code in some realistic scenario, maybe by giving us some code with functions whose bodies are like `#80 lines of complicated stuff to get the value for spam` followed by `?spam`, and then a fake console dump or interactive session log showing the program being started and displaying its output?
On Sun, Jul 28, 2019 at 01:19:38AM -0400, James Lu wrote:
That's what assert does, in part. Since print is now a function, not a statement, you can do this: assert print(expression) or True to get the same effect.
debug context
So "context" is also going to be keyword? That's two new keywords, breaking twice as much existing code: anything that uses "debug", anything that uses "context". By the way, you know that Python has a read-only global variable that tells you whether you are in debug mode? You can write a function to display anything you like, and wrap it in a test like this: if __debug__: display(locals()) -- Steven
Steven D'Aprano wrote:
Oh interesting, I wasn't aware of this global. For anyone else just finding out about it, here's the [documentation](https://docs.python.org/3.9/library/constants.html#__debug__). Thanks for mentioning it, I feel like I end up learning something new in almost every python-ideas and python-dev thread, even if the proposal from the author isn't accepted. I should've subscribed a while ago (: Is there any other way to toggle/specify debug status other than the use of `-O`?
James Lu wrote:
Wouldn't be dropping into PDB more efficient here? Anyway you could still use something like `inspect.stack()` and inspect the closure of the function. Together with a dedicated variable (from OS or via argv) you could distinguish between debug and non-debug mode.
This you can do in Python 3.8 via f-strings: [`f'{indentifier=}`](https://docs.python.org/3.8/whatsnew/3.8.html#f-strings-now-support-for-quic...)
This is possible via [`breakpoint`](https://docs.python.org/3/library/functions.html#breakpoint).
The behavior of this descriptor in non-debug mode is TBD. Debug mode may be specified per-module at interpreter launch.
On Jul 27, 2019, at 22:19, James Lu <jamtlu@gmail.com> wrote:
You really mean expression, not statement here? That doesn’t seem very useful. Sure, some expressions are used for side effects, but most are used for values, while most things you do want to execute for side effects are statements (and not just expression statements). Also, why is it not sufficient to use if DEBUG: expr (which already works, and can take a statement, even a compound statement)?
debug context
Prints all the variables of the enclosing closure and all the variable names accessed within that block. For example, if in foo you access the global variable spam, spam would be printed. The format would be:
What do you want this for? The previous statement seems like it would only be useful for non-interactive use, but this one seems like it would be useless there. If I’m trying to debug a daemon or a webservice or a GUI app or something, I probably want the debug output to go to my logs, or at least stderr, rather than stdout. An inspect function that gave a string I could print (or a stack of functions similar to the traceback module ones, where the lowest one gives a dict, but there are convenience helpers above that to give a pre-formatted string instead, or to just dump that string directly) seems like it would be more useful (although still, the top one seems like it should dump to stderr rather than stdout). How does the interpreter know which globals are accessed from the current frame? Looking at the frame’s code object’s names gets you about 70% of the way there, but that’s the names of all identifiers that aren’t local/cell/free vars, which includes more than global vars, and it doesn’t include globals looked up indirectly, and it includes globals that are only referenced in dead code, and so on. Plus, if you wanted that, you could write it today as a 3-line function using inspect; no need for a new statement. If you want the interpreter to keep track of things at runtime instead of relying on the statically available information, it seems like the only way to do that is to have debug mode do the equivalent of a per-opcode trace function or an all-variables watchpoint so it can maintain the information in case you ask for it. This would probably be pretty slow. Also, most of the globals used in most functions are the top-level functions and classes used in the function. Since these don’t usually change at runtime, and their reprs don’t tell you anything useful beyond their name, what’s the point in dumping them? Also, what if you have a variable named context? That’s hardly rare as a name, and your two statements seem to be ambiguous in that case. Also, except for this being a statement that needs a new keyword and a new context-sensitive keyword instead of a plain old function you stick in builtins named debug_context(), it seems like any feasible version of this could already be written and used in Python today. (The first one, using static frame info, should be about 3 lines of inspect calls.)
?identifier
would print "identifier: value." Repr as before. Using this in non-debug mode emits a warning.
This would break iPython’s improved interactive console, which already uses this syntax to provide a similar feature. And in non-interactive use, I can’t see any use for this. Especially since it would have to be wrapped in #if DEBUG, so the one-liner brevity doesn’t seem to buy you anything.
?identifier.property.property
is also valid.
Why not just allow anything that’s valid as a target, like identifier[expr]? Or even just any expression at all? Is there an advantage to defining and using a similar but more limited syntax here?
A new property descriptor on the global variable, “debugger.” This is an alias for importing PDB and causing the debugger to pause there.
You want this added to every module’s globals? Why not put it in builtins instead? And why isn’t the existing builtin breakpoint sufficient? I can only see two differences. First, you have to call breakpoint as a function, not just reference it, but I’m having trouble imagining when that’s a problem. Second, your magic property would always use pdb, even if the breakpoint hook has been set to another debugger, but that seems like it’s just always worse when it’s different.
Debug mode may be specified per-module at interpreter launch.
Wait, this new debug mode is separate from the existing debug mode? Then how do they interact? It might help to show us how all this stuff would actually be used in debugging some realistic code in some realistic scenario, maybe by giving us some code with functions whose bodies are like `#80 lines of complicated stuff to get the value for spam` followed by `?spam`, and then a fake console dump or interactive session log showing the program being started and displaying its output?
On Sun, Jul 28, 2019 at 01:19:38AM -0400, James Lu wrote:
That's what assert does, in part. Since print is now a function, not a statement, you can do this: assert print(expression) or True to get the same effect.
debug context
So "context" is also going to be keyword? That's two new keywords, breaking twice as much existing code: anything that uses "debug", anything that uses "context". By the way, you know that Python has a read-only global variable that tells you whether you are in debug mode? You can write a function to display anything you like, and wrap it in a test like this: if __debug__: display(locals()) -- Steven
Steven D'Aprano wrote:
Oh interesting, I wasn't aware of this global. For anyone else just finding out about it, here's the [documentation](https://docs.python.org/3.9/library/constants.html#__debug__). Thanks for mentioning it, I feel like I end up learning something new in almost every python-ideas and python-dev thread, even if the proposal from the author isn't accepted. I should've subscribed a while ago (: Is there any other way to toggle/specify debug status other than the use of `-O`?
participants (5)
-
Andrew Barnert -
Dominik Vilsmeier -
James Lu -
Kyle Stanley -
Steven D'Aprano