On Sep 5, 2017, at 19:31, Giampaolo Rodola' wrote:

True. Personally I have a shortcut in my IDE (Sublime) so I when I type "pdb" -> TAB it auto completes it.

Somehow I think debug() would make this a bit harder as it's more likely a "debug()" line will pass unnoticed. For this reason I would give a -1 to this proposal.

I think if your linter or editor can take note of the pdb idiom, it can also do so for the debug() built-in.

Personally I would find it helpful if there was a hook to choose the default debugger to use on "pdb.set_trace()" via .pdbrc or PYTHONDEBUGGER environment variable or something. I tried (unsuccessfully) to run ipdb on "pdb.set_trace()", I gave up and ended up emulating auto completion and commands history with this: https://github.com/giampaolo/sysconf/blob/master/home/.pdbrc.py

I don’t think that’s a good idea. pdb is a thing, and that thing is the standard library debugger. I don’t think ‘pdb’ should be the term we use to describe a generic Python debugger interface. That to me is one of the advantages of PEP 553; it separates the act of invoking the debugging from the actual debugger so invoked. Cheers, -Barry

On Sep 6, 2017, at 14:59, Terry Reedy wrote:

Currently, the debugger is started in response to a menu seletion in the IDLE process while the python process is idle. One reason for the 'idle' requirement' is because when code is exec-uting, the loop that reads commands, executes them, and sends responses is blocked on the exec call. The IDLE process sets up its debugger window, its ends of the rpc channels, and commands to python process to set up Idb and the other ends of the channels. The challenge would be to initiate setup from the server process and deal with the blocked loop.

Would the environment variable idea in the latest version of the PEP help you here? Cheers, -Barry

On Wed, Sep 6, 2017 at 11:59 PM, Terry Reedy wrote:

On 9/6/2017 6:45 PM, Barry Warsaw wrote:

...

On Sep 6, 2017, at 14:59, Terry Reedy wrote:

...

Currently, the debugger is started in response to a menu seletion in the IDLE process while the python process is idle. One reason for the 'idle' requirement' is because when code is exec-uting, the loop that reads commands, executes them, and sends responses is blocked on the exec call. The IDLE process sets up its debugger window, its ends of the rpc channels, and commands to python process to set up Idb and the other ends of the channels. The challenge would be to initiate setup from the server process and deal with the blocked loop.

Would the environment variable idea in the latest version of the PEP help you here?

That seems to be a mechanism for users to convey which function to call. It does not simplify the function call chosen.

pdb works because a) it is designed to be imported into and run in a user module and b) it uses the existing stdin and stdout streams to interact with the user. Idb, on the other hand, is designed to be import in idlelib.run, which is (almost) invisible to user code, and use a special channel to the GUI window.

Having said that, I see a solution: before running user code, set things up as now, except do not start tracing and do not show the debugger window, but do set a run.idb_ready flag. Then breakpoint_handler() can check the flag and start tracing either through idb or pdb.

Without following all this (or the PEP, yet) super exactly, does this mean you are satisfied with what the PEP currently proposes or would you like changes? It's a little unclear from what you wrote.

What makes breakpoint() less than super useful in IDLE is that IDLE already allows setting of persistent breakpoints on a line by right-clicking. Since they are not stored in the file, there is nothing to erase when one runs the code in Python directly.

I think that's true for any IDE that has existing integrated debug capabilities. However for every IDE I would hope that calling breakpoint() will *also* break into the IDE's debugger. My use case is that sometimes I have a need for a *conditional* breakpoint where it's just much simpler to decide whether to break or not in Python code than it would be to use the IDE's conditional breakpoint facilities. (Plus there's something about old dogs and new tricks, but I've forgotten exactly how that one goes. :-) -- --Guido van Rossum (python.org/~guido)

On 9/7/2017 12:52 PM, Barry Warsaw wrote:

On Sep 7, 2017, at 07:46, Guido van Rossum wrote:

...

Without following all this (or the PEP, yet) super exactly, does this mean you are satisfied with what the PEP currently proposes or would you like changes? It's a little unclear from what you wrote.

I’m also unsure whether Terry is good with the existing PEP or suggesting changes.

It seems to me that simplifying 'import pdb; pbd.set_trace()' to 'breakpoint()' is not, in itself, justification for a new builtin. Rather, the justification is the possibility of invoking dbs other than pdb. So I have been examining the feasibility of doing that, with IDLE's Idb + rpc + Debugger widget as a test case and example. My conclusion: possible, but not as graceful as I would like. In response to Barry's idea of PYTHONBREAKPOINTHOOK (default pdb.set_trace, and is 'HOOK' really needed), I examined further how IDLE's debugger is different and how the differences cause problems. Then a thought occurred to me: how about rewriting it as a pure tkinter GUI debugger, much more like pdb. Call the module either gdb or tkdb. PYTHONBREAKPOINT=[package.].gdb.set_trace would then make more sense. As with pdb, the UI would run in the same process as the debugger and user code. No rpc setup needed. No interference between the debug gui and the rest of IDLE (as sometimes happens now). No need to pickle objects to display the bindings of global and local names. The reasons for separating user code from IDLE mostly do not apply to the debugger. I would make it a single window with 2 panes, much like turtledemo, but with the turtle screen replaced with the GUI display. (Try IDLE to see what that would look like now, or pull PR 2494 first to see it with one proposed patch.) Gdb should not be limited to running from IDLE but could be launched even when running code from the console. After an initial version, the entry point could accept a list of breakpoint lines and a color mapping for syntax coloring. I think breakpoint() should have a db= parameter so one can select a debugger in one removable line. The sys interface is more useful for IDEs to change the default, possible with other args (like breakpoints and colors) bound to the callable. Breakpoint() should pass on other args. In particular, people who invoke gdb from within a tkinter program should be able to pass in the root widget, to be the master of the debug window. This has to be done from within the user program, and the addition of breakpoint allows that, and makes running the gui with user code more feasible. --- A somewhat separate point: the name breakpoint() is slightly misleading, which has consequences if it is (improperly) called more than once. While breakpoint() acts as a breakpoint, what it does (at least in the default pdb case) is *initialize* and start a *new* debugger, possibly after an import. Re-importing a module is no big deal. Replacing an existing debugger with a *new* one, and tossing away all defined aliases and breakpoints and Bdb's internal caches, is. It is likely not what most people would want or expect. I think it more likely that people will call breakpoint() multiple times than they would, for instance, call pdb.set_trace() multiple times. With a gui debugger, having one window go and another appear might be additionally annoying. If the first window is not immediately GCed, having two windows would be confusing. Perhaps breakpoint() could be made a no-op after the first call.

...

I think that's true for any IDE that has existing integrated debug capabilities. However for every IDE I would hope that calling breakpoint() will *also* break into the IDE's debugger. My use case is that sometimes I have a need for a *conditional* breakpoint where it's just much simpler to decide whether to break or not in Python code than it would be to use the IDE's conditional breakpoint facilities.

That certainly aligns with my own experience and expectations. I guess I’m a fellow old dog. :)

-- Terry Jan Reedy

On Sep 7, 2017, at 16:19, Terry Reedy wrote:

I think breakpoint() should have a db= parameter so one can select a debugger in one removable line. The sys interface is more useful for IDEs to change the default, possible with other args (like breakpoints and colors) bound to the callable.

I’m skeptical about that. I think any particular user is going to overwhelmingly use the same debugger, so having to repeat themselves every time they want to enter the debugger is going to get tedious fast. I know it would annoy *me* if I had to tell it to use pdb every time I wrote `breakpoint()`, and I’m almost never going to use anything else. I’m also not sure what useful semantics for `db` would be. E.g. what specifically would you set `db` to in order to invoke idle or tkdb (‘gdb’ would be an unfortunate name I think, given the popular existing GNU Debugger ;). I don’t even know what useful thing I’d set `db` to to mean “invoke pdb”. Please don’t say “well, pdb will still be the default so you don’t have to set it to anything” because that does kind of miss the point. I also want to keep breakpoint() as generic as possible. I think doing so allows it to be a nice API into whatever interesting and sophisticated implementations folks can think up underneath it. So *if* you came up with a cool thing that interpreted `db` in some special way, there should be a very low barrier to providing that to your users, e.g.: * pip install myfancymetadebugger * put a snippet which sets sys.breakpointhook in your $PYTHONSTARTUP file * profit! That second item could be replaced with export PYTHONBREAKPOINTHOOK=myfancymetadebugger.invoke (hmm, does that mean the envar is getting more interesting?)

Breakpoint() should pass on other args.

I strongly believe it should pass through *all* args.

A somewhat separate point: the name breakpoint() is slightly misleading, which has consequences if it is (improperly) called more than once. While breakpoint() acts as a breakpoint, what it does (at least in the default pdb case) is *initialize* and start a *new* debugger, possibly after an import. Re-importing a module is no big deal. Replacing an existing debugger with a *new* one, and tossing away all defined aliases and breakpoints and Bdb's internal caches, is. It is likely not what most people would want or expect. I think it more likely that people will call breakpoint() multiple times than they would, for instance, call pdb.set_trace() multiple times.

Multiple calls to pdb.set_trace() is fairly common in practice today, so I’m not terribly concerned about it. There’s nothing fundamentally different with multiple calls to breakpoint() today. If we care, we can provide a more efficient/different API and make that the default. The machinery in PEP 553 can easily support that, but doing it is outside the scope of the PEP.

With a gui debugger, having one window go and another appear might be additionally annoying. If the first window is not immediately GCed, having two windows would be confusing. Perhaps breakpoint() could be made a no-op after the first call.

Your sys.breakpointhook could easily implement that, with a much better user experience than what built-in breakpoint() could do anyway. Cheers, -Barry

If I may suggest a small API tweak, I think it would be useful if breakpoint() accepted an optional header argument. In IPython, the equivalent for non-postmortem debugging is IPython.embed, which can be given a header. This is useful to provide the user with some information about perhaps where the breakpoint is coming from, relevant data they might want to look at, etc: ``` from IPython import embed def f(x=10): y = x+2 embed(header="in f") return y x = 20 print(f(x)) embed(header="Top level") ``` I understand in most cases these are meant to be deleted right after usage and the author is likely to have a text editor open next to the terminal where they're debugging. But still, I've found myself putting multiple such calls in a code to look at what's going on in different parts of the execution stack, and it can be handy to have a bit of information to get your bearings. Just a thought... Best f

On 2017-09-07 00:20:17 +0000, Barry Warsaw said:

Thanks Fernando, this is exactly the kind of feedback from other debuggers that I’m looking for. It certainly sounds like a handy feature; I’ve found myself wanting something like that from pdb from time to time.

Glad it's useful, thanks for the pep!

The PEP has an open issue regarding breakpoint() taking *args and **kws, which would just be passed through the call stack. It sounds like you’d be in favor of that enhancement.

If you go witht the `(*a, **k)` pass-through API, would you have a special keyword-only arg called 'header' or similar? That seems like a decent compromise to support the feature with the builtin while allowing other implementations to offer more features. In any case, +1 to a pass-through API, as long as the built-in supports some kind of mechanism to help the user get their bearings with "you're here" type messages. Cheers f

On Sep 7, 2017, at 14:25, Barry Warsaw wrote:

I’ll see what it takes to add `header` to pdb.set_trace(), but I’ll do that as a separate PR (i.e. not as part of this PEP).

Turns out to be pretty easy. https://bugs.python.org/issue31389 https://github.com/python/cpython/pull/3438 Cheers, -Barry

On Sep 7, 2017, at 18:03, Fernando Perez wrote:

Ah, perfect! I've subscribed to the PR on github and can pitch in there further if my input is of any use.

Awesome, thanks! -Barry

Yeah, I like the idea, but I don't like the debug() name -- IIRC there's a helper named debug() in some codebase I know of that prints its arguments under certain circumstances. On Tue, Sep 5, 2017 at 7:58 PM, Nathaniel Smith wrote:

...

I’ve written a PEP proposing the addition of a new built-in function called debug(). Adding this to your code would invoke a debugger through

On Tue, Sep 5, 2017 at 6:14 PM, Barry Warsaw wrote: the hook function sys.debughook().

The 'import pdb; pdb.set_trace()' dance is *extremely* obscure, so replacing it with something more friendly seems like a great idea.

Maybe breakpoint() would be a better description of what set_trace() actually does? This would also avoid confusion with IPython's very useful debug magic: https://ipython.readthedocs.io/en/stable/interactive/ magics.html#magic-debug and which might also be worth stealing for the builtin REPL. (Personally I use it way more often than set_trace().)

Example:

In [1]: def f(): ...: x = 1 ...: raise RuntimeError ...:

In [2]: f() ------------------------------------------------------------ --------------- RuntimeError Traceback (most recent call last) <ipython-input-2-0ec059b9bfe1> in <module>() ----> 1 f()

<ipython-input-1-db0dc90ff5b9> in f() 1 def f(): 2 x = 1 ----> 3 raise RuntimeError

RuntimeError:

In [3]: debug

...

<ipython-input-1-db0dc90ff5b9>(3)f() 1 def f(): 2 x = 1 ----> 3 raise RuntimeError

ipdb> p x 1

-n

-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org

-- --Guido van Rossum (python.org/~guido)

IIRC they indeed insinuate debug() into the builtins. My suggestion is also breakpoint(). On Wed, Sep 6, 2017 at 7:39 AM, Barry Warsaw wrote:

On Sep 5, 2017, at 20:15, Guido van Rossum wrote:

...

Yeah, I like the idea, but I don't like the debug() name -- IIRC there's

a helper named debug() in some codebase I know of that prints its arguments under certain circumstances.

...

On Tue, Sep 5, 2017 at 7:58 PM, Nathaniel Smith wrote:

Maybe breakpoint() would be a better description of what set_trace() actually does?

Originally I was thinking of a keyword like ‘break here’, but once (after discussion with a few folks at the sprint) I settled on a built-in function, I was looking for something concise that most directly reflected the intent. Plus I knew I wanted to mirror the sys.*hooks, so again I looked for something short. debug() was the best I could come up with!

breakpoint() could work, although would the hooks then be sys.breakpointhook() and sys.__breakpointhook__? Too bad we can’t just use break() :).

Guido, is that helper you’re thinking of implemented as a built-in? If you have a suggestion, it would short-circuit the inevitable bikeshedding.

...

This would also avoid confusion with IPython's very useful debug magic: https://ipython.readthedocs.io/en/stable/interactive/ magics.html#magic-debug and which might also be worth stealing for the builtin REPL. (Personally I use it way more often than set_trace().)

Interesting. I’m not an IPython user. Do you think its %debug magic would benefit from PEP 553?

(Aside: improving/expanding the stdlib debugger is something else I’d like to work on, but this is completely independent of PEP 553.)

Cheers, -Barry

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org

-- --Guido van Rossum (python.org/~guido)

On Sep 6, 2017, at 07:46, Guido van Rossum wrote:

IIRC they indeed insinuate debug() into the builtins. My suggestion is also breakpoint().

breakpoint() it is then! I’ll rename the sys hooks too, but keep the naming scheme of the existing sys hooks. Cheers, -Barry

On Sep 6, 2017, at 10:14, Fabio Zadrozny wrote:

I think it's a nice idea.

Great!

Related to the name, on the windows c++ there's "DebugBreak": https://msdn.microsoft.com/en-us/library/windows/desktop/ms679297(v=vs.85).a..., which I think is a better name (so, it'd be debug_break for Python -- I think it's better than plain breakpoint(), and wouldn't clash as debug()).

It’s important to understand that a new built-in is technically never going to clash with existing code, regardless of what it’s called. Given Python’s name resolution rules, if your code uses any built, it’ll just shadow it. That’s one big reason why the PEP proposed a built-in rather than say a keyword. That said, while I like the more succinct `debug()` name, Guido prefers `breakpoint()` and that works fine for me.

I think I could change the hook on a custom sitecustomize (there's already one in place in PyDev) so that the debug_break() would actually read some env var to do that work (and provide some utility for users to pre-setup it when not launching from inside the IDE).

I had meant to add an open issue about the idea of adding an environment variable, such as $PYTHONBREAKPOINTHOOK which could be set to the callable to bind to sys.breakpointhook(). I’ve now added that to PEP 553, and I think that handles the use case you outline above.

Still, there may be other settings that the user needs to pass to settrace() when doing a remote debug session -- i.e.: things such as the host, port to connect, etc -- see: https://github.com/fabioz/PyDev.Debugger/blob/master/pydevd.py#L1121, so, maybe the debug_break() method should accept keyword arguments to pass along to support other backends?

Possibly, and there’s an open issue about that, but I’m skeptical about that for your use case. Will a user setting a breakpoint know what to pass there? Cheers, -Barry

On Sep 6, 2017, at 23:10, Terry Reedy wrote:

Environmental variables are set to strings, not objects. It is not clear how you intend to handle the conversion.

The environment variable names a module import path. Without quibbling about the details of the syntax (because honestly, I’m not convinced it’s a useful feature), it would work roughly like: * The default value is equivalent to PYTHONBREAKPOINTHOOK=pdb.set_trace * breakpoint() splits the value on the rightmost dot * modules on the LHS are imported, then the RHS is getattr’d out of that * That’s the callable breakpoint() calls -Barry

On Sep 7, 2017, at 10:00, Christian Heimes wrote:

Setuptools' entry points [1] use colon between import and function, e.g. "pdb:set_trace" would import pdb and then execute set_trace. The approach can be augmented to allow calling a class method, too.

So

"package.module:myclass.classfunc"

would do :

from package.module import myclass myclass.classfunc

Yep, that’s how it's described in the PEP 553 open issue. I just didn’t include that complication in my response. -Barry

On Thu, Sep 7, 2017 at 4:52 PM, Terry Reedy wrote:

Environmental variables tend to be a pain on Windows and nigh unusable by beginners. Leaving that aside, I see these problems with trying to use one for IDLE's *current* debugger.

pdb is universal, in the sense of working with any python run with actual or simulated stdin and stdout. IDLE's idb is specific to working with IDLE. So one could not set an EV to 'idlelib.idb.start' and leave it while switching between IDLE and console.

Would it work for IDLE to set the environment variable for the child process? The user certainly should not need to be involved in that. That doesn't address the issue of setting up the communications channel before breakpoint() is called, but allows the breakpointhook that gets used to work with whatever has been arranged. -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein

On Sep 7, 2017, at 13:52, Terry Reedy wrote:

pdb.set_trace is a public and stable interface. IDLE's is private and likely to be initially unstable. I can imagine that the function that I would want to bind to sys.__breakpoint__ would be a bound method

To be pedantic, you’re not supposed to touch sys.__breakpointhook__ although like sys.__excepthook__ and sys.__displayhook__ they are not enforced to be read-only. Cheers, -Barry

99% of the time I use a debugger I use pdb.set_trace(). The pm() stuff is typically useful for debugging small, simple programs only -- complex programs likely hide the exception somewhere (after logging it) so there's nothing for pdb.pm() to look at. I think Barry is wisely focusing on just the ability to quickly and programmatically insert a breakpoint. On Wed, Sep 6, 2017 at 10:00 AM, Nathaniel Smith wrote:

On Wed, Sep 6, 2017 at 7:39 AM, Barry Warsaw wrote:

...

...

On Tue, Sep 5, 2017 at 7:58 PM, Nathaniel Smith wrote: This would also avoid confusion with IPython's very useful debug magic: https://ipython.readthedocs.io/en/stable/interactive/ magics.html#magic-debug and which might also be worth stealing for the builtin REPL. (Personally I use it way more often than set_trace().)

Interesting. I’m not an IPython user. Do you think its %debug magic would benefit from PEP 553?

Not in particular. But if you're working on making debugger entry more discoverable/human-friendly, then providing a friendlier alias for the pdb.pm() semantics might be useful too?

Actually, if you look at the pdb docs, the 3 ways of entering the debugger that merit demonstrations at the top of the manual page are:

pdb.run("...code...") # "I want to debug this code" pdb.set_trace() # "break here" pdb.pm() # "wtf just happened?"

The set_trace() name is particularly opaque, but if we're talking about adding a friendly debugger abstraction layer then I'd at least think about whether to make it cover all three of these.

-n

-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org

-- --Guido van Rossum (python.org/~guido)