Here’s an update to PEP 553, which makes $PYTHONBREAKPOINT a first class feature. I’ve also updated PR #3355 with the implementation to match. Cheers, -Barry PEP: 553 Title: Built-in breakpoint() Author: Barry Warsaw <barry@python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2017-09-05 Python-Version: 3.7 Post-History: 2017-09-05, 2017-09-07, 2017-09-13 Abstract ======== This PEP proposes adding a new built-in function called ``breakpoint()`` which enters a Python debugger at the point of the call. Additionally, two new names are added to the ``sys`` module to make the debugger pluggable. Rationale ========= Python has long had a great debugger in its standard library called ``pdb``. Setting a break point is commonly written like this:: foo() import pdb; pdb.set_trace() bar() Thus after executing ``foo()`` and before executing ``bar()``, Python will enter the debugger. However this idiom has several disadvantages. * It's a lot to type (27 characters). * It's easy to typo. The PEP author often mistypes this line, e.g. omitting the semicolon, or typing a dot instead of an underscore. * It ties debugging directly to the choice of pdb. There might be other debugging options, say if you're using an IDE or some other development environment. * Python linters (e.g. flake8 [linters]_) complain about this line because it contains two statements. Breaking the idiom up into two lines further complicates the use of the debugger, These problems can be solved by modeling a solution based on prior art in other languages, and utilizing a convention that already exists in Python. Proposal ======== The JavaScript language provides a ``debugger`` statement [java]_ which enters the debugger at the point where the statement appears. This PEP proposes a new built-in function called ``breakpoint()`` which enters a Python debugger at the call site. Thus the example above would be written like so:: foo() breakpoint() bar() Further, this PEP proposes two new name bindings for the ``sys`` module, called ``sys.breakpointhook()`` and ``sys.__breakpointhook__``. By default, ``sys.breakpointhook()`` implements the actual importing and entry into ``pdb.set_trace()``, and it can be set to a different function to change the debugger that ``breakpoint()`` enters. ``sys.__breakpointhook__`` then stashes the default value of ``sys.breakpointhook()`` to make it easy to reset. This exactly models the existing ``sys.displayhook()`` / ``sys.__displayhook__`` and ``sys.excepthook()`` / ``sys.__excepthook__`` [hooks]_. The signature of the built-in is ``breakpoint(*args, **kws)``. The positional and keyword arguments are passed straight through to ``sys.breakpointhook()`` and the signatures must match or a ``TypeError`` will be raised. The return from ``sys.breakpointhook()`` is passed back up to, and returned from ``breakpoint()``. Since ``sys.breakpointhook()`` by default calls ``pdb.set_trace()`` by default it accepts no arguments. Environment variable ==================== The default implementation of ``sys.breakpointhook()`` consults a new environment variable called ``PYTHONBREAKPOINT``. This environment variable can have various values: * ``PYTHONBREAKPOINT=0`` disables debugging. Specifically, with this value ``sys.breakpointhook()`` returns ``None`` immediately. * ``PYTHONBREAKPOINT= `` (i.e. the empty string). This is the same as not setting the environment variable at all, in which case ``pdb.set_trace()`` is run as usual. * ``PYTHONBREAKPOINT=some.importable.callable``. In this case, ``sys.breakpointhook()`` imports the ``some.importable`` module and gets the ``callable`` object from the resulting module, which it then calls. The value may be a string with no dots, in which case it names a built-in callable, e.g. ``PYTHONBREAKPOINT=int``. (Guido has expressed the preference for normal Python dotted-paths, not setuptools-style entry point syntax [syntax]_.) This environment variable allows external processes to control how breakpoints are handled. Some uses cases include: * Completely disabling all accidental ``breakpoint()`` calls pushed to production. This could be accomplished by setting ``PYTHONBREAKPOINT=0`` in the execution environment. Another suggestion by reviewers of the PEP was to set ``PYTHONBREAKPOINT=sys.exit`` in this case. * IDE integration with specialized debuggers for embedded execution. The IDE would run the program in its debugging environment with ``PYTHONBREAKPOINT`` set to their internal debugging hook. ``PYTHONBREAKPOINT`` is re-interpreted every time ``sys.breakpointhook()`` is reached. This allows processes to change its value during the execution of a program and have ``breakpoint()`` respond to those changes. It is not considered a performance critical section since entering a debugger by definition stops execution. (Of note, the implementation fast-tracks the ``PYTHONBREAKPOINT=0`` case.) Overriding ``sys.breakpointhook`` defeats the default consultation of ``PYTHONBREAKPOINT``. It is up to the overriding code to consult ``PYTHONBREAKPOINT`` if they want. If access to the ``PYTHONBREAKPOINT`` callable fails in any way (e.g. the import fails, or the resulting module does not contain the callable), a ``RuntimeWarning`` is issued, and no breakpoint function is called. Open issues =========== Confirmation from other debugger vendors ---------------------------------------- We want to get confirmation from at least one alternative debugger implementation (e.g. PyCharm) that the hooks provided in this PEP will be useful to them. Evaluation of $PYTHONBREAKPOINT ------------------------------- There has been some mailing list discussion around this topic. The basic behavior as described above does not appear to be controversial. Guido has expressed a preference for ``$PYTHONBREAKPOINT`` consultation happening in the default implementation of ``sys.breakpointhook`` [envar]_. The one point of discussion relates to whether the value of ``$PYTHONBREAKPOINT`` should be loaded on interpreter start, and whether its value should be cached the first time it's accessed. It is the PEP author's opinion that the environment variable need only be looked up at the time of use. It is also the author's opinion that the value of the environment variable can be accessed each time ``sys.breakpointhook`` is run, to allow for maximum functional flexibility. Because this feature enters the debugger, any performance improvements for caching will be negligible and do not outweigh the flexibility. Further, because the special case of ``PYTHONBREAKPOINT=0`` is fast-tracked, the no-op code path is quite fast, and should be in the noise given the function calls of ``breakpoint()`` -> ``sys.breakpointhook()``. Breakpoint bytecode ------------------- Related, there has been an idea to add a bytecode that calls ``sys.breakpointhook()``. Whether built-in ``breakpoint()`` emits this bytecode (or gets peephole optimized to the bytecode) is an open issue. The bytecode is useful for debuggers that actively modify bytecode streams to trampoline into their own debugger. Having a "breakpoint" bytecode might allow them to avoid bytecode modification in order to invoke this trampoline. *NOTE*: It probably makes sense to split this idea into a separate PEP. Call a fancier object by default -------------------------------- Some folks want to be able to use other ``pdb`` interfaces such as ``pdb.pm()``. Although this is a less commonly used API, it could be supported by binding ``sys.breakpointhook`` to an object that implements ``__call__()``. Calling this object would call ``pdb.set_trace()``, but the object could expose other methods, such as ``pdb.pm()``, making invocation of it as handy as ``breakpoint.pm()``. Implementation ============== A pull request exists with the proposed implementation [impl]_. Rejected alternatives ===================== A new keyword ------------- Originally, the author considered a new keyword, or an extension to an existing keyword such as ``break here``. This is rejected on several fronts. * A brand new keyword would require a ``__future__`` to enable it since almost any new keyword could conflict with existing code. This negates the ease with which you can enter the debugger. * An extended keyword such as ``break here``, while more readable and not requiring a ``__future__`` would tie the keyword extension to this new feature, preventing more useful extensions such as those proposed in PEP 548. * A new keyword would require a modified grammar and likely a new bytecode. Each of these makes the implementation more complex. A new built-in breaks no existing code (since any existing module global would just shadow the built-in) and is quite easy to implement. sys.breakpoint() ---------------- Why not ``sys.breakpoint()``? Requiring an import to invoke the debugger is explicitly rejected because ``sys`` is not imported in every module. That just requires more typing and would lead to:: import sys; sys.breakpoint() which inherits several of the problems this PEP aims to solve. Version History =============== * 2017-09-13 * The ``PYTHONBREAKPOINT`` environment variable is made a first class feature. * 2017-09-07 * ``debug()`` renamed to ``breakpoint()`` * Signature changed to ``breakpoint(*args, **kws)`` which is passed straight through to ``sys.breakpointhook()``. References ========== .. [linters] http://flake8.readthedocs.io/en/latest/ .. [java] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements... .. [hooks] https://docs.python.org/3/library/sys.html#sys.displayhook .. [syntax] http://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=console... .. [impl] https://github.com/python/cpython/pull/3355 .. [envar] https://mail.python.org/pipermail/python-dev/2017-September/149447.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: