[Python-Dev] API design question: how to extend sys.settrace()?

[Victor Stinner]

Hi,

In bpo-29400, it was proposed to add the ability to trace not only
function calls but also instructions at the bytecode level. I like the
idea, but I don't see how to extend sys.settrace() to add a new
"trace_instructions: bool" optional (keyword-only?) parameter without
breaking the backward compatibility. Should we add a new function
instead?

Would it be possible to make the API "future-proof", so we can extend
it again later?

I almost never used sys.settrace(), so I prefer to ask on this python-dev list.

Copy of the George King's message:
https://bugs.python.org/issue29400#msg298441

"""
After reviewing the thread, I'm reminded that the main design problem
concerns preserving behavior of this idiom:
"old=sys.gettrace(); ...; sys.settrace(old)"

If we add more state, i.e. the `trace_instructions` bool, then the
above idiom no longer correctly stores/reloads the full state.

Here are the options that I see:

1. New APIs:
* `gettrace() -> Callable # no change.`
* `settrace(tracefunc: Callable) -> None # no change.`
* `gettraceinst() -> Callable # new.`
* `settraceinst(tracefunc: Callable) -> None # new.`
Behavior:
* `settrace()` raises if `gettraceinst()` is not None.
* `settraceinst()` raises if `gettrace()` is not None.


2. Add keyword arg to `settrace`.
* `gettrace() -> Callable # no change.`
* `settrace(tracefunc: Callable,
trace_instructions:Optional[bool]=False) -> None # added keyword.`
* `gettracestate() -> Dict[str, Any] # new.`
Behavior:
* `gettracestate()` returns the complete trace-related state:
currently, `tracefunc` and `trace_instructions` fields.
* `gettrace()` raises an exception if any of the trace state does not
match the old behavior, i.e. if `trace_instructions` is set.


3. New API, but with extensible keyword args:
* `settracestate(tracefunc: Callable,
trace_instructions:Optional[bool]=False) -> None # new.`
* `gettracestate() -> Dict[str, Any] # new.`
Behavior:
* `settrace()` raises if `gettracestate()` is not None.
* `settracestate()` raises if `gettrace()` is not None.


As I see it:
* approach #1 is more conservative because it does not modify the old API.
* approach #2 is more extensible because all future features can be
represented by the state dictionary.
* approach #3 has both of these strengths, but is also the most work.

Examples of possible future features via keywords:
* branch-level tracing, as briefly disscussed above.
* instruction filtering set, which could be a more general version of
the branch tracing.

I intend to prototype one or both of these, but I'm also feeling a bit
of time pressure to get the basic feature on track for 3.7.
"""

settraceinst() doesn't seem "future-proof" to me.

Victor