[Python-Dev] Unifying trace and profile

[Robert Brewer]

I, Robert, wrote:
> 1. Allow trace hooks to receive c_call, c_return,
> and c_exception events (like profile does).

and Nicholas Bastin replied:
> I can easily make this modification.  You can also
> register the same bound method for trace and profile,
> which sort of eliminates this problem.

Wonderful! It looked easy. :)

I worked around this by registering one function for trace, and another for profile. The profile function rejects any non-C event and then calls the trace function.

Robert:
> 3. Expose new sys.gettrace() and getprofile() methods,
> so trace and profile functions that want to play nice
> can call sys.settrace/setprofile(None) only if they
> are the current hook.

Nicholas:
> Not a bad idea, although are you really running into
> this problem a lot?

Well, not "a lot", as I don't expect I'll write very many debuggers in my lifetime ;) But it's important when you have multiple, different debugging systems running at once, either to take advantage of the strengths of each, or to debug a debugger.

Bob:
> 4. Make "the same move" that sys.exitfunc -> atexit made
> (from a single function to multiple functions via
> registration), so multiple tracers/profilers can play
> nice together.

Nick:
> It seems very unlikely that you'll want to have a trace
> hook and profile hook installed at the same time, given
> the extreme unreliability this will introduce into the
> profiler.

True; this request is partly driven by the differing capabilities of each (only profile can handle C events at the moment). Being able to compose debuggers as described above is another reason. Anything else is just the usual (often ignorable) desire for elegance.

Bob:
> 5. Allow the core to filter on the "event" arg before
> hook(frame, event, arg) is called.

Nick:
> What do you mean by this, exactly?  How would you use
> this feature?

As you hinted, I mean that call_trace would only call the trace function if the current event were in a list of "events I want to monitor"; that list of events could be supplied, for example, with a new sys.settrace(func[, events]) signature, with "events" deafulting to all events for backward compatibility. A single int could be used internally, where each bit represents one of the event types.

This would be necessary if trace and profile were unified (see next). If they're not, it's less compelling.

Bob:
> 6. Unify tracing and profiling, which would remove a
> lot of redundant code in ceval and sysmodule and free
> up some space in the PyThreadState struct to boot.

Nick:
> The more events you throw in profiling makes it slow,
> however.  Line events, while a nice thing to have,
> theoretically, would probably make a profiler useless.

Sure. If trace functions can receive C events, then there's no need to add that to profiling. I guess I just see profiling as a "stripped down" version of the general trace architecture, and wonder if it couldn't be that in reality as well as appearance; that is, 
profiling becomes tracing with the 'line' events ignored (before they reach back into your Python trace function and slow everything down). But I also note that the current hotshot uses PyEval_SetTrace "if (self->lineevents)", and PyEval_SetProfile otherwise.

Bob:
> 7. As if the above isn't enough of a dream, it would be nice
> to have a bytecode tracer, which didn't bother with the
> f_lineno logic in maybe_call_line_trace, but just called
> the hook on every instruction.

Nick:
> I'm working on one, but given how much time I've had to
> work on my profiler in the last year, I'm not even going
> to guess when I'll get a real shot at looking at that.
> 
> My long-term goal is to eliminate profiling and tracing
> from the core interpreter entirely and implement the
> functionality in such a way that they don't cost you
> when not in use (i.e., implement profilers and debuggers
> which poke into the process from the outside, rather
> than be supported natively through events).  This isn't
> impossible, but it's difficult because of the large
> variety of platforms.  I have access to most of them,
> but again, my time is hugely constrained right now for
> python development work.

Ah. Sorry to hear that. :/ But no worries on my end; if only #1 can be done someday, I'll be extremely happy. Find me at PyCon, I'll buy you a drink. :)


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060222/9f11a3d4/attachment.html