Understanding some python internals

[Michael Hudson]

gerson.kurz at t-online.de (Gerson Kurz) writes:

> I'm trying to understand how the python debugger package works. 

Ah.

> To that effect, I've written a kind of "strace for python", that
> should tell me each instruction as it happens, in a tracefile.
> 
> The code basically does this:
> 
> - use sys.settrace() to attach a tracefunc to every statement executed
> - try to analyze the arguments passed to the tracefunc
> 
> So far, so good. Preliminary results can be seen here:
> 
> http://p-nand-q.com/python/strace.zip
> 
> The test code I want to analyze is this:
> 
> ------------------------
> test = 60
> 
> def test(x):
>     return x*2
>     
> i = 0
> while i < 5:
>     print test(i)
>     i += 1
>     
> exec("test(42)")
> ------------------------
> 
> Nothing too deviant, right? Try to run strace.py over this, and look
> into the result (STRACE.LOG). 
> 
> Ok, my questions:
> 
> (1) I want to disassemble the current step that the tracefunc is
> called for - nothing else. 

So you want to print the instruction that gets executed next?

> dis.disassemble takes a code object, but will disassemble the
> *whole* code object, not just some part. (Same, if I pass it frame
> rather than code). How can I do that?

Well, you can't *use* dis (I think).  You can probably crib the code
you need from that module, though.

> I would guess that it has something to do with the members
> frame.f_code.co_code and co_lnotab - the "documentation" says the
> latter is "encoded mapping of line numbers to bytecode indices", can
> somebody enlighten me on that?

f_code & f_lasti should be all you need.  You don't really want to try
to understand co_lnotab, but if you do, grep for it in
Python/compile.c.  There are some comments that explain how it works.

> (2) I want to print the source for the current step that the tracefunc
> is called for - nothing else. My first attempt (which is in the
> current source) is to read the file from frame.f_code.co_filename and
> read the line that is given in frame.f_lineno). This has two problems:
> 
> a) if the source spans more than one line, only the first line will be
> shown

Yup.  Not much that can be done about that, I think.  It might be the
last line in some circumstances, too...

> b) if the source is in an exec() statement, co_filename is "<string>".
> (See also question (3))

Well, in this case the source may have gone to the big bitbucket in
the sky.

> Is there are a "canonic" way of getting the code about to be executed
> inside a tracefunc?

No.  See above.  The inspect module may help...

> I tried inspect.getsource(frame) - but that prints the source for the
> whole code object, not the current frame. Bug or intended behaviour?

Oh :) Don't understand the distinction here?  If you mean you get the
code for the whole function rather than just for the currently
executing line, then surely that's the intended behaviour -- as that's
what it does.

> (3) If the code object is in an exec statement, co_filename is
> "<string>" and as far as I can see the code executed cannot be read
> from inside the tracefunc. Or can it?

No.  See above.

Cheers,
M.

-- 
 "Sturgeon's Law (90% of everything is crap) applies to Usenet."
 "Nothing guarantees that the 10% isn't crap, too."
                -- Gene Spafford's Axiom #2 of Usenet, and a corollary