[Python-Dev] SET_LINENO and python options

[Vladimir Marangozov]

[Tim & Ken voting -1 and arguing about]

> [Vladimir]
> > 1) python         - code without SET_LINENO
> > 2) python -g      - code for debugging, with SET_LINENO
> > 3) python -O      - code without doc-strings.
> 
> unless SET_LINENO isn't needed for debugging.

To be honest, I was -1 myself shortly after sending the message.
The consequences, at least for IDLE, make this truly insane, unless,
as you both pointed out, SET_LINENO's functionality is achieved without it.

As a matter of fact, its functionality *can* be achieved, at least for
debugging purposes. That is, for generating the callbacks from the main
loop to a Python function, called on every new source line. This is what
basically the patch I ended up with does.

To clarify the current state of the challenge to remove SET_LINENO, 
here's a short summary of the idea we dicussed previously and which
I implemented (the patch does look scary without explanation <wink>).

Consider the following code sequence diagram (current state), assuming
that each position is an opcode, arguments included:

              0         1        2        3
              01234567890123456780123456780123

co_code ->    [#....#......#....#..#....#....]    where  '.' - an opcode
                 ^                                       '#' - SET_LINENO
                 IP (instruction pointer)

Whenever a sys.settrace function is set, it is called on every # and
this is basically how the debugger gets control for each source line.
It finds out the source line by reading it from the current frame -
f.lineno

Now, here's the picture with the patch intended to obviate SET_LINENO:

              0         1        2        3
              01234567890123456780123456780123

co_code ->    [........................]   the same co_code w/o SET_LINENO

copy of ->    [+...+.....+...+.+...+...]   '+' is a 1-byte CALL_TRACE opcode
co_code         ^                          overriding the 1st opcode of each
internal to     IP                         new source line
eval_code2()

Whenever a sys.settrace function is set, a copy of the original code
stream is created and the IP is redirected to execute the copy before
entering the main loop. In this copy, the 1st instruction for each line
is set to CALL_TRACE. These locations are computed from the co_lnotab
table. CALL_TRACE updates f.lineno and invokes the sys.settrace function.
On return from sys.settrace, it reads the opcode byte it has overwritten
from the original co_code and jumps directly to the argument fetch before
the big switch.

All this works like a charm, except that, a priori, it has 2 drawbacks
regarding full compatibility with the current state with SET_LINENO:

a) f.lineno is not updated on a regular basis like SET_LINENO does; only
   when sys.settrace is set & CALL_TRACE is called.

b) the IP is redirected to the copy of co_code before entering the main
   loop. In practical terms, this means that only entire code objects are
   traceable, that is, tracing can't start in the middle of a code object.

I am not sure whether these 2 things hurt. If so, probably a) hurts more
than b), but this is where I stopped working on this at the time, being
short of more ideas... If you have one, you're welcome :-)

P!ng?? Florx bignal zupkin moognacious today?

If not, this scheme probably looses due to a) and b) and SET_LINENO has
to stay, at least, for full backward compatibility.

Conclusion: the functionality is achieved, full back/compat is not :-/

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252