[Python-Dev] line numbers, pass statements, implicit returns

[Jeremy Hylton]

On 4/2/06, Guido van Rossum <guido at python.org> wrote:
> On 4/1/06, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
> > There are several test cases in test_trace that are commented out.  We
> > did this when we merged the ast-branch and promised to come back to
> > them.  I'm coming back to them now, but the test aren't documented
> > well and the feature they test isn't specified well.
> >
> > The failing tests I've looked at so far involving pass statements and
> > implicit "return None" statements generated by the compiler.  The
> > tests seem to verify that
> >
> >     1) if you have a function that ends with an if/else where the body
> > of the else is pass,
> >         there is no line number associated with the return
> >     2) if you have a function that ends with a try/except where the
> > body of the except is pass,
> >         there is a line number associated with the return.
> >
> > Here's a failing example
> >
> > def ireturn_example():
> >     a = 5
> >     b = 5
> >     if a == b:
> >         b = a+1
> >     else:
> >         pass
> >
> > The code is traced and test_trace verifies that the return is
> > associated with line 4!
> >
> > In these cases, the ast compiler will always associate a line number
> > with the return.  (Technically with the LOAD_CONST preceding the
> > RETURN_VALUE.)  This happens pretty much be accident.  It always
> > associates a line number with the first opcode generated after a new
> > statement is visited.  Since a Pass statement node has no opcode, the
> > return generates the first opcode.
> >
> > Now I could add some special cases to the compiler to preserve the old
> > behavior, the question is: Why bother?  It's such an unlikely case (an
> > else that has no effect).  Does anyone depend on the current behavior
> > for the ireturn_example()?  It seems sensible to me to always generate
> > a line number for the pass + return case, just so you see the control
> > flow as you step through the debugger.
>
> Makes sense to me. I can't imagine what this was testing except
> perhaps a corner case in the algorithm for generating the (insanely
> complicated) linenumber mapping table.
>
> > The other case that has changed is that the new compiler does not
> > generate code for "while 0:"  I don't remember why <0.5 wink>.  There
> > are several test cases that verify line numbers for code using this
> > kind of bogus construct.  There are no lines anymore, so I would
> > change the tests so that they don't expect the lines in question.  But
> > I have no idea what they are trying to test.  Does anyone know?
>
> Not me. This is definitely not part of the language spec! :-)

It needs to be part of some spec, I think.  It's probably part of the
implementation rather than the language, but we need some description
of what the trace hooks are supposed to do.  The unittests are some
kind of spec, but not a great one (because they don't seem exhaustive
and don't explain why they work the way they do).  Should we write an
informational PEP, then, that explains the conditions under which you
get a line event in trace mode?  And how much do we care about
compatibility from release to release?

Here's another case that came up while pursuing these tests.  It looks
like the old compiler generated lnotab entries with a 0 change in line
number, merely so that the trace code would generate an event on that
line.  The problem case is a for loop, where there is a line number
set for the SETUP_LOOP instruction.  You'd like a line event for the
last time through the for loop, when the iterator is consumed, and to
get that it appears that the lnotab has an entry with a 0 increment in
line number and a non-zero increment in bytecode offset.  There's no
comment in the lnotab code that mentions that as a possibility, so it
came as a surprise to me :-).  The question, then, is how many cases
are there like this.  It may only be possible to answer this by
walking through the old compiler code and noting all the uncommented
special cases.

Looks like a project for alpha 2.

Jeremy