co_firstlineno on decorated functions
[Since I received no replies on this in python-list, perhaps python-dev is more appropriate] Hello, I've been tinkering with __code__.co_firstlineno for testing the trace.py module (Python Issue 9315), and ran into an interesting problem. Consider this code: def dummydecorator(f): return f def foo(a): return a @dummydecorator def bar(a): return a if __name__ == "__main__": print foo.__code__.co_firstlineno print bar.__code__.co_firstlineno ---- The first print out correctly specifies the line "def foo" is in. However, the second one points to the line with "@dummydecorator" instead of "def bar". [Python 2.6] The side-effects of this behavior can be easily seen in the output of modules like trace and profile. Would you say it's normal, or could this be considered a bug? Eli
On Tue, Aug 3, 2010 at 1:40 PM, Eli Bendersky <eliben@gmail.com> wrote:
The first print out correctly specifies the line "def foo" is in. However, the second one points to the line with "@dummydecorator" instead of "def bar". [Python 2.6]
The side-effects of this behavior can be easily seen in the output of modules like trace and profile. Would you say it's normal, or could this be considered a bug?
Since the decorator is as much a part of the function definition as the def line is, I would say that it is correct (the name says "firstlineno", not "deflineno"). inspect.getsource() and friends actually rely on this behaviour, so changing it really isn't an option. However, that does mean *using* it as though it always points to the def line is incorrect, suggesting there are related bugs in trace and profile. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, 3 Aug 2010 22:25:01 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Aug 3, 2010 at 1:40 PM, Eli Bendersky <eliben@gmail.com> wrote:
The first print out correctly specifies the line "def foo" is in. However, the second one points to the line with "@dummydecorator" instead of "def bar". [Python 2.6]
The side-effects of this behavior can be easily seen in the output of modules like trace and profile. Would you say it's normal, or could this be considered a bug?
Since the decorator is as much a part of the function definition as the def line is, I would say that it is correct (the name says "firstlineno", not "deflineno").
That's debatable. Since writing: @b def a(): ... is equivalent to: def a(): ... a = b(a) and in the latter case co_firstlineno points to the "def a()" line. Furthermore, co_firstlineno is an attribute of the code object, not the function object, so it shouldn't ideally depend on whether a decorator was applied or not. Regards Antoine.
On Tue, Aug 3, 2010 at 10:48 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Furthermore, co_firstlineno is an attribute of the code object, not the function object, so it shouldn't ideally depend on whether a decorator was applied or not.
You cut the part about the inspect module (and no doubt other code) relying on the current meaning, though. While I'd agree with you for a clean slate definition, that's not what we're dealing with here: "co_firstlineno" has an existing meaning, and it *isn't* "the line containing the def keyword", it's "the first line of the function, including any decorator lines". The decision could (and arguably should) have gone the other way when decorator syntax was first added, but changing our minds now would be making the situation worse rather than better. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Aug 3, 2010 at 6:05 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Aug 3, 2010 at 10:48 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Furthermore, co_firstlineno is an attribute of the code object, not the function object, so it shouldn't ideally depend on whether a decorator was applied or not.
You cut the part about the inspect module (and no doubt other code) relying on the current meaning, though. While I'd agree with you for a clean slate definition, that's not what we're dealing with here: "co_firstlineno" has an existing meaning, and it *isn't* "the line containing the def keyword", it's "the first line of the function, including any decorator lines". The decision could (and arguably should) have gone the other way when decorator syntax was first added, but changing our minds now would be making the situation worse rather than better.
What are the use cases for co_firstlineno? Even if it is for displaying the source code, I can find virtue for both sides of this argument. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 3, 2010 at 10:14 AM, Guido van Rossum <guido@python.org> wrote:
What are the use cases for co_firstlineno? Even if it is for displaying the source code, I can find virtue for both sides of this argument.
nose uses co_firstlineno to determine order of the test functions and decorating a test function can change such order. To keep the ordering, it provides nose.tools.make_decorator() which explicitly keeps the line number of the original function. Check the following thread for a discussion in this regard: http://groups.google.com/group/nose-users/browse_thread/thread/3e354cbb5b1fa... Thanks, Raghu
Le mardi 03 août 2010 à 11:05 -0400, Raghuram Devarakonda a écrit :
On Tue, Aug 3, 2010 at 10:14 AM, Guido van Rossum <guido@python.org> wrote:
What are the use cases for co_firstlineno? Even if it is for displaying the source code, I can find virtue for both sides of this argument.
nose uses co_firstlineno to determine order of the test functions and decorating a test function can change such order. To keep the ordering, it provides nose.tools.make_decorator() which explicitly keeps the line number of the original function. Check the following thread for a discussion in this regard:
That's pretty much orthogonal, though. We're talking about the case of a decorator which returns the original function object (and therefore doesn't change co_firstlineno). Anyway, when co_firstlineno is used for ordering purposes, it doesn't matter whether some values are offset by one when a decorator is applied. Regards Antoine.
Guido van Rossum wrote:
What are the use cases for co_firstlineno? Even if it is for displaying the source code, I can find virtue for both sides of this argument.
Seems to me that if the code is being displayed to a human, the decorators are an important thing to know about, so including them is good. -- Greg
On Tue, Aug 3, 2010 at 9:05 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
.. While I'd agree with you for a clean slate definition, that's not what we're dealing with here: "co_firstlineno" has an existing meaning, and it *isn't* "the line containing the def keyword", it's "the first line of the function, including any decorator lines". The decision could (and arguably should) have gone the other way when decorator syntax was first added, but changing our minds now would be making the situation worse rather than better.
I agree with Nick. What I see here is an ambiguity with good arguments available either way. Since the decision has been made, it is better to just stick to it and educate users through better documentation. Here is a copy of my private response to Eli a few days ago: """ I am not sure there is a bug there. What is wrong with the following trace? 1: def d(x): 1: return x 1: @d def f(): 1: x = 0 6: for i in range(5): 5: x += 2 1: f() Arguably, the "def" line should show up in coverage, but it is not that dissimilar from 1: def \ f(): 1: x = 0 6: for i in range(5): 5: x += 2 1: f() If you think of decorator syntax as a part of the "def" line, the first trace makes as much sense as the second. """
On 8/3/2010 8:48 AM, Antoine Pitrou wrote:
On Tue, 3 Aug 2010 22:25:01 +1000 Nick Coghlan<ncoghlan@gmail.com> wrote:
On Tue, Aug 3, 2010 at 1:40 PM, Eli Bendersky<eliben@gmail.com> wrote:
The first print out correctly specifies the line "def foo" is in. However, the second one points to the line with "@dummydecorator" instead of "def bar". [Python 2.6]
The side-effects of this behavior can be easily seen in the output of modules like trace and profile. Would you say it's normal, or could this be considered a bug?
Since the decorator is as much a part of the function definition as the def line is, I would say that it is correct (the name says "firstlineno", not "deflineno").
That's debatable. Since writing:
@b def a(): ...
is equivalent to:
def a(): ... a = b(a)
The difference is that 'a=b(a)' is a standalone statement which could moved down with other statements interposed, while '@b' is neither a statement nor expression but a def statement prefix, and is documented as such. A dynamic difference between the constructs, as least with CPython, is that the decorator form does just one namespace binding instead of two.
and in the latter case co_firstlineno points to the "def a()" line.
Furthermore, co_firstlineno is an attribute of the code object, not the function object, so it shouldn't ideally depend on whether a decorator was applied or not.
Perhaps. A practical consideration is that it is easier to search forward from the first '@' line to the 'def' line than the reverse. -- Terry Jan Reedy
participants (8)
-
Alexander Belopolsky -
Antoine Pitrou -
Eli Bendersky -
Greg Ewing -
Guido van Rossum -
Nick Coghlan -
Raghuram Devarakonda -
Terry Reedy