On behalf of the steering council, I am happy to announce that as BDFL-Delegate I am accepting PEP 626 -- Precise line numbers for debugging and other tools. I am confident this PEP will result in a better experience for debuggers, profilers and tools that rely on tracing functions. All the existing concerns regarding out-of-process debuggers and profilers have been addressed by Mark in the latest version of the PEP. The acceptance of the PEP comes with the following requests:
- The PEP must be updated to explicitly state that the API functions described in the "Out of process debuggers and profilers" must remain self-contained in any potential future modifications or enhancements.
- The PEP states that the "f_lineno" attribute of the code object will be updated to point to the current line being executed even if tracing is off. Also, there were some folks concerned with possible performance implications. Although in my view there is no reason to think this will impact performance negatively, I would like us to confirm that indeed this is the case before merging the implementation (with the pyperformance test suite, for example).
Congratulations Mark Shannon!
Thanks also to everyone else who provided feedback on this PEP!
Regards from rainy London, Pablo Galindo Salgado
Hi,
That's great. Thanks Pablo.
On 29/10/2020 1:32 am, Pablo Galindo Salgado wrote:
On behalf of the steering council, I am happy to announce that as BDFL-Delegate I am accepting PEP 626 -- Precise line numbers for debugging and other tools. I am confident this PEP will result in a better experience for debuggers, profilers and tools that rely on tracing functions. All the existing concerns regarding out-of-process debuggers and profilers have been addressed by Mark in the latest version of the PEP. The acceptance of the PEP comes with the following requests:
- The PEP must be updated to explicitly state that the API functions described in the "Out of process debuggers and profilers" must remain self-contained in any potential future modifications or enhancements.
- The PEP states that the "f_lineno" attribute of the code object will be updated to point to the current line being executed even if tracing is off. Also, there were some folks concerned with possible performance implications. Although in my view there is no reason to think this will impact performance negatively, I would like us to confirm that indeed this is the case before merging the implementation (with the pyperformance test suite, for example).
Performance compared to what?
The current behavior of f_lineno is ill-defined, so mimicking it would
be tricky.
What's the reason for supposing that it will be slower?
Cheers, Mark.
Congratulations Mark Shannon!
Thanks also toeveryone else who provided feedback on this PEP!
Regards from rainy London, Pablo Galindo Salgado
Performance compared to what?
Compared before the patch. The comparison that I mentioned is before and after the PR with the PEP implementation.
The current behavior of f_lineno is
ill-defined, so mimicking it would
be tricky
Maybe I failed to express myself: that's fine, we don't need to mimick the current behaviour of f_lineno or change anything in the PEP regarding that. I just want to check that the new semantics do not slow down anything in a subtle way.
What's the reason for supposing that it will be slower?
There is no real concern, but as there were some conversations about performance and the pep mentions that "the "f_lineno" attribute of the code object will be updated to point the current line being executed" I just want to make sure that updating that field on every bytecode line change does not affect anything. Again, I am pretty sure it will be negligible impact and the performance check should be just a routine confirmation.
Cheers, Pablo
On Thu, 29 Oct 2020, 09:47 Mark Shannon, mark@hotpy.org wrote:
Hi,
That's great. Thanks Pablo.
On 29/10/2020 1:32 am, Pablo Galindo Salgado wrote:
On behalf of the steering council, I am happy to announce that as BDFL-Delegate I am accepting PEP 626 -- Precise line numbers for debugging and other tools. I am confident this PEP will result in a better experience for debuggers, profilers and tools that rely on tracing functions. All the existing concerns regarding out-of-process debuggers and profilers have been addressed by Mark in the latest version of the PEP. The acceptance of the PEP comes with the following requests:
- The PEP must be updated to explicitly state that the API functions described in the "Out of process debuggers and profilers" must remain self-contained in any potential future modifications or enhancements.
- The PEP states that the "f_lineno" attribute of the code object will be updated to point to the current line being executed even if tracing is off. Also, there were some folks concerned with possible performance implications. Although in my view there is no reason to think this will impact performance negatively, I would like us to confirm that indeed this is the case before merging the implementation (with the pyperformance test suite, for example).
Performance compared to what?
The current behavior of f_lineno is ill-defined, so mimicking it would
be tricky.
What's the reason for supposing that it will be slower?
Cheers, Mark.
>
Congratulations Mark Shannon!
Thanks also toeveryone else who provided feedback on this PEP!
Regards from rainy London, Pablo Galindo Salgado
Hi Pablo,
On 29/10/2020 9:56 am, Pablo Galindo Salgado wrote:
Performance compared to what?
Compared before the patch. The comparison that I mentioned is before and after the PR with the PEP implementation.
PEP 626 changes the line number table and the compiler.
Such a comparison would not test the performance impact of the change to
f_lineno, as it will likely be swamped by the other changes.
The current
behavior of f_lineno is ill-defined, so mimicking it would
be tricky
Maybe I failed to express myself: that's fine, we don't need to mimick the current behaviour of f_lineno or change anything in the PEP regarding that. I just want to check that the new semantics do not slow down anything in a subtle way.
The new semantics may well result in some slowdowns. That's stated in the PEP. I don't think I can reliably isolate the effects of the (very slight) change in the behavior of f_lineno.
What's the reason for supposing that it will be slower?
There is no real concern, but as there were some conversations about performance and the pep mentions that "the "f_lineno" attribute of the code object will be updated to point the current line being executed" I just want to make sure that updating that field on every bytecode line change does not affect anything. Again, I am pretty sure it will be negligible impact and the performance check should be just a routine confirmation.
When you say updating "field", are you thinking of a C struct? That's an implementation detail. The PEP states that the f_lineno attribute of the code object will be updated.
Note that the behavior of 3.9 is weird in some cases:
test.py: import sys
def print_line():
print(sys._getframe(1).f_lineno)
def test():
print_line()
sys._getframe(0).f_trace = True
print_line()
print_line()
test()
$ python3.9 ~/test/test.py 7 8 8
With PEP 626 this is required to print: 7 9 10
Cheers, Mark.
Cheers, Pablo
On Thu, 29 Oct 2020, 09:47 Mark Shannon, <mark@hotpy.org
mailto:mark@hotpy.org> wrote:
Hi,
That's great. Thanks Pablo.
On 29/10/2020 1:32 am, Pablo Galindo Salgado wrote:
> On behalf of the steering council, I am happy to announce that as
> BDFL-Delegate I am
> accepting PEP 626 -- Precise line numbers for debugging and other
tools.
> I am confident this PEP will result in a better experience for
> debuggers, profilers and tools
> that rely on tracing functions. All the existing concerns regarding
> out-of-process debuggers
> and profilers have been addressed by Mark in the latest version
of the
> PEP. The acceptance of
> the PEP comes with the following requests:
>
> * The PEP must be updated to explicitly state that the API functions
> described in the
> "Out of process debuggers and profilers" must remain
self-contained
> in any potential
> future modifications or enhancements.
> * The PEP states that the "f_lineno" attribute of the code object
will
> be updated to point to
> the current line being executed even if tracing is off. Also,
there
> were some folks concerned with
> possible performance implications. Although in my view there
is no
> reason to think this will impact
> performance negatively, I would like us to confirm that
indeed this
> is the case before merging the
> implementation (with the pyperformance test suite, for example).
Performance compared to what?
The current behavior of `f_lineno` is ill-defined, so mimicking it
would
be tricky.
What's the reason for supposing that it will be slower?
Cheers,
Mark.
>
> Congratulations Mark Shannon!
>
> Thanks also toeveryone else who provided feedback on this PEP!
>
> Regards from rainy London,
> Pablo Galindo Salgado
The new semantics may well result in some slowdowns. That's stated in the PEP. I don't think I can reliably isolate the effects of the (very slight) change in the behavior of f_lineno.
Ok, then let's make at least we measure the general slowdowns.
When you say updating "field", are you thinking of a C struct? That's an implementation detail. The PEP states that the f_lineno attribute of the code object will be updated.
Any of them, but as you mentioned isolating the effect of that it may be not worth it. Let's measure the impact of the whole change as a whole then.
Cheers, Pablo
On Thu, 29 Oct 2020 at 10:55, Mark Shannon mark@hotpy.org wrote:
Hi Pablo,
On 29/10/2020 9:56 am, Pablo Galindo Salgado wrote:
Performance compared to what?
Compared before the patch. The comparison that I mentioned is before and after the PR with the PEP implementation.
PEP 626 changes the line number table and the compiler.
Such a comparison would not test the performance impact of the change to
f_lineno, as it will likely be swamped by the other changes.
>
The
current behavior of f_lineno is ill-defined, so mimicking it
would
be tricky
Maybe I failed to express myself: that's fine, we don't need to mimick the current behaviour of f_lineno or change anything in the PEP regarding that. I just want to check that the new semantics do not slow down anything in a subtle way.
The new semantics may well result in some slowdowns. That's stated in the PEP. I don't think I can reliably isolate the effects of the (very slight) change in the behavior of f_lineno.
>
What's the reason for supposing that it will be slower?
There is no real concern, but as there were some conversations about performance and the pep mentions that "the "f_lineno" attribute of the code object will be updated to point the current line being executed" I just want to make sure that updating that field on every bytecode line change does not affect anything. Again, I am pretty sure it will be negligible impact and the performance check should be just a routine confirmation.
When you say updating "field", are you thinking of a C struct? That's an implementation detail. The PEP states that the f_lineno attribute of the code object will be updated.
Note that the behavior of 3.9 is weird in some cases:
test.py: import sys
def print_line():
print(sys._getframe(1).f_lineno)
def test():
print_line()
sys._getframe(0).f_trace = True
print_line()
print_line()
test()
$ python3.9 ~/test/test.py 7 8 8
With PEP 626 this is required to print: 7 9 10
Cheers, Mark.
>
Cheers, Pablo
On Thu, 29 Oct 2020, 09:47 Mark Shannon, <mark@hotpy.org
mailto:mark@hotpy.org> wrote:
Hi,
That's great. Thanks Pablo.
On 29/10/2020 1:32 am, Pablo Galindo Salgado wrote:
> On behalf of the steering council, I am happy to announce that as
> BDFL-Delegate I am
> accepting PEP 626 -- Precise line numbers for debugging and other
tools.
> I am confident this PEP will result in a better experience for
> debuggers, profilers and tools
> that rely on tracing functions. All the existing concerns
regarding
> out-of-process debuggers
> and profilers have been addressed by Mark in the latest version
of the
> PEP. The acceptance of
> the PEP comes with the following requests:
>
> * The PEP must be updated to explicitly state that the
API functions
> described in the
> "Out of process debuggers and profilers" must remain
self-contained
> in any potential
> future modifications or enhancements.
> * The PEP states that the "f_lineno" attribute of the code object
will
> be updated to point to
> the current line being executed even if tracing is off. Also,
there
> were some folks concerned with
> possible performance implications. Although in my view there
is no
> reason to think this will impact
> performance negatively, I would like us to confirm that
indeed this
> is the case before merging the
> implementation (with the pyperformance test suite, for
example).
Performance compared to what?
The current behavior of `f_lineno` is ill-defined, so mimicking it
would
be tricky.
What's the reason for supposing that it will be slower?
Cheers,
Mark.
>
> Congratulations Mark Shannon!
>
> Thanks also toeveryone else who provided feedback on this PEP!
>
> Regards from rainy London,
> Pablo Galindo Salgado
Hi Pablo,
On 29/10/2020 11:08 am, Pablo Galindo Salgado wrote:
The new semantics may well result in some slowdowns. That's stated in the PEP.I don't think I can reliably isolate the effects of the (very slight) change in the behavior of f_lineno.
Ok, then let's make at least we measure the general slowdowns.
Except that we can't measure the performance of a specification. We can only measure the performance of entire implementations.
I can make an implementation that conforms to PEP 626 that is slower than master, or I can make one that's faster :) It doesn't change the value of the PEP itself.
Let me give you a toy example.
def f(): while 1: body()
3.9 compiles this to:
(The trailing, implicit return has been stripped for clarity)
3 >> 0 LOAD_GLOBAL 0 (body) 2 CALL_FUNCTION 0 4 POP_TOP 6 JUMP_ABSOLUTE 0
A naive implementation that conforms to PEP 626 would this compile to:
2 >> 0 NOP 3 2 LOAD_GLOBAL 0 (body) 4 CALL_FUNCTION 0 6 POP_TOP 8 JUMP_ABSOLUTE 0
But a better implementation could produce this:
2 0 NOP 3 >> 2 LOAD_GLOBAL 0 (body) 4 CALL_FUNCTION 0 6 POP_TOP 2 8 JUMP_ABSOLUTE 2
Which has the same bytecodes as 3.9 in the loop, and has the correct line numbers.
Cheers, Mark.
Except that we can't measure the performance of a specification. We can only measure the performance of entire implementations. I can make an implementation that conforms to PEP 626 that is slower than master, or I can make one that's faster :) It doesn't change the value of the PEP itself.
Oh, I see now where the confusion is. Regarding the benchmarks, unless is catastrophically worse and there is no way to make it competitive, the PEP acceptance is not conditional to the specific result of the benchmarks: my request is just to have a benchmark on the implementation :)
On Thu, 29 Oct 2020 at 13:29, Mark Shannon mark@hotpy.org wrote:
Hi Pablo,
On 29/10/2020 11:08 am, Pablo Galindo Salgado wrote:
The new semantics may well result in some slowdowns. That's stated in the PEP.I don't think I can reliably isolate the effects of the (very slight) change in the behavior of f_lineno.
Ok, then let's make at least we measure the general slowdowns.
Except that we can't measure the performance of a specification. We can only measure the performance of entire implementations.
I can make an implementation that conforms to PEP 626 that is slower than master, or I can make one that's faster :) It doesn't change the value of the PEP itself.
Let me give you a toy example.
def f(): while 1: body()
3.9 compiles this to:
(The trailing, implicit return has been stripped for clarity)
3 >> 0 LOAD_GLOBAL 0 (body) 2 CALL_FUNCTION 0 4 POP_TOP 6 JUMP_ABSOLUTE 0
A naive implementation that conforms to PEP 626 would this compile to:
2 >> 0 NOP 3 2 LOAD_GLOBAL 0 (body) 4 CALL_FUNCTION 0 6 POP_TOP 8 JUMP_ABSOLUTE 0
But a better implementation could produce this:
2 0 NOP 3 >> 2 LOAD_GLOBAL 0 (body) 4 CALL_FUNCTION 0 6 POP_TOP 2 8 JUMP_ABSOLUTE 2
Which has the same bytecodes as 3.9 in the loop, and has the correct line numbers.
Cheers, Mark.
-
Mark Shannon -
Pablo Galindo Salgado