Re: A “big picture” question.
On Fri, Jun 10, 2022 at 09:59:36PM -0500, James Johnson wrote:
No stress -- opening issues up for discussion is not a waste of time. This would be a good time to mention that there have been previous requests to have more control of what optimizations the Python byte-code compiler performs, mostly for the benefit of profiling applications. While the compiler doesn't do many, or any, large complex optimizations like a C compiler may do, it does do some keyhole optimizations. Sometimes those keyhole optimizations interfere with the ability of programs to analyse Python code and report on code coverage. While the keyhole optimization doesn't change the semantics of the code, it does change the structure of it, and makes it harder to analyse whether or not each clause in a statement is covered by tests. So other people have also requested the ability to tell the compiler to turn off all optimizations. Another factor is that as we speak, Mark Shannon is doing a lot of work on optimization for the CPython byte-code compiler, including adding JIT compilation techniques. (PyPy has had this ability for many years.) So it is possible that future compiler optimizations may start to move into the same areas that C/C++ compiler optimizations take, possibly even changing the meaning of code. It would be good to plan ahead, and start considering more fine grained optimization control, rather than the underpowered -O and -OO flags we have now. -- Steve
I would note that the accepted [PEP 626](https://peps.python.org/pep-0626/) explicitly constrains line-tracing behavior: """Python should guarantee that when tracing is turned on, “line” tracing events are generated for all lines of code executed and only for lines of code that are executed.""" So even peephole optimizations should now theoretically follow PEP 626 and produce the expected line-tracing events. For example, the line "try:" typically emits a "NOP" instruction that is kept around just for the sake of tracing. If I recall correctly, there might not be 100% compliance with PEP 626 so far, but in general, things have recently gotten more well-specified and predictable in this regard, not less. The interpreter is still allowed to go wild by, e.g., executing type-specialized versions of different opcodes (PEP 659), but just not in such a way as to change language semantics, including the semantics of tracing when tracing is enabled.
I would note that the accepted [PEP 626](https://peps.python.org/pep-0626/) explicitly constrains line-tracing behavior: """Python should guarantee that when tracing is turned on, “line” tracing events are generated for all lines of code executed and only for lines of code that are executed.""" So even peephole optimizations should now theoretically follow PEP 626 and produce the expected line-tracing events. For example, the line "try:" typically emits a "NOP" instruction that is kept around just for the sake of tracing. If I recall correctly, there might not be 100% compliance with PEP 626 so far, but in general, things have recently gotten more well-specified and predictable in this regard, not less. The interpreter is still allowed to go wild by, e.g., executing type-specialized versions of different opcodes (PEP 659), but just not in such a way as to change language semantics, including the semantics of tracing when tracing is enabled.
participants (2)
-
Dennis Sweeney -
Steven D'Aprano