On Sat, Jan 1, 2011 at 1:52 AM, Cesare Di Mauro
2011/1/1 Steven D'Aprano
I wonder whether we need to make that guarantee? Perhaps we should distinguish between "safe" optimizations, like constant folding which can't change behaviour, and "unsafe" optimizations which can go wrong under (presumably) rare circumstances. The compiler can continue to apply whatever safe optimizations it likes, but unsafe optimizations must be explicitly asked for by the user. If subtle or not subtle bugs occur, well, Python does allow people to shoot themselves in the foot.
Do we consider local variable removing (due to internal optimizations) a safe or unsafe operation?
I would consider it safe unless the function locals() is called directly in the function -- always assuming the compiler does not see obvious other evidence (like a local being used by a nested function). Locals are "special" in many ways already. There should be a way to disable this (globally) in case you want to step through the code with a debugger though -- IDEs like WingIDE and PyCharm make stepping through code very easy to set up, and variable inspection is a big part of the process of debugging this way. It's probably fine if such optimizations are only enabled by -O. Still, I wonder if it isn't much better to try to do this using a JIT instead of messing with the bytecode. You'll find that the current compiler just really doesn't have the datastructures needed to do these kind of optimizations right.
Do we consider local variable values "untouchable"? Think about a locals() call that return a list for a variable; lists are mutable objects, so they can be changed by the caller, but the internally generated bytecode can work on a "private" (on stack) copy which doesn't "see" the changes made due to the locals() call.
Are you sure? locals() makes only a shallow copy, so changes to the list's contents made via the list returned by locals() should be completely visible by the bytecode.
Also, there's the tracing to consider. When trace is enabled, the "handler" cannot find some variables due to some optimizations.
Tracing is a special case of debugging.
Another funny thing that can happen is that if I "group together" some assignment operations into a single, "multiassignment", one (it's another optimization I was thinking about from long time) and you are tracing it, only one tracing event will be generated instead of n. Are such optimizations "legal" / "safe"? For me the answer is yes, because I think that they must be implementation-specific.
I've traced through C code generated by gcc with an optimization setting. It can be a bit confusing to be jumping around in the optimized code, and it's definitely easier to trace through unoptimized code, but if you have the choice between tracing the (optimized) binary you have, or not tracing at all, I'll take what I can get. Still, when you're planning to trace/debug it's better to have a flag to disable it, and not using -O sounds like the right thing to me. -- --Guido van Rossum (python.org/~guido)