[Python-ideas] Optimizing builtins

[Guido van Rossum]

On Sat, Jan 1, 2011 at 1:52 AM, Cesare Di Mauro
<cesare.di.mauro at gmail.com> wrote:
> 2011/1/1 Steven D'Aprano <steve at pearwood.info>
>>
>> I wonder whether we need to make that guarantee? Perhaps we should
>> distinguish between "safe" optimizations, like constant folding which can't
>> change behaviour, and "unsafe" optimizations which can go wrong under
>> (presumably) rare circumstances. The compiler can continue to apply whatever
>> safe optimizations it likes, but unsafe optimizations must be explicitly
>> asked for by the user. If subtle or not subtle bugs occur, well, Python does
>> allow people to shoot themselves in the foot.
>
> Do we consider local variable removing (due to internal optimizations) a
> safe or unsafe operation?

I would consider it safe unless the function locals() is called
directly in the function -- always assuming the compiler does not see
obvious other evidence (like a local being used by a nested function).
Locals are "special" in many ways already. There should be a way to
disable this (globally) in case you want to step through the code with
a debugger though -- IDEs like WingIDE and PyCharm make stepping
through code very easy to set up, and variable inspection is a big
part of the process of debugging this way. It's probably fine if such
optimizations are only enabled by -O.

Still, I wonder if it isn't much better to try to do this using a JIT
instead of messing with the bytecode. You'll find that the current
compiler just really doesn't have the datastructures needed to do
these kind of optimizations right.

> Do we consider local variable values "untouchable"? Think about a locals()
> call that return a list for a variable; lists are mutable objects, so they
> can be changed by the caller, but the internally generated bytecode can work
> on a "private" (on stack) copy which doesn't "see" the changes made due to
> the locals() call.

Are you sure? locals() makes only a shallow copy, so changes to the
list's contents made via the list returned by locals() should be
completely visible by the bytecode.

> Also, there's the tracing to consider. When trace is enabled, the "handler"
> cannot find some variables due to some optimizations.

Tracing is a special case of debugging.

> Another funny thing that can happen is that if I "group together" some
> assignment operations into a single, "multiassignment", one (it's another
> optimization I was thinking about from long time) and you are tracing it,
> only one tracing event will be generated instead of n.
> Are such optimizations "legal" / "safe"? For me the answer is yes, because I
> think that they must be implementation-specific.

I've traced through C code generated by gcc with an optimization
setting. It can be a bit confusing to be jumping around in the
optimized code, and it's definitely easier to trace through
unoptimized code, but if you have the choice between tracing the
(optimized) binary you have, or not tracing at all, I'll take what I
can get. Still, when you're planning to trace/debug it's better to
have a flag to disable it, and not using -O sounds like the right
thing to me.

-- 
--Guido van Rossum (python.org/~guido)