[Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Eugene Toder]

>> Experience shows that optimizations are always error prone, no matter
>> what framework or internal representation you use.
>
> On that basis, I believe that we ought to declare peephole.c as being
> somewhat off-limits for further development (except for small
> adaptations if the underlying opcodes change meaning).

With this attitude, it would be more logical to remove the whole thing
completely. Even if you don't touch the optimizer, any changes in the
compiler can start producing code that exposes a previously hidden
bug.

> The bright ideas for other enhancements aren't actually new ideas.
> They were considered at the outset and not included for a reason.

It would help if these were documented.

> It would be more correct

How can you be more correct than tautological truth? :)

> 1) much of the semantic information available the AST has been lost
> by the time the bytecode is generated

Not really. Many systems use stack based bytecode as an intermediate
form and reconstruct different IRs from it. Examples are JVM, CLR and
PyPy.
The real problem with bytecode is that it's a representation optimized
for storage and interpretation. Transforming it is harder, less
efficient and more error-prone than transforming a tree structure. But
transformations of trees are in no way automatically bug free.

> 2) peephole.c is having to disassemble the bytecode, reconstruct
> semantic relationships as well as it can, recognize transformation
> patterns and then regenerate bytecode.

v8 does peephole directly on x86 machine code. No kittens were harmed.

Now, I do agree with you that moving optimizations to AST is a step
forward. Moving them to a specialized IR in SSA form would be even
better.
If there was a framework for AST-based optimizations in the current
code, I'd write my code to use it. However, I couldn't find it.

What I don't understand is why doing 90% of the work and refuse to do
the last 10%. Looking at the actual patches, I do not see significant
increase in complexity or code size -- all the complex cases are
already handled by existing code. In some cases the code became
cleaner and simpler.

> Not really.  It is darned difficult to design tests to catch all the corner cases.

You either do that or do not write optimizations. No one said it should be easy.

> You've got wishful thinking if you think a handful of tests can catch
> errors in code that sophisticated.

Why limit yourself with a handful of tests? Python is widespread,
there's *a lot* of code in Python. Unlike with libraries, any code you
run tests the optimizer, so just run a lot of code. And, as I've said,
write a test generator.

Cheers,
Eugene