[Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

[Raymond Hettinger]

On Mar 11, 2011, at 11:11 PM, Eugene Toder wrote:

> Experience shows that optimizations are always error prone, no matter
> what framework or internal representation you use.

On that basis, I believe that we ought to declare peephole.c as being
somewhat off-limits for further development (except for small
adaptations if the underlying opcodes change meaning).

The code is stable and most of the low-hanging fruit has already been taken.

The bright ideas for other enhancements aren't actually new ideas.
They were considered at the outset and not included for a reason.
We were intentionally very conservative with the optimizer.


> I don't think we
> should assume that simply rewriting all optimizations to work on AST
> will make them bug free once and for all. On the contrary, I think
> such a rewrite will introduce some bugs of it's own.

You make an automatic truth when say, "I don't think we should assume X".

It would be more correct to say that peepholing is the wrong technique
for constant folding because:

1) much of the semantic information available the AST has been lost
by the time the bytecode is generated

2) peephole.c is having to disassemble the bytecode, reconstruct
semantic relationships as well as it can, recognize transformation
patterns and then regenerate bytecode.
Moving constant folding to the AST means that the disassembly and reassembly
don't have to be done and that more semantic information is available.

In other words, the two techniques are equal just because it is always
possible to make a mistake using any technique.


> The remedy against this is well known.
> Instead of being afraid to
> touch the code we should add more tests and verifiers.

Not really.  It is darned difficult to design tests to catch all the corner cases.
You've got wishful thinking if you think a handful of tests can catch
errors in code that sophisticated.  Look at GCC's bug history to get
a sense for what can go wrong with optimization.

I admire your and Antoine's enthusiasm for wanting to optimize,
but think your efforts are misdirected and that you should be
taking some advice from the guy who wrote thing in the first place.


Raymond