[Python-ideas] Python-ideas Digest, Vol 90, Issue 30

[Ned Batchelder]

On 5/22/14 11:43 AM, Ethan Furman wrote:
> On 05/22/2014 08:32 AM, Ned Batchelder wrote:
>> On 5/22/14 9:49 AM, Skip Montanaro wrote:
>>> On Thu, May 22, 2014 at 8:05 AM, Chris Angelico wrote:
>>>>
>>>> Correct me if I'm wrong, but as I understand it, the problem is that
>>>> the peephole optimizer eliminated an entire line of code. Would it be
>>>> possible to have it notice when it merges two pieces from different
>>>> lines, and somehow mark that the resulting bytecode comes from both
>>>> lines? That would solve the breakpoint and coverage problems
>>>> simultaneously.
>>>
>>> It seems to me that Ned has revealed a bug in the peephole optimizer.
>>> It zapped an entire source line's worth of bytecode, but failed to
>>> delete the relevant entry in the line number table of the resulting
>>> code object. If I had my druthers, that would be the change I'd
>>> prefer.
>>
>> I think it is the nature of optimization that it will destroy useful 
>> information.  I don't think it will always be
>> possible to retain enough back-mapping that the optimized code can be 
>> understood as if it had not been optimized.   For
>> example, the debug issue would still be present: if you run pdb and 
>> set a breakpoint on the "continue" line, it will
>> never be hit.  Even if the optimizer cleaned up after itself 
>> perfectly (in fact, especially so), that breakpoint will
>> still not be hit.  You simply cannot reason about optimized code 
>> without having to mentally understand the
>> transformations that have been applied.
>>
>> The whole point of this proposal is to recognize that there are times 
>> (debugging, coverage measurement) when
>> optimizations are harmful, and to avoid them.
>
> Having read through the issue on the tracker, I find myself swayed 
> towards Neds point of view.  However, I do still agree with Raymond 
> that a full-fledged command-line switch is overkill, especially since 
> the unoptimized runs are very special-cased (meaning useful for 
> debugging, coverage, curiosity, learning about optimizing, etc).
>

I'm perfectly happy to drop the idea of the command-line switch.  An 
environment variable would be a fine way to control this behavior.

> If we had a sys flag that could be set before a module was loaded, 
> then coverage, pdb, etc., could use that to recompile the source, not 
> save a .pyc file, and move forward.  For debugging purposes perhaps a 
> `__no_optimize__ = True` or `from __future__ import no_optimize` would 
> help in those cases where you're dropping into the debugger.

I don't understand these ideas, but having to add an import to the top 
of the file seems like a non-starter to me.

>
> The dead-code elimination still has a bug to be fixed, though, because 
> if a line has been optimized away trying to set a break-point at it 
> should fail.

If we get a way to disable optimization, we don't need to fix that bug.  
Everyone knows that optimized code acts oddly in debuggers. :)

>
> -- 
> ~Ethan~