[Python-Dev] PEP 488: elimination of PYO files

[Brett Cannon]

On Fri, Mar 6, 2015 at 1:27 PM Neil Girdhar <mistersheik at gmail.com> wrote:

> On Fri, Mar 6, 2015 at 1:11 PM, Brett Cannon <brett at python.org> wrote:
>
>>
>>
>> On Fri, Mar 6, 2015 at 1:03 PM Mark Shannon <mark at hotpy.org> wrote:
>>
>>>
>>> On 06/03/15 16:34, Brett Cannon wrote:
>>> > Over on the import-sig I proposed eliminating the concept of .pyo files
>>> > since they only signify that /some/ optimization took place, not
>>> > /what/ optimizations took place. Everyone on the SIG was positive with
>>> > the idea so I wrote a PEP, got positive feedback from the SIG again,
>>> and
>>> > so now I present to you PEP 488 for discussion.
>>> >
>>> [snip]
>>>
>>> Historically -O and -OO have been the antithesis of optimisation, they
>>> change the behaviour of the program with no noticeable effect on
>>> performance.
>>> If a change is to be made, why not just drop .pyo files and be done with
>>> it?
>>>
>>
>> I disagree with your premise that .pyo files don't have a noticeable
>> effect on performance. If you don't use asserts a lot then there is no
>> effect, but if you use them heavily or have them perform expensive
>> calculations then there is an impact. And the dropping of docstrings does
>> have an impact on memory usage when you use Python at scale.
>>
>> You're also assuming that we will never develop an AST optimizer that
>> will go beyond what the peepholer can do based on raw bytecode, or
>> something that involves a bit of calculation and thus something you
>> wouldn't want to do at startup.
>>
>
> I don't want to speak for him, but you're going to get the best results
> optimizing ASTs at runtime, which is what I thought he was suggesting.
> Trying to optimize Python at compile time is setting your sights really
> low.   You have so little information then.
>

OK, I don't want to derail the discussion of the PEP into one over how best
to optimize CPython's performance relative to bytecode vs. runtime like
PyPy. The point is that we have -O and -OO and people do have uses for
those flags. People can also do custom optimizations thanks to the
flexibility of loaders.

All of this leads to wanting different bytecode files for different
optimization levels to make sure you're actually executing your code with
the optimizations you expect. If people think that optimizing code and
surfacing it in bytecode files is a waste and want to suggest either
dropping .pyo files entirely or dropping -O and only having -OO that's
fine, but that is not what this PEP is proposing nor a PEP I want to bother
writing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150306/9b76c9e2/attachment.html>