[Python-ideas] PEP 511: API for code transformers

[Nick Coghlan]

On 17 January 2016 at 04:28, Brett Cannon <brett at python.org> wrote:
>
> On Fri, 15 Jan 2016 at 09:40 Victor Stinner <victor.stinner at gmail.com>
> wrote:
>>
>> 2016-01-15 18:22 GMT+01:00 Brett Cannon <brett at python.org>:
>> > I just wanted to point out to people that the key part of this PEP is
>> > the
>> > change in semantics of `-O` accepting an argument.
>>
>> The be exact, it's a new "-o arg" option, it's different from -O and
>> -OO (uppercase). Since I don't know what to do with -O and -OO, I
>> simply kept them :-D
>>
>> > I should also point out that this does get tricky in terms of how to
>> > handle
>> > the stdlib if you have not pre-compiled it, e.g., if the first module
>> > imported by Python is the encodings module then how to make sure the AST
>> > optimizers are ready to go by the time that import happens?
>>
>> Since importlib reads sys.implementation.optim_tag at each import, it
>> works fine.
>>
>> For example, you start with "opt" optimizer tag. You import everything
>> needed for fatoptimizer. Then calling sys.set_code_transformers() will
>> set a new optimizer flag (ex: "fat-opt"). But it works since the
>> required code transformers are now available.
>
>
> I understand all of that; my point is what if you don't compile the stdlib
> for your optimization? You have to import over 20 modules before user code
> gets imported. My question is how do you expect the situation to be handled
> where you didn't optimize the stdlib since the 'encodings' module is
> imported before anything else? If you set your `-o` flag and you want to
> fail imports if the .pyc isn't there, then wouldn't that mean you are going
> to fail immediately when you try and import 'encodings' in Py_Initialize()?

I don't think that's a major problem - it seems to me that it's the
same as going for "pyc only" deployment with an embedded Python
interpreter, and then forgetting to a precompiled standard library in
addition to your own components. Yes, it's going to fail, but the bug
is in the build process for your deployment artifacts rather than in
the runtime behaviour of CPython.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia