[Python-ideas] Changes to the existing optimization levels

[Nick Coghlan]

On 29 September 2017 at 05:02, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 28 Sep 2017 12:48:15 -0600
> Diana Clarke
> <diana.joan.clarke at gmail.com> wrote:
>>
>>     2) Added a new command line option N that allows you to specify
>> any number of individual optimization flags.
>>
>>     For example:
>>
>>         python -N nodebug -N noassert -N nodocstring
>
> We could instead reuse the existing -X option, which allows for
> free-form implementation-specific flags.

And declaring named optimisation flags to be implementation dependent
is likely a good way to go.

The one downside is that it would mean there was no formally
interpreter independent way of requesting the "nodebug,nodocstring"
configuration, but informal conventions around particular uses of "-X"
may be sufficient for that purpose.

>> I'm also not certain if the various compile signatures are even open
>> for change (int optimize => PyObject *optimizations), or if that's a
>> no-no.
>
> You probably want to keep the existing signatures for compatibility:
> - in C, add new APIs with the new convention
> - in Python, add a new (optional) function argument for the new
>   convention

This approach should also reduce the overall amount of code churn,
since any CPython (or external) code currently passing "optimize=-1"
won't need to change at all: that already says "get the optimization
settings from the interpreter state", so it will pick up any changes
to how that configuration works "for free".

That said, we may also want to consider a couple of other options
related to changing the meaning of *existing* parameters to these
APIs:

1. We have the PyCompilerFlags struct that's currently only used to
pass around feature flags for the __future__ module. It could gain a
second bitfield for optimisation options
2. We could reinterpret "optimize" as a bitfield instead of a regular
integer, special casing the already defined values:

- all zero: no optimizations
- sign bit set: negative -> use global settings
- 0x0001: nodebug+noassert
- 0x0002: nodebug+noassert+nodocstrings
- 0x0004: nodebug
- 0x0008: noassert
- 0x0010: nodocstrings

The "redefine optimizations as a bitfield" approach seems particularly
promising to me - it's a full integer, so even with all negative
numbers disallowed and the two low order bits reserved for the legacy
combinations, that's still 29 different optimisation flags given
32-bit integers. We currently have 3, so that's room for an 866%
increase in the number of defined flags :)

The opt-N values in pyc files would be somewhat cryptic-to-humans, but
still relatively easy to translate back to readable strings given the
bitfield values, and common patterns (like 0x14 -> 20 for
nodebug+nodocstrings) would likely become familiar pretty quickly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia