[Python-Dev] Release of astoptimizer 0.3

[Guido van Rossum]

On Tue, Sep 11, 2012 at 10:06 AM, MRAB <python at mrabarnett.plus.com> wrote:
> On 11/09/2012 13:06, Victor Stinner wrote:
>>>>
>>>> * Call builtin functions if arguments are constants. Examples:
>>>>
>>>>   - len("abc") => 3
>>>>   - ord("A") => 65
>>>
>>>
>>> Does it preserve python semantics? What if you change the len builtin?
>>
>>
>> This optimization is disabled by default (in the version 0.3), because
>> builtin functions may be shadowed. Examples: "len=ord;
>> print(len('A'))" or exec(compile("print(len('A'))", "test", "exec"),
>> {'len': ord}).
>>
>> If you know that one specific builtin function is shadowed (ex:
>> print), you can modify the configuration to enable optimizations on
>> builtin functions except the specified function.
>>
>> Do you projects where builtin functions are shadowed?
>>
>> The idea is that the author of the application knows its application
>> (... and all modules used by applications) and is able to explicitly
>> specify what is that "constant". You can for example declare some
>> variables as constant using Config.add_constant("module.name", value).
>> In the same manner, you can declare "pure" functions (without border
>> effect).
>>
> Do you know what the cost would be of, say, replacing:
>
>     len("abc")
>
> with:
>
>     3 if len is __builtins__.len else len("abc")
>
> if possible where the lookup __builtins__.len is been done early, such
> as at compile time?

Doing the lookup at compile time would defeat the whole purpose (the
dynamic shadowing of len could happen at any time). Also there are two
forms of shadowing: patching the globals of the module where the call
occurs, or patching the builtins.

Doing the lookup at run time would also defeat the purpose, because
most of the cost of calling len() *is* the lookup. Actually two
lookups -- the LOAD_GLOBAL opcode is used, which first looks in the
globals (this will normally always fail) and then in the builtins.

FWIW, I expect that there are few places where len(<constant string>)
is actually used. However I expect it to be quite common for ord(),
where the same reasoning applies.

-- 
--Guido van Rossum (python.org/~guido)