[Python-ideas] Optimizing builtins

[Stefan Behnel]

Guido van Rossum, 01.01.2011 17:50:
> On Fri, Dec 31, 2010 at 5:52 PM, Steven D'Aprano wrote:
>> and "unsafe" optimizations which can go wrong under
>> (presumably) rare circumstances. The compiler can continue to apply whatever
>> safe optimizations it likes, but unsafe optimizations must be explicitly
>> asked for by the user. If subtle or not subtle bugs occur, well, Python does
>> allow people to shoot themselves in the foot.
>>
>> There's precedence for this. Both -O and -OO optimization switches
>> potentially change behaviour. -O *should* be safe if code only uses asserts
>> for assertions, but many people (especially beginners) use assert for input
>> checking. If their code breaks under -O they have nobody to blame but
>> themselves. Might we not say that -OO will optimize access to builtins, and
>> if things break, the solution is not to use -OO?
>
> Maybe. But that means it will probably rarely be used --
> realistically, who uses -O or -OO? I don't ever. Even so, there would
> have to be a way to turn the optimization off even under -OO for a
> particular module or function or code location, or for a particular
> builtin (again, open() comes to mind).

If this ever happes, -O and -OO will no longer be expressive enough (IMHO, 
-OO currently isn't anyway). There would be a need to support options like 
"-Ostatic-builtins" and the like. The problem then is how to keep users 
from applying a particular optimisation to a particular module. New 
settings in distutils could help to enable optimisations and maybe even to 
explicitly forbid optimisations, but life would certainly become more error 
prone for distributors and users. It's hard to keep track of the amount of 
bug reports and help requests we get from (mostly new) Cython users about a 
missing "-fno-strict-aliasing" when compiled modules don't work in Python 
2. I expect the same to come up when users start to install Python modules 
with all sorts of great CPython optimisations. Test suites may well fail to 
catch the one bug that an optimisation triggers.


>> Here's another thought... suppose (say) "builtin" became a reserved word.
>> builtin.range (for example) would always refer to the built-in range, and
>> could be optimized by the compiler. It wouldn't do much for the general case
>> of wanting to optimize non-built-in globals, but this could be optimized
>> safely:
>>
>> def f():
>>     for i in builtin.range(10): builtin.print(i)
>>
>> while this would keep the current semantics:
>>
>> def f():
>>     for i in range(10): print(i)
>
> That defaults the wrong way.

My impression exactly, so I'm -1. But the trade-off behind this is: 
complicating new code versus breaking old code (Python 3 classic).

Stefan