[Python-ideas] Optimizing builtins

[Guido van Rossum]

On Fri, Dec 31, 2010 at 5:52 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Guido van Rossum wrote:
>> That can't be the answer, because then the question would become "how
>> does the compiler know it can use the special opcode". This particular
>> issue (generating special opcodes for certain builtins) has actually
>> been discussed many times before. Alas, given Python's extremely
>> dynamic promises it is very hard to do it in a way that is
>> *guaranteed* not to change the semantics.
>
> Just tossing ideas out here... pardon me if they've been discussed before,
> but I read the three PEPs you mentioned later (266, 267 and 280) and they
> didn't cover any of this.
>
> I wonder whether we need to make that guarantee? Perhaps we should
> distinguish between "safe" optimizations, like constant folding which can't
> change behaviour,

(Though notice that our historic track record indicates that they are
very dangerous -- we've introduced subtle bugs several times in
"trivial" constant folding optimizations.)

> and "unsafe" optimizations which can go wrong under
> (presumably) rare circumstances. The compiler can continue to apply whatever
> safe optimizations it likes, but unsafe optimizations must be explicitly
> asked for by the user. If subtle or not subtle bugs occur, well, Python does
> allow people to shoot themselves in the foot.
>
> There's precedence for this. Both -O and -OO optimization switches
> potentially change behaviour. -O *should* be safe if code only uses asserts
> for assertions, but many people (especially beginners) use assert for input
> checking. If their code breaks under -O they have nobody to blame but
> themselves. Might we not say that -OO will optimize access to builtins, and
> if things break, the solution is not to use -OO?

Maybe. But that means it will probably rarely be used --
realistically, who uses -O or -OO? I don't ever. Even so, there would
have to be a way to turn the optimization off even under -OO for a
particular module or function or code location, or for a particular
builtin (again, open() comes to mind).

> Here's another thought... suppose (say) "builtin" became a reserved word.
> builtin.range (for example) would always refer to the built-in range, and
> could be optimized by the compiler. It wouldn't do much for the general case
> of wanting to optimize non-built-in globals, but this could be optimized
> safely:
>
> def f():
>    for i in builtin.range(10): builtin.print(i)
>
> while this would keep the current semantics:
>
> def f():
>    for i in range(10): print(i)

That defaults the wrong way. You want the optimization to work (if the
compiler does not see explicit reasons to avoid it) except in rare
cases where the compiler cannot know that you're dynamically modifying
the ennvironment (globals or builtins). Also I would very much worry
that people would start putting this in everywhere out of a mistaken
defensive attitude. (Like what has happened to certain
micro-optimization patterns, which are being overused, making code
less readable.)

-- 
--Guido van Rossum (python.org/~guido)