[Python-Dev] Release of astoptimizer 0.3

[Serhiy Storchaka]

On 11.09.12 13:41, Victor Stinner wrote:
> Here are some progress on my astoptimizer project. If you are interested by
> the optimizer, run it on your own project or help me to implement more
> optimizations.
>
> http://pypi.python.org/pypi/astoptimizer
> https://bitbucket.org/haypo/astoptimizer

It's a very interesting work.

> * Call builtin functions if arguments are constants. Examples:
>
>    - len("abc") => 3
>    - ord("A") => 65

Does this mean transformation print("A") => None and output at compile time?

> * Loop: replace range() with xrange() on Python 2,

range(0, 10**10, 10**9)

> * Evaluate unary and binary operators, subscript and comparaison if all
>    arguments are constants. Examples:
>
>    - 1 + 2 * 3 => 7

Does this mean 1/0 raises exception at compile time?

> * Remove dead code. Examples:
>
>    - def f(): return 1; return 2 => def f(): return 1
>    - if DEBUG: print("debug") => pass with DEBUG declared as False
>    - while 0: ... => pass

As Nick said, it could change the semantics, if there were local 
variable assignment or yield in removed code.

> Unsafe optimizations are disabled by default. Optimizations can be enabled
> using a Config class with "features" like "builtin_funcs" (builtin functions
> like len()) or "pythonbin" (optimized code will be execute by the same
> Python binary executable).

It would be good if the optimizer checked if the built-in names 
overridden in this module or function and disabled this dangerous 
optimization in such case.

> I plan to implement other optimizations like unrolling loop or convert
> a loop to a list comprehension, see the TODO file.
>
> Don't hesitate to propose more optimizations if you have some ideas ;-)

set([1, 2, 3]) => {1, 2, 3}
set([x for ...]) => {x for ...}
dict([(k, v) for ...]) => {k: v for ...}
dict((k, v) for ...) => {k: v for ...}
''.join([s for ...]) => ''.join(s for ...)
a.extend([s for ...]) => a.extend(s for ...)
(f(x) for x in a) => map(f, a)
(x.y for x in a) => map(operator.attrgetter('y'), a)
(x[0] for x in a) => map(operator.itemgetter(0), a)
(2 * x for x in a) => map((2).__mul__, a)
(x in b for x in a) => map(b.__contains__, a)
map(lambda x: x.strip(), a) => (x.strip() for x in a)
x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}
x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}
a = []; for ...: a.append(x) => a = [x for ...]
a = (); for ...: a.add(x) => a = {x for ...}
a = {}; for ...: a[k] = v => a = {k: v for ...}
for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)
x = x + 1 => x += 1
x = x + ' ' => x += ' '
x = x + [y] => x.append(y)
x = x + [y, z] => x.extend([y, z])
'x=%s' % repr(x) => 'x=%a' % (x,)
'x=%s' % x + s => 'x=%s%s' % (x, s)
x = x + ', [%s]' % y => x = '%s, [%s]' % (x, y)
range(0, x) => range(x)
for i in range(len(a)): x = a[i] ... => for i, x in enumerate(a): ...
i = 0; for x in a: i += 1 ... => for i, x in enumerate(a, 1): ...
i = 1; for x in a: ... i += 1 => for i, x in enumerate(a, 1): ...
s = 0; for ...: if ...: s += 1 => s = sum(1 for ... if ...)
while True: s = f.readline(); if not s: break; ... => for s in f: ...
def f(x): ... len() ... => def f(x, __len = len): ... __len() ...

Not all such transformations are always safe (and I know code in stdlib 
where they are not).