[Python-Dev] Second milestone of FAT Python

[Maciej Fijalkowski]

How do you check that someone did not e.g. bind something different to "len"?

On Wed, Nov 4, 2015 at 8:50 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> I'm writing a new "FAT Python" project to try to implement optimizations in
> CPython (inlining, constant folding, move invariants out of loops, etc.)
> using a "static" optimizer (not a JIT). For the background, see the thread
> on python-ideas:
> https://mail.python.org/pipermail/python-ideas/2015-October/036908.html
>
> See also the documentation:
> https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
> https://hg.python.org/sandbox/fatpython/file/tip/ASTOPTIMIZER.rst
>
> I implemented the most basic optimization to test my code: replace calls to
> builtin functions (with constant arguments) with the result. For example,
> len("abc") is replaced with 3. I reached the second milestone: it's now
> possible to run the full Python test suite with these optimizations enabled.
> It confirms that the optimizations don't break the Python semantic.
>
> Example:
> ---
>>>> def func():
> ...     return len("abc")
> ...
>>>> import dis
>>>> dis.dis(func)
>   2           0 LOAD_GLOBAL              0 (len)
>               3 LOAD_CONST               1 ('abc')
>               6 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
>               9 RETURN_VALUE
>
>>>> len(func.get_specialized())
> 1
>>>> specialized=func.get_specialized()[0]
>>>> dis.dis(specialized['code'])
>   2           0 LOAD_CONST               1 (3)
>               3 RETURN_VALUE
>>>> len(specialized['guards'])
> 2
>
>>>> func()
> 3
>
>>>> len=lambda obj: "mock"
>>>> func()
> 'mock'
>>>> func.get_specialized()
> []
> ---
>
> The function func() has specialized bytecode which returns directly 3
> instead of calling len("abc"). The specialized bytecode has two guards
> dictionary keys: builtins.__dict__['len'] and globals()['len']. If one of
> these keys is modified, the specialized bytecode is simply removed (when the
> function is called) and the original bytecode is executed.
>
>
> You cannot expect any speedup at this milestone, it's just to validate the
> implementation. You can only get speedup if you implement *manually*
> optimizations. See for example posixpath.isabs() which inlines manually the
> call to the _get_sep() function. More optimizations will be implemented in
> the third milestone. I don't know yet if I will be able to implement
> constant folding, function inlining and/or moving invariants out of loops.
>
>
> Download, compile and test FAT Python with:
>
>     hg clone http://hg.python.org/sandbox/fatpython
>     ./configure && make && ./python -m test test_astoptimizer test_fat
>
>
> Currently, only 24 functions are specialized in the standard library.
> Calling a builtin function with constant arguments in not common (it was
> expected, it's only the first step for my optimizer). But 161 functions are
> specialized in tests.
>
>
> To be honest, I had to modify some tests to make them pass in FAT mode. But
> most changes are related to the .pyc filename, or to the exact size in bytes
> of dictionary objects.
>
> FAT Python is still experimental. Currently, the main bug is that the AST
> optimizer can optimize a call to a function which is not the expected
> builtin function. I already started to implement code to understand
> namespaces (detect global and local variables), but it's not enough yet to
> detect when a builtin is overriden. See TODO.rst for known bugs and
> limitations.
>
> Victor
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>