[Python-Dev] Second milestone of FAT Python

[Sven R. Kunze]

Hi Victor,

great to hear. I think everybody here appreciates your efforts.

Do you think there will be any change of merging this back into CPython?

Best,
Sven

On 04.11.2015 09:50, Victor Stinner wrote:
> Hi,
>
> I'm writing a new "FAT Python" project to try to implement 
> optimizations in CPython (inlining, constant folding, move invariants 
> out of loops, etc.) using a "static" optimizer (not a JIT). For the 
> background, see the thread on python-ideas:
> https://mail.python.org/pipermail/python-ideas/2015-October/036908.html
>
> See also the documentation:
> https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
> https://hg.python.org/sandbox/fatpython/file/tip/ASTOPTIMIZER.rst
>
> I implemented the most basic optimization to test my code: replace 
> calls to builtin functions (with constant arguments) with the result. 
> For example, len("abc") is replaced with 3. I reached the second 
> milestone: it's now possible to run the full Python test suite with 
> these optimizations enabled. It confirms that the optimizations don't 
> break the Python semantic.
>
> Example:
> ---
> >>> def func():
> ...     return len("abc")
> ...
> >>> import dis
> >>> dis.dis(func)
>   2           0 LOAD_GLOBAL              0 (len)
>               3 LOAD_CONST               1 ('abc')
>               6 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
>               9 RETURN_VALUE
>
> >>> len(func.get_specialized())
> 1
> >>> specialized=func.get_specialized()[0]
> >>> dis.dis(specialized['code'])
>   2           0 LOAD_CONST               1 (3)
>               3 RETURN_VALUE
> >>> len(specialized['guards'])
> 2
>
> >>> func()
> 3
>
> >>> len=lambda obj: "mock"
> >>> func()
> 'mock'
> >>> func.get_specialized()
> []
> ---
>
> The function func() has specialized bytecode which returns directly 3 
> instead of calling len("abc"). The specialized bytecode has two guards 
> dictionary keys: builtins.__dict__['len'] and globals()['len']. If one 
> of these keys is modified, the specialized bytecode is simply removed 
> (when the function is called) and the original bytecode is executed.
>
>
> You cannot expect any speedup at this milestone, it's just to validate 
> the implementation. You can only get speedup if you implement 
> *manually* optimizations. See for example posixpath.isabs() which 
> inlines manually the call to the _get_sep() function. More 
> optimizations will be implemented in the third milestone. I don't know 
> yet if I will be able to implement constant folding, function inlining 
> and/or moving invariants out of loops.
>
>
> Download, compile and test FAT Python with:
>
>     hg clone http://hg.python.org/sandbox/fatpython
>     ./configure && make && ./python -m test test_astoptimizer test_fat
>
>
> Currently, only 24 functions are specialized in the standard library. 
> Calling a builtin function with constant arguments in not common (it 
> was expected, it's only the first step for my optimizer). But 161 
> functions are specialized in tests.
>
>
> To be honest, I had to modify some tests to make them pass in FAT 
> mode. But most changes are related to the .pyc filename, or to the 
> exact size in bytes of dictionary objects.
>
> FAT Python is still experimental. Currently, the main bug is that the 
> AST optimizer can optimize a call to a function which is not the 
> expected builtin function. I already started to implement code to 
> understand namespaces (detect global and local variables), but it's 
> not enough yet to detect when a builtin is overriden. See TODO.rst for 
> known bugs and limitations.
>
> Victor
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/srkunze%40mail.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20151104/3210f0ec/attachment.html>