[Python-Dev] Second milestone of FAT Python

[Maciej Fijalkowski]

Uh, sorry, misread your full mail, scratch that

On Wed, Nov 4, 2015 at 9:07 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> How do you check that someone did not e.g. bind something different to "len"?
>
> On Wed, Nov 4, 2015 at 8:50 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
>> Hi,
>>
>> I'm writing a new "FAT Python" project to try to implement optimizations in
>> CPython (inlining, constant folding, move invariants out of loops, etc.)
>> using a "static" optimizer (not a JIT). For the background, see the thread
>> on python-ideas:
>> https://mail.python.org/pipermail/python-ideas/2015-October/036908.html
>>
>> See also the documentation:
>> https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
>> https://hg.python.org/sandbox/fatpython/file/tip/ASTOPTIMIZER.rst
>>
>> I implemented the most basic optimization to test my code: replace calls to
>> builtin functions (with constant arguments) with the result. For example,
>> len("abc") is replaced with 3. I reached the second milestone: it's now
>> possible to run the full Python test suite with these optimizations enabled.
>> It confirms that the optimizations don't break the Python semantic.
>>
>> Example:
>> ---
>>>>> def func():
>> ...     return len("abc")
>> ...
>>>>> import dis
>>>>> dis.dis(func)
>>   2           0 LOAD_GLOBAL              0 (len)
>>               3 LOAD_CONST               1 ('abc')
>>               6 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
>>               9 RETURN_VALUE
>>
>>>>> len(func.get_specialized())
>> 1
>>>>> specialized=func.get_specialized()[0]
>>>>> dis.dis(specialized['code'])
>>   2           0 LOAD_CONST               1 (3)
>>               3 RETURN_VALUE
>>>>> len(specialized['guards'])
>> 2
>>
>>>>> func()
>> 3
>>
>>>>> len=lambda obj: "mock"
>>>>> func()
>> 'mock'
>>>>> func.get_specialized()
>> []
>> ---
>>
>> The function func() has specialized bytecode which returns directly 3
>> instead of calling len("abc"). The specialized bytecode has two guards
>> dictionary keys: builtins.__dict__['len'] and globals()['len']. If one of
>> these keys is modified, the specialized bytecode is simply removed (when the
>> function is called) and the original bytecode is executed.
>>
>>
>> You cannot expect any speedup at this milestone, it's just to validate the
>> implementation. You can only get speedup if you implement *manually*
>> optimizations. See for example posixpath.isabs() which inlines manually the
>> call to the _get_sep() function. More optimizations will be implemented in
>> the third milestone. I don't know yet if I will be able to implement
>> constant folding, function inlining and/or moving invariants out of loops.
>>
>>
>> Download, compile and test FAT Python with:
>>
>>     hg clone http://hg.python.org/sandbox/fatpython
>>     ./configure && make && ./python -m test test_astoptimizer test_fat
>>
>>
>> Currently, only 24 functions are specialized in the standard library.
>> Calling a builtin function with constant arguments in not common (it was
>> expected, it's only the first step for my optimizer). But 161 functions are
>> specialized in tests.
>>
>>
>> To be honest, I had to modify some tests to make them pass in FAT mode. But
>> most changes are related to the .pyc filename, or to the exact size in bytes
>> of dictionary objects.
>>
>> FAT Python is still experimental. Currently, the main bug is that the AST
>> optimizer can optimize a call to a function which is not the expected
>> builtin function. I already started to implement code to understand
>> namespaces (detect global and local variables), but it's not enough yet to
>> detect when a builtin is overriden. See TODO.rst for known bugs and
>> limitations.
>>
>> Victor
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>>