(FAT Python) Convert keyword arguments to positional?
(FAT Python: http://faster-cpython.readthedocs.org/fat_python.html)
FAT Python uses guards to check whether a global name (for example, the name for a function) has changed its value. Idea: If you know exactly which function will be called, you can also optimize based on the properties of that function.
According to Eli Bendersky's 2012 blog post[1] (which might be outdated), a function call with keyword arguments is potentially slower than one with only positional arguments.
""" If the function in question accepts no arguments (marked by the METH_NOARGS flag when the function is created) or just a single object argument (METH_0 flag), call_function doesn't go through the usual argument packing and can call the underlying function pointer directly.
...
do_call ... implements the most generic form of calling. However, there's one more optimization - if func is a PyFunction (an object used internally to represent functions defined in Python code), a separate path is taken - fast_function.
...
... PyCFunction objects that do [receive] keyword arguments [use do_call instead of fast_function]. A curious aspect of this fact is that it's somewhat more efficient to not pass keyword arguments to C functions that either accept them or are fine with just positional arguments. """
So maybe, in a function which uses FAT Python's guards, we can replace some of the keyworded-calls to global function with positional-only calls. It might be a micro-optimization, but it's one that the Python programmer doesn't have to worry about.
Concerns: 1. Is it possible to correctly determine, for a given function, which positional parameters have which names? 2. Is it possible to change a function object's named parameters some time after it's created (and inspected)?
PS: I didn't feel like this was appropriate for either of Victor's running PEP threads, and the third milestone thread is in the previous month's archives, so I thought that making a new thread would be best.
[1] http://eli.thegreenplace.net/2012/03/23/python-internals-how-callables-work
2016-01-14 11:32 GMT+01:00 Franklin? Lee leewangzhong+python@gmail.com:
(FAT Python: http://faster-cpython.readthedocs.org/fat_python.html)
FYI I moved the optimizer into a new project at GitHub to ease contributions and experiments: https://github.com/haypo/fatoptimizer
Running tests work on Python 3.4, but running optimized code required a patched Python 3.6 (http://hg.python.org/sandbox/fatpython repository which includes all patches).
FAT Python uses guards to check whether a global name (for example, the name for a function) has changed its value. Idea: If you know exactly which function will be called, you can also optimize based on the properties of that function.
You need a guard on the function. The fat module provides such guard: https://fatoptimizer.readthedocs.org/en/latest/fat.html#GuardFunc
Right now, it only watch for func.__code__. I'm not sure that it's enought. A function has many attributes which can change its behaviour if they are modified: __defaults__, __closure__, __dict__, __globals__, __kwdefaults__, __module__ (?), __name__ (?), __qualname__ (?).
According to Eli Bendersky's 2012 blog post[1] (which might be outdated), a function call with keyword arguments is potentially slower than one with only positional arguments.
Yeah, ext_do_call() has to create a temporary dictionary, while calling a function only with indexed parameters can avoid *completly* the creation of any temporary object. PyEval_EvalFrameEx() takes a stack (array of objects) which is used to pass parameters from CALL_FUNCTION, but only for pure Python functions.
So maybe, in a function which uses FAT Python's guards, we can replace some of the keyworded-calls to global function with positional-only calls. It might be a micro-optimization, but it's one that the Python programmer doesn't have to worry about.
It looks like you have a plan, and I think that you can implement the optimization without changing the Python semantics.
Concerns:
- Is it possible to correctly determine, for a given function, which
positional parameters have which names?
I think so. Just "read" the function prototype no? Such info is available in AST.
- Is it possible to change a function object's named parameters some
time after it's created (and inspected)?
What do you think?
PS: I didn't feel like this was appropriate for either of Victor's running PEP threads, and the third milestone thread is in the previous month's archives, so I thought that making a new thread would be best.
Yeah, it's better to start a separated thread, thanks.
Victor
Maybe also have it substitute in the function's default args, if default args take extra work (though it would take extra memory (new local variables) and probably doesn't give any savings).
On Jan 14, 2016 1:08 PM, "Victor Stinner" victor.stinner@gmail.com wrote:
2016-01-14 11:32 GMT+01:00 Franklin? Lee leewangzhong+python@gmail.com:
Concerns:
- Is it possible to correctly determine, for a given function, which
positional parameters have which names?
I think so. Just "read" the function prototype no? Such info is available in AST.
- Is it possible to change a function object's named parameters some
time after it's created (and inspected)?
What do you think?
I'm not too familiar (yet) with the details of the AST.
I had function wrappers in mind. In particular, I would like to permit "faked"/computed function signatures for wrappers based on what they wrap (e.g. lru_cache, partial), and I'm not sure (though I suspect) that computed signatures are compatible with immutable signatures (that is, fixed upon creation).
(Sorry for the double-mail, Victor. I will try to remember not to post from the phone.)
participants (2)
-
Franklin? Lee -
Victor Stinner