I'm willing to take this conversation offline as it seems this thread has
cooled down quite a bit.
I would still like to hear more, though, about how adding this as a
facility in the language improves over the current, external
implementations of Python code optimizers. Python already has tools for
reading in source files, parsing them into AST, modifying that AST, and
writing the final bytecode to files as part of the standard library. I
don't see anything in PEP0511 that improves upon that.
Out of curiosity, do you consider this PEP as adding something to Python
that didn't previously exist or do you consider this PEP to more aligned
with PEP0249 (DB2API) and PEP0484 (Type Hints) which are primarily designed
to marshal the community in a common direction? I understand that you have
other PEPs in flight that are designed to make certain optimizations easier
(or possible). Looking at this PEP in isolation, however, leaves me wanting
more explanation as to its value.
You mention the need for monkey-patching or hooking into the import process
as a part of the rational. The PyCC project, while it may not be the best
example for optimizer design, does not need to patch or hook into any thing
to function. Instead, it acts as an alternative bytecode compiler that
drops .pyc just like the standard compiler would. Other than the trade-off
of using a 3rd party library versus adding a -o flag, what significant
advantage does a sys.add_optimizer() call provide?
Again, I'm very much behind your motivation and hope you are incredibly
successful in making Python a faster place to live. I'm only trying to get
in your head and see what you see.
On Wed, Jan 27, 2016 at 10:45 AM Victor Stinner
Hi,
2016-01-16 17:56 GMT+01:00 Kevin Conway
: I'm a big fan of your motivation to build an optimizer for cPython code. What I'm struggling with is understanding why this requires a PEP and language modification. There are already several projects that manipulate the AST for performance gains such as [1] or even my own ham fisted attempt [2].
Oh cool, I didn't know PyCC [2]! I added it to the PEP 511 in the AST optimizers section of Prior Art.
I wrote astoptimizer [1] and this project uses monkey-patching of the compile() function, I mentioned this monkey-patching hack in the rationale of the PEP: https://www.python.org/dev/peps/pep-0511/#rationale
I would like to avoid monkey-patching because it causes various issues.
The PEP 511 also makes transformations more visible: transformers are explicitly registered in sys.set_code_transformers() and the .pyc filename is modified when the code is transformed.
It also adds a new feature: it becomes possible to run transformed code without having to register the tranformer at runtime. This is made possible with the addition of the -o command line option.
Victor