On Fri, 14 Sep 2001, Martin von Loewis wrote: (In response to Samuele Pedroni)
- if it's purpose is to offer a framework for small languages
support, there are already modules around that support that (SPARK, PLY ...), the only advantage of PEP 269 being speed wrt to the pure python solutions, because of the use of the internal CPython parser, OTOH the other solutions are more flexible...
I agree. I'd like to see (or perhaps write myself) a proposal for adding one or two of these packages to the Python core (two are probably better, since there is no one-size-fits-all parser framework, and adding two avoids the impression that there is a single "blessed" parser).
I would like to note that integration with these other systems are in the plans for my Basil project (http://wildideas.org/basil/).
Which is not in the scope of the PEP.
I just felt that a pgen integration would be better suited to the native code base rather than copying the code over to another project and building it as an extension module (which was a route being explored by Mobius Python.)
I see, but as you can see there are other issues that come into play. Jython and first the appropriateness of the interface.
It should be considered that Jython does not contain a parser similar to CPython one. Because of this jython does not offer parser module support. So implementing the PEP for Jython would require writing a Java or pure python equivalent of the CPython parser.
I am all for writing a pgen implementation in pure Python. The reason I am not going this route from the get go is to do what is easy before we do what is less easy.
If, for example, a reference implementation of a Python type system was to be adopted as standard, I would think that making the new system easy to add to Jython would be a prerequisite. Hence, we would need to develop a Jython parser that uses the grammar from CPython.
Of course Jython has a (different and not run-time configurable) parser that can be extended in case, so I don't get the point.
If the goal is to play with extensions to the Python grammar, I think this is less of an issue. Of course, anybody wanting to extend the C grammar could easily modify the Python interpreter itself.
So I think I'm -1 on this PEP, on the basis that this is code bloat (i.e. new functionality used too rarely).
As stated in the PEP, one of the primary motiviations for the proposal is to allow grammar extensions to be prototyped in Python (esp. optional static typing.) I would argue that making actual changes to CPython is much more expensive than writing a front end in Python.
But you could write that using one of the SPARK, PLY ... tools And in any case the PEP is ignoring the part about how to produce the actual code from a Python front-end. And how to add possibly necessary new bytecodes...
By adding a pgen module to Python, I feel that we are not bloating Python so much as we are exposing funtionality already built into Python.
Yes but how much is worth to expose such functionality depends on the whole picture: how you want to concretely use the exposed functionality?
It is all but clear, how you can exploit a python exposed pgen and parser in order to make as easy as possible for a casual user to experiment with a grammar extension? In an ideal scenario the user would install somefile in his python installation and start python having access to the extension. How this can work is unanswered by the PEP.
I think that a PEP that adress the whole problem would make more sense and would be more easy to evaluate.