[Python-ideas] Hooking between lexer and parser

Nick Coghlan ncoghlan at gmail.com
Sat Jun 6 07:04:00 CEST 2015

On 6 June 2015 at 15:00, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 6 June 2015 at 12:21, Neil Girdhar <mistersheik at gmail.com> wrote:
>> I'm curious what other people will contribute to this discussion as I think
>> having no great parsing library is a huge hole in Python.  Having one would
>> definitely allow me to write better utilities using Python.
> The design of *Python's* grammar is deliberately restricted to being
> parsable with an LL(1) parser. There are a great many static analysis
> and syntax highlighting tools that are able to take advantage of that
> simplicity because they only care about the syntax, not the full
> semantics.
> Anyone actually doing their *own* parsing of something else *in*
> Python, would be better advised to reach for PLY
> (https://pypi.python.org/pypi/ply ). PLY is the parser underlying
> https://pypi.python.org/pypi/pycparser, and hence the highly regarded
> CFFI library, https://pypi.python.org/pypi/cffi

For the later stages of the pipeline (i.e. AST -> code generation),
CPython now uses Eli Bendersky's asdl_parser:

More background on that:


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list