[Python-ideas] Built-in parsing library

Stephen J. Turnbull turnbull.stephen.fw at u.tsukuba.ac.jp
Mon Apr 1 02:16:01 EDT 2019

David Mertz writes:

 > While I can imagine proposing one for inclusion in the standard
 > library, you'd have to choose one (or write a new one) and explain
 > why that one is better for everyone (or at least a better starting
 > point) than all the others are.

In principle, no, one just needs to explain why this battery fits most
of the toys encountered in practice.  That's good enough, and if
during discussion somebody shows another one is better on a lot of
fronts, sure, do that instead.  We should avoid letting the perfect be
the enemy of the good (as people keep insisting about str.cutsuffix).

Politically, sure, it's almost 100% certain that somebody will object
that there's a whole class of cases handled by the PackMule parser
that the ShavedYacc parser doesn't handle, and somebody else will
point out the opposite, so neither is acceptable.  Ignore them,
they're both wrong about "acceptable". ;-)

 > You're also have to explain why it needs to be in the standard
 > library rather than installed by 'pip install someparser'.

Again, the bar isn't so high as "needs".  There's a balance of
equities, such as people with Python installations restricted by QA or
security vetting, applications where you really don't want to spend
most of your hour allocated to teaching the feature downloading
requirements, and cases where pretty much everybody performs the task
frequently (for some value of frequently), vs. costs of maintenance
(we generally require that a core developer vouch for someone who
volunteers to take responsibility for it for 3-5 years) and effects on
complexity of learning Python (usually not great for such a module,
since the excess burden on documentation ends up being one line in the
TOC and a half-dozen in the index).

Yes, Nam should be prepared for pushback on both grounds.  Most
pressingly, without a specific package being proposed, discussion will
just go in circles indefinitely.  But a parser generator package is
something that's been lurking, waiting for an enthusiastic proponent
for a long time.  There's a lot of low-level support for it.  Maybe it
just needs a specific proposal to take off.  And maybe it won't.  He
won't know unless he tries.


P.S. Guido mentioned lib2to3.pgen2, which is in the stdlib.  But
help(pgen2) isn't very helpful, so there's at least some documentation
work to be done there.

More information about the Python-ideas mailing list