[Python-ideas] PEP 426, YAML in the stdlib and implementation discovery

[Andrew Barnert]

On Jun 4, 2013, at 6:14, Paul Moore <p.f.moore at gmail.com> wrote:

> Agreed, but adoption of an existing library (presumably PyYAML as it's the only mature one I'm aware of) is the only sensible option, surely? Writing something brand new for the stdlib - or worse still, writing something *partial* and brand new - can't be a sensible option. (And I'd consider a pure-python implementation "partial" in that sense, I see no reason to have a version that is deliberately slower than the competition in the stdlib).

PyYAML has two implementations: a pure Python one, and a set of bindings around libyaml. Not having a dependency on an external C library may be a good reason to have a version that's slower than the competition.

It's quite possible that the pure python implementation of PyYAML is already fast enough for the vast majority of use cases. Most people are talking about using it for config files. Even the massive, unwieldy config files used by some mail, web, and streaming servers are on the order of 10KB, not 10MB.

Since libyaml is maintained by the same author as PyYAML, it might not be a problem to just incorporate it into the python source tree. But if there are other projects using it directly, forking it like that might be a problem.

And I assume making Python require libyaml is not acceptable.

Of course we could make libyaml an optional dependency at build time, and link it statically into _yaml.so, so it isn't a runtime dependency. Or make it a runtime dependency.

Other options are to build a new C implementation based on the pure Python PyYAML or on the libyaml code, or to look for ways to optimize the pure Python code by implementing various helper pieces in C, instead of the whole thing.

But first, let's make sure the problem actually needs to be solved.