[Distutils] distlib and wheel metadata

[Donald Stufft]

> On Feb 14, 2017, at 1:15 PM, Nathaniel Smith <njs at pobox.com> wrote:
> 
> It's too bad that the JSON thing didn't work out, but I think we're
> better off working on better specifying the one source of truth
> everything already uses (METADATA) instead of bringing in *new*
> partially-incompatible-and-poorly-specified formats.
> 



TBH I don’t think we’re going to stick with METADATA forever and it’s likely we, at some point, get to a JSON representation for this information but that is not today. We have far more pressing issues to deal with besides whether things are in one format or another.

Yes, we still have a fair amount of behavior that is defined as “whatever setuptools/distutils does”, but we’re slowly trying to break away from that.

WRT to “standard implementations” versus “standards”, the idea of a “standard implementation” being the source of truth and no longer needing to do all the work to define standards is a nice idea, but I think it is an idea that is never actually going to work out as well as real standardization. There is *always* going to be a need for tools that aren’t the blessed tools to interact with these items. Even if you can authoritatively say that this one Python implementation is the only implementation that any Python program will ever need, there is still the problem that people need to consume this information in languages that aren’t Python. Another problem there is it becomes incredibly difficult to know what is something that is supported as an actual feature and what is something that just sort of works because of that way that something was implemented.

My goal with the packaging library is to more or less strictly implement accepted PEPs (and while I will make in progress PRs for PEPs that are currently being worked on, I won’t actually land a PR until the PEP is accepted). The only other real code there is extra utilities that make the realities of working with the specified PEPs easier (for example, we have a Version object which implements PEP 440 versions, but we also have a LegacyVersion object that implements what setuptools used to do).

This not only gives us the benefit of a single implementation for people who just want to use that single blessed implementation, but it gives us the benefit of standards. This has already been useful in the packaging library where an implementation defect caused versions to get parsed slightly wrong, and we had the extensively documented PEP 440 to declare what the expected behavior was.

I do not think the problem is "We've gotten so used to how pip and setuptools work, and because they are "good enough", there is a real failure of imagination to see how things might be done better.”. The hard work of doing this isn’t in writing an implementation that achieves it for 80% of projects, it’s for doing it in a way that achieves it for 95% of projects. Managing backwards compatibility is probably the single most important thing we can do here. There are almost 800,000 files on PyPI that someone can download and install, telling all of them they need to switch to some new system or things are going to break for them is simply not tenable. That being said, I don’t think there is anything stopping us from getting to a better point besides time and effort.

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170214/fdb3eef4/attachment.html>