On 12 October 2015 at 08:23, Nathaniel Smith <njs@pobox.com> wrote:
I mean the distribution name - there's no way to guarantee that building foo-1.0.zip won't spit out bar-7.4.whl, where by "no way" I mean "it's literally undecideable". I mean, if someone actually did this it would be super weird and we would all shun them, but our code and specs still need to be prepared for the possibility.
But the whole point of a spec is to *make* that guarantee. Pip doesn't expect to ever work with random code that can do literally anything, that's silly. Pip works with code that conforms to a spec. At the moment, the spec is "it does what setuptools/distutils does", which is a lousy spec because (a) nobody else can implement the spec without reading the huge mess that is distutils/setuptools, (b) it's too broad (we don't *really* need every corner case that distutils covers) and (c) distutils/setuptools don't satisfy the needs of a number of communities, notably the scientific community. So we want to define a more usable spec for the future. Pip will always have to deal with backward compatibility, and all of the hacks that implies, but we can at least say "if a project declares that it follows the new spec, we can rely on certain things". One of those things we can rely on is that building foo-1.0.zip won't spit out bar-7.4.whl - that would be part of the contract of that spec. The debate is over on the one hand, we want to be able to rely on when writing packaging tools, vs what flexibility we want to have when writing build tools. But it's not over ignoring the other use cases, it's about agreeing a *contract* that satisfies the needs of *both* packaging tools and build tools. At the moment, the dial is 100% over to the packaging side - builders have zero flexibility, the rule is "use distutils and setuptools". That doesn't feel like zero flexibility, because distutils/setuptools let you do a lot of things. But when you do finally hit the limits you're stopped cold. It also is a bad choice for packaging tools, because distutils and setuptools are a dreadful API for automation. So nobody wins. Two suggestions that have been around for a long while that give more flexibility to build tools while at the same time giving packaging tools a better interface to work with are: 1. Declarative setup - you have to put your metadata in file X in format Y, and it's static. But when you do, your build tool can do whatever it wants as long as it spits out binaries that confirm to your declared metadata. The downside to this is that it's totally static, and developers don't like that ("what if I want to generate my version number from my VCS tags?") 2. A fixed API - we document a set of commands that packaging tools can use and build tools have to provide. This is where we talk about documenting the distutils commands that pip relies on ("setup.py egg-info", "setup.py install", "setup.py develop"...) This one usually falls down because nobody likes the idea of writing a "dummy" setup.py that translates the interface, and because no-one has done anything more with this idea than suggest documenting the setuptools commands that pip *currently* uses, even though these are probably a lousy API design. It also does very little for packaging tools *other* than pip (for example, it does nothing for PyPI, which cannot accept an API that requires running user-supplied code). The key drivers here have been about defining something that packaging tools can use effectively, and build tools can cope with being required to provide. While not constraining *either* type of tool beyond the minimum needed for the contract. Paul