On Mon, Mar 25, 2013 at 5:08 PM, Paul Moore <p.f.moore@gmail.com> wrote:
There's a longer-term issue that occurred to me when thinking about pip's role as a "builder" or an "installer" (to use Nick's terminology).
As I understand Nick's vision for the future, installers (like pip) will locate built wheels and download and install them, and builders (like distutils and bento) will be responsible for building wheels. But there's an intermediate role which shouldn't get forgotten in the transition - the role that pip currently handles with the "pip wheel" command. This is where I specify a list of distributions, and pip locates sdists, downloads them, checks dependencies, and ultimately builds all of the wheels. I'm not sure whether the current idea of builders includes this "locate, download and resolve dependencies" function (distutils and bento certainly don't have that capability).
Yes, and to make things even more interesting, consider the cases where there are build-time dependencies. ;-) I would guess that installing from sdists (and revision control) is probably here to stay, along with the inherent coupling between "build" and "fetch" functions. Right now, the "build" side of setuptools fetches build-time dependencies, but in the New World, ISTM that a top-level build tool would just be something that reads the package metadata, finds build-time dependencies, and then runs some entry points to ask for a wheel to be spit out (or to get back a data structure describing the wheel, anyway). This part could be standardized and indeed could be just something that pip does. Another piece that hasn't seemed well-specified so far is what Nick called "archiving" - creating the sdist. Or perhaps more precisely, generating an sdist PKG-INFO and putting the stuff together. IMO, a good install tool needs to be able to run the archiving and building steps as well as installing directly from wheels. However, although metadata 2.0 provides us with a good basis for running build and install steps, there really isn't anything yet in the way of a standard for sdist generation. Of course, we have "setup.py sdist", and the previous work on setup.cfg by the distutil2 team. We could either build on setup.cfg, or perhaps start over with a simple spec to say how the package is to be built: i.e., just a simple set of entry points saying what archiver, builder, etc. are used. Such a file wouldn't change much over the life of the package, and would avoid the need for all the dynamic hooks provided by distutils2's setup.cfg. In the degenerate case, I suppose, it could just be "pyarchiver.cfg" and contain a couple lines saying what tool is used to generate PKG-INFO. On the other hand, we could draw the line at saying, pip only ever installs from sdists, no source checkouts or tarballs. I'm not sure that's a reasonable limitation, though. On the *other* other hand, Perhaps it would be best to just use the setup.cfg work, updated to handle the full metadata 2.0 spec. As I recall, the setup.cfg format handled a ridiculously large number of use cases in a very static format, and IIRC still included the possibility for dynamic hooks to affect the metadata-generation and archive content selection processes, let alone the build and other stages. But then, is that biasing against e.g. bento.info? Argh. Packaging is hard, let's go shopping. ;-) On balance, I think I lean towards just having a simple way to specify your chosen archiver, so that installing from source checkouts and dumps is possible. I just find it annoying that you have to have *two* files in your checkout, one to say what tool you're using, and another one to configure it. (What'd be nice is if you could just somehow detect files like bento.info and setup.cfg and thereby detect what archiver to use. But that would have limited extensibility unless there was a standard naming convention for the files, or a standardized format for at least the first line in the file or something like that, so you could identify the needed tool.)