On Thu, Oct 18, 2012 at 5:21 PM, PJ Eby wrote:

On Wed, Oct 17, 2012 at 11:25 PM, Daniel Holth wrote:

...

An installer downloads an sdist. For each installed build plugin, .recognize(dir) is called. The first plugin to return True is used.

Why not just have a standard bit of metadata in an sdist that tells the installer what builder(s) needs to be installed? i.e., the equivalent of setup_requires on steroids.

(Granted, for backward compatibility, it's not hard to recognize setuptools, distribute, or distutils-based sdists.)

* not an actual project * In practice you would have a tiny standard bit of metadata "setup_requires on steroids" and you would almost always follow it, but the ordered-list-of-builders design lets you override the sdist's decision.

On Wed, Feb 6, 2013 at 3:42 PM, Marcus Smith wrote:

...

What if pip did not depend on setuptools or distutils and the stdlib did not include distutils or any other build system? Instead, the installer can only install binary packages, and build systems do not install but only build binary packages.

A very simple meta-build system "mebs" is used to recognize sdists and build binary packages. Build systems provide plugins having three methods,

.recognize() .metadata() .build()

An installer downloads an sdist. For each installed build plugin, .recognize(dir) is called. The first plugin to return True is used.

The plugin's .metadata(dir) is called, returning Metadata 1.3+ format metadata in a simple email.parser.Parser() multi-dict interface. Used to fetch any build/install requirements.

.build(dir) is called to create the binary package.

The installer installs the binary package.

I didn't really catch this the first time around, but this is interesting. btw, any other threads related to this?

Not really (and I had missed this post as well). As I believe you've seen, I have some half developed thoughts along these lines at http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packagin..., but the whole concept is fairly nebulous. The basic concept of having a pluggable toolchain along the lines of: source control -> Archiver -> sdist sdist -> Builder -> wheel wheel -> Installer -> deployed software is a sound one, though. (and one I've discussed with a few people off-list) The choice of Archiver and Builder can then be left up to the project developer, while the choice of Installer will be up to the end user. The task of the PEP process then becomes up matter of coming up with appropriate metadata standards that allow the info that is needed for the various steps to be passed along the chain. PEP 426, in combination with wheel, focuses primarily on the needs of the Installer step in the chain, with some support for the Builder step (mostly through Setup-Requires-Dist and Extension field) The principle flaw in distutils (which is inherited by setuptools and distribute) is that it is a combination Archiver+Builder+Installer, without well defined formats for exchanging metadata between the steps. That means the developer's choice of build system then impacts the end user's choice of installation tool, which is not a good place to be. By moving to a pluggable meta-build-system, the stdlib can ship a simple Archiver, a simple Builder and a simple Installer, leaving people to choose Archivers and Builders provided by other tools (such as bento) when they outgrow the simple ones. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia