[Distutils] mebs, the meta-build system

[Nick Coghlan]

On Wed, Feb 6, 2013 at 3:42 PM, Marcus Smith <qwcode at gmail.com> wrote:
>
>> What if pip did not depend on setuptools or distutils and the stdlib
>> did not include distutils or any other build system? Instead, the
>> installer can only install binary packages, and build systems do not
>> install but only build binary packages.
>>
>> A very simple meta-build system "mebs" is used to recognize sdists and
>> build binary packages. Build systems provide plugins having three
>> methods,
>>
>> .recognize()
>> .metadata()
>> .build()
>>
>> An installer downloads an sdist. For each installed build plugin,
>> .recognize(dir) is called. The first plugin to return True is used.
>>
>> The plugin's .metadata(dir) is called, returning Metadata 1.3+ format
>> metadata in a simple email.parser.Parser() multi-dict interface. Used
>> to fetch any build/install requirements.
>>
>> .build(dir) is called to create the binary package.
>>
>> The installer installs the binary package.
>>
>
> I didn't really catch this the first time around, but this is interesting.
> btw, any other threads related to this?

Not really (and I had missed this post as well). As I believe you've
seen, I have some half developed thoughts along these lines at
http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html,
but the whole concept is fairly nebulous.

The basic concept of having a pluggable toolchain along the lines of:

source control -> Archiver -> sdist
sdist -> Builder -> wheel
wheel -> Installer -> deployed software

is a sound one, though. (and one I've discussed with a few people off-list)

The choice of Archiver and Builder can then be left up to the project
developer, while the choice of Installer will be up to the end user.

The task of the PEP process then becomes up matter of coming up with
appropriate metadata standards that allow the info that is needed for
the various steps to be passed along the chain.

PEP 426, in combination with wheel, focuses primarily on the needs of
the Installer step in the chain, with some support for the Builder
step (mostly through Setup-Requires-Dist and Extension field)

The principle flaw in distutils (which is inherited by setuptools and
distribute) is that it is a combination Archiver+Builder+Installer,
without well defined formats for exchanging metadata between the
steps. That means the developer's choice of build system then impacts
the end user's choice of installation tool, which is not a good place
to be.

By moving to a pluggable meta-build-system, the stdlib can ship a
simple Archiver, a simple Builder and a simple Installer, leaving
people to choose Archivers and Builders provided by other tools (such
as bento) when they outgrow the simple ones.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia