[Distutils] Builders vs Installers

[Donald Stufft]

On Mar 27, 2013, at 6:41 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Wed, Mar 27, 2013 at 1:51 PM, Donald Stufft <donald at stufft.io> wrote:
>> I don't think the packaging formats should dictate the development flow at all. .IN files and such all dictate how that should be.
> 
> You can't *not* dictate the flow.  If you don't have something to
> generate the file with, then you're dictating that developers must
> ship an sdist and users must manually run build tools to make wheels.
> 
> (Or, more likely, you're creating the motivation for somebody to
> create a meta-meta-build system that solves the problem, probably by
> having a bunch of plugins to detect what build system a raw source
> directory is using.)
> 
> 
>> To me this is an installer issue not a packaging issue and it's best solved in the installers. Obviously there is some benefit to a "standard" way for installers to treat these but I don't think it should be defined in terms of the packaging formats.
> 
> It definitely doesn't have to be.  distutils2's setup.cfg isn't
> actually a bad human-readable format, but it's not a *packaging*
> format.
> 
> In any case, the only thing an installer needs is a way to get the
> setup-requires-dist, or the portion of it that pertains to identifying
> the metadata hooks. The rest could be handled with entry points
> registered for configuration file names.  For example, Bento could
> expose an entry point like:
> 
>  [mebs.metadata.generators]
>  bento.info = some.module.in.bento:hookfunction
> 
> And then an installer builds a list of these hooks that are in the
> setup-requires-dists, and runs them based on the filenames found in
> the project directory.  All done.
> 
> Basically, the only thing we need is a way to avoid having to either:
> 
> 1. Make every user install Bento/etc. before trying to install a
> package from source that uses it, or
> 2. Embed a registry of every possible build configuration file name
> into every installer.
> 
> And this can be done in any of several ways:
> 
> * Have a standardized naming pattern like pybuild.*, and make the .*
> part indicate the build tool
> 
> * Have a standardized single name (like pybuild.cfg), and encode the
> build tool in a string that can be read regardless of file format, so
> it can be embedded in whatever format the build tool itself uses
> 
> * Have a separate file that *only* lists the build tool or
> setup-requires-dists (and maybe can be extended to contain other
> information for use with a stdlib-supplied build tool)
> 
> I personally lean towards the last one, especially if it reuses
> setup.cfg, because setup.cfg already exists and is fairly
> standardized.  There are even tools that work today to let you do a
> metadata-free setup.py and specify everything needed in setup.cfg,
> with environment markers and everything.
> 
> Heck, IIUC, there's at least one library you can use today with
> *setuptools* to do that -- it doesn't need distutils2 or any of that,
> it just translates setup.cfg to setup.py arguments.
> 
> But an even more important reason to standardize is that there should
> be one, and preferably only one, obvious way to do it.  AFAIK, the
> distutils2 effort didn't fail because of setup.cfg -- heck, setup.cfg
> was the main *benefit* I saw in the distutils2 work, everything else
> about it AFAIK was just setuptools warmed over -- it failed because of
> trying to boil the ocean and *implement* everything, rather than just
> standardizing on interfaces.  A minimal boilerplate setup.cfg could be
> something like
> 
> [build]
> builder = bento >1.6
> 
> And leave it at that.  Disadvantage is that it's a dumb boilerplate
> file for tools that don't use setup.cfg for their configuration --
> i.e., it's a minor disadvantage to users of those tools.  However, if
> your preferred build tool generates the file for you, it's no big
> deal, as long as the generated file doesn't change all the time and
> you check it into source control.
> 
> Such a usage pattern is teachable and provides what's needed, without
> dictating anything about the development workflow, other than that you
> need to tell installers how to make an sdist if you want people to
> install stuff you shipped without an sdist or a wheel, or if you want
> to use any generic build-running tools that need to know your build
> hook(s).

repurposing a single line of setup.cfg for this usecase wouldn't be the worst thing in the world. I don't like setup.cfg and I especially don't like it as the format to exchange the _actual_ metadata, but as a configuration format (configure which build system to use) it's ok. I still think I prefer a setup.py develop or develop.py to invoke the build system for development builds, but atm the difference between something like echo "[build]\nbuilder = bento > 1.6" > setup.cfg and develop.py is not a hill I care to die on. Maybe Nick has different ides for how VCS/install from an unpacked directory (E.g. explicitly not a package) should look I don't know.

> 
> 
>> development checkouts should be discouraged unless you're actively working on that project.
> 
> Perhaps Jim can chime in on this point, but when you work with a whole
> bunch of people developing a whole bunch of libraries making up a
> larger project (e.g. Zope), it doesn't seem very sensible to expect
> that everybody manually check out and manage all the dependencies
> they're using.  Maybe you could mitigate that somewhat with some sort
> automated continuous build/release system, but that's not always a
> practical option.


Sorry my statement was a bit unclear, those people would all fall under actively working on that project (Zope in this case). I mean installs from VCS's should be discouraged for end users.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130327/1a36e72a/attachment.pgp>