[Distutils] command hooks...

[Erik Bray]

On Wed, May 30, 2012 at 12:19 PM, PJ Eby <pje at telecommunity.com> wrote:
>
>
> On Wed, May 30, 2012 at 11:54 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
>>
>> On Tue, May 29, 2012 at 7:41 PM, PJ Eby <pje at telecommunity.com> wrote:
>> >> > I might be confused; I haven't been following the goings-on of late
>> >> > with
>> >> > distutils2.  At one point, I thought the plan was not to bless or
>> >> > include dependency-managing installers with the stdlib, or something
>> >> > like that.  i.e., I thought the plan wasn't to support or bless
>> >> > full-service tools like buildout, easy_install, or pip, or anything
>> >> > comparable to them.
>> >>
>> >> Right, yeah, the plans in this area were fluid for awhile, but the
>> >> eventual conclusion was that the stdlib should have a command-line
>> >> utility capable of installing packages with dependencies. That exists
>> >> in
>> >> default branch now; it's called pysetup. It doesn't have nearly all the
>> >> features of easy_install, buildout, or pip, but it can install packages
>> >> from an index with deps.
>> >>
>> >
>> > In any case, it still doesn't change the part where it's a good idea to
>> > ship
>> > a static setup.cfg, with hooks only needing to run on the sdist-building
>> > machine, unless they are actually part of the build process.  There are
>> > use
>> > cases for calculated data to be in the initial setup.cfg, where the
>> > calculation machinery doesn't need to be on the target (like generating
>> > the
>> > file list or version from revision control info).  So, a setup_requires
>> > (or
>> > maybe better named "build_requires") would still be helpful, but
>> > probably
>> > shouldn't be used for setup.cfg stability.
>>
>> That's not a bad idea for certain kinds of metadata--version/vcs info
>> for example.  I like the idea of including a generated "static"
>> setup.cfg in a source dist as a solution to that kind of problem.  But
>> that doesn't eliminate the need for setup_hooks (or even more
>> complicated objects like custom commands) in an sdist.
>>
>> For example, the majority of projects I work on require Numpy to build
>> one or two extension modules.  They require hooks to check that the
>> numpy package is importable, and then to use numpy's API to get the
>> paths to the numpy headers and and them to the include_dirs for each
>> extension module that requires them.  That's not the only one
>> though--one could have a whole suite of setup_hooks common to a bunch
>> of projects.  Custom Compiler classes are a possibility now too.
>>
>> One could ship a copy of those dependencies with each project, or have
>> some kind of bootstrap script.  But to be able to automatically
>> download and add build dependencies to the path (a la setup_requires)
>> would be much nicer.  And packaging will have pysetup, so it should be
>> doable.  (Having the same capability for test dependencies and doc
>> dependencies would be nice too, but not nearly as important).
>
>
> Certainly.  I was just saying that the generated-metadata cases need
> handling, too, and that people should also be informed that they don't need
> (and shouldn't use) setup_requires for simple metadata generation, and that
> it might be a good idea to call the feature "build_requires", and perhaps
> distinguish between "setup hooks" (for developers to have nice things) and
> "build hooks" (for stuff that absolutely has to run on the install machine).

I think we're on the same page then :)  packaging/d2 also supports
pre/post-command hooks which might, in most cases, be more appropriate
for the "build hooks that absolutely have to run" case.  In fact, I
had completely forgotten this, but my aforementioned Numpy extension
module hook is a pre-build_ext hook.  That makes it absolutely clear
that this is something we have to do before we can build an extension
module, and that it doesn't have any purpose outside that context and
shouldn't be executed ever time I run `pysetup something`.

Erik