[Distutils] waf can be used to compile Python extension modules

[Paul Moore]

On 23 March 2014 00:00, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 23 Mar 2014 09:35, "Paul Moore" <p.f.moore at gmail.com> wrote:
>>
>> On 22 March 2014 22:37, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> >> 1. Formally decouple the setup.py CLI from the distutils command system
>> >
>> > Agreed - it looks like a compatible CLI needs to be part of the
>> > transition
>> > to any new system (I assume that's what you meant).
>>
>> Does this statement mean "document the exact setup.py invocations
>> which need to be supported by any tool" or does it mean "create a new
>> standard cli and update pip to use it and sort out a means of wrapping
>> existing setup.py scripts to expose it"? If the former, then I've
>> informally done that earlier in the thread (basically "setup.py
>> bdist_wheel -d XXX" is it[1], if we assume that by the time it's
>> official pip will have completed the move to installing from sdist by
>> using wheels as an intermediate step). Writing a wrapper setup.py that
>> simply invokes a different tool such as bento is trivial.
>
> I mean actually writing it up and documenting all the *build options*
> (including cross compilation support). As far as I am aware, that hasn't
> been done to date (at least, not in an easily consumable form that people
> can link to).

Hmm. How about if, to start the ball rolling, I add a short section to
the pip documentation entitled "setup.py interface" that explains the
basic commands that pip uses. That can then be expanded on as
additional data is collected.

One thing though. I may be being dense here, but I'm not aware of any
way for pip to provide build options via "pip wheel". I certainly
don't know the pip wheel command you'd use to (for example)
cross-compile psutil for Windows on a Linux box. Or how you'd use pip
wheel to build pyYAML with libyaml support (you need to indicate where
the libyaml headers and library live). Obviously you can do this with
*distutils*, but that's different. We're not expecting bento, or
distil, or waf, or whatever, to support the full distutils command
line interface, surely?

The pip wheel command includes --build-options and --build-options
flags that translate to global and command-specific options on the
bdist_wheel command. Those are as follows:

Global options:
  --verbose (-v)  run verbosely (default)
  --quiet (-q)    run quietly (turns verbosity off)
  --dry-run (-n)  don't actually do anything
  --help (-h)     show detailed help message
  --no-user-cfg   ignore pydistutils.cfg in your home directory

Options for 'bdist_wheel' command:
  --bdist-dir (-b)  temporary directory for creating the distribution
  --plat-name (-p)  platform name to embed in generated filenames (default:
                    win-amd64)
  --keep-temp (-k)  keep the pseudo-installation tree around after creating
                    the distribution archive
  --dist-dir (-d)   directory to put final built distributions in
  --skip-build      skip rebuilding everything (for testing/debugging)
  --relative        build the archive using relative paths(default: false)
  --owner (-u)      Owner name used when creating a tar file [default: current
                    user]
  --group (-g)      Group name used when creating a tar file [default: current
                    group]
  --skip-scripts    skip building the setuptools console_scripts
  --universal       make a universal wheel (default: false)

Note that none of those control the build - the ones that do that are
options to the build_ext and build commands, but *distutils* doesn't
allow you to supply them in a bdist_wheel command. The same is true
for pip install, BTW. That sends the options to the setup.py install
command - again, not to setup.py build_ext.

I can imagine good ways of using the existing pip interface to drive
build tools (even distutils!) more effectively than we do at present.
But I don't think the current interface supports compilation arguments
(even if it looks like it does).

Paul.