Re: [Distutils] setup_requires: the obvious option(?)

30 Aug 2016

      On Tue, Aug 30, 2016 at 8:09 AM Erik Bray  wrote:
...
...
Hi all,
The `setup_requires` option to `setup()` is well-known to suffer from
multiple issues.  Most importantly, as it is a keyword argument to
`setup()`, it appears too late for modules that may need to be imported
for
the build to occur (e.g., Cython, for which support must explicitly
On Mon, Aug 29, 2016 at 7:29 PM, Antony Lee  wrote:
provided
...
by setuptools itself rather than by letting Cython hook into it);
additionally, there are various contorsions that people go to to avoid
some
`setup_requires` when not building the package (such as checking the
value
of `sys.argv`).  `setup_requires` also uses `easy_install` rather than
`pip`, but I do not see why this could not be fixed; let's focus on the
first issue instead.
If `setup_requires` appears too late to be useful, the obvious(?) option
is
to move it earlier: provide a function, say,
`setuptools.setup_requires()`,
that should be called *before* `setup()` itself, e.g.:
from setuptools import setup, setup_requires
    setup_requires("numpy", needed_for=["build_ext"])
    try:
        import numpy as np
    except ImportError:
        np = None
    setup(..., include_dirs=[np.get_include()] if np else [])
I mean this sort of already exists but it's spelled:
from setuptools import Distribution
Distribution({'setup_requires': ['numpy'])
Granted it's non-obvious and doesn't have the needed_for flag, which I
like.  It's not entirely clear how needed_for would work though.  For
example, what if the package you're requiring provides the command
that you need that package to run?
The same can be done by subclassing commands, and there can be some
corner cases where that gets extra tricky (Cython comes to mind).
...
When `setup.py` is invoked, either directly or by pip, upon the call to
`setup_requires()`, if `sys.argv[0]` is in the `needed_for` kwarg, and at
least one requirement is missing, `setup_requires()` calls asks pip to
install the required packages (similarly to
`https://bitbucket.org/dholth/setup-requires`
https://bitbucket.org/dholth/setup-requires) in a temporary directory,
and
the whole Python process replaces itself (in the `os.execv()` sense) by a
new call to `python setup.py` with this temporary directory prepended to
the
PYTHONPATH.  In this new process, the arguments to `setup_requires()` are
now available and we can proceed to the rest of `setup.py`.
I feel like this idea is natural enough that someone must already have
come
up with it... but I may be missing something :-)
I'm glad you mentioned Daniel Holth's setup-requires hack.  Although I
haven't used it myself directly I generally like the concept.
Yeah, setup_requires is a mess, but I'd be skeptical of solving the
problem by depending on any new features in setuptools :/
In June I updated my setup_requires hack to be a PEP 518 implementation. It
reads build-system.requires out of pyproject.toml and installs them into an
isolated directory if necessary. The basic feature is 14 lines of code not
counting blanks, and you use it by prepending the code to your setup.py.
Once pip implements PEP 518 that code becomes a no-op.

Antony is right that the main problem with setup_requires is that it
happens too late, or that you have to write a setuptools extension to use
it. Most people do not know that it is possible to write a setuptools
extension, let alone want to write one.