On Mon, Aug 29, 2016 at 7:29 PM, Antony Lee <anntzer.lee@gmail.com> wrote:
Hi all,
The `setup_requires` option to `setup()` is well-known to suffer from multiple issues. Most importantly, as it is a keyword argument to `setup()`, it appears too late for modules that may need to be imported for the build to occur (e.g., Cython, for which support must explicitly provided by setuptools itself rather than by letting Cython hook into it); additionally, there are various contorsions that people go to to avoid some `setup_requires` when not building the package (such as checking the value of `sys.argv`). `setup_requires` also uses `easy_install` rather than `pip`, but I do not see why this could not be fixed; let's focus on the first issue instead.
If `setup_requires` appears too late to be useful, the obvious(?) option is to move it earlier: provide a function, say, `setuptools.setup_requires()`, that should be called *before* `setup()` itself, e.g.:
from setuptools import setup, setup_requires setup_requires("numpy", needed_for=["build_ext"]) try: import numpy as np except ImportError: np = None setup(..., include_dirs=[np.get_include()] if np else [])
I mean this sort of already exists but it's spelled: from setuptools import Distribution Distribution({'setup_requires': ['numpy']) Granted it's non-obvious and doesn't have the needed_for flag, which I like. It's not entirely clear how needed_for would work though. For example, what if the package you're requiring provides the command that you need that package to run? The same can be done by subclassing commands, and there can be some corner cases where that gets extra tricky (Cython comes to mind).
When `setup.py` is invoked, either directly or by pip, upon the call to `setup_requires()`, if `sys.argv[0]` is in the `needed_for` kwarg, and at least one requirement is missing, `setup_requires()` calls asks pip to install the required packages (similarly to `https://bitbucket.org/dholth/setup-requires`) in a temporary directory, and the whole Python process replaces itself (in the `os.execv()` sense) by a new call to `python setup.py` with this temporary directory prepended to the PYTHONPATH. In this new process, the arguments to `setup_requires()` are now available and we can proceed to the rest of `setup.py`.
I feel like this idea is natural enough that someone must already have come up with it... but I may be missing something :-)
I'm glad you mentioned Daniel Holth's setup-requires hack. Although I haven't used it myself directly I generally like the concept. Yeah, setup_requires is a mess, but I'd be skeptical of solving the problem by depending on any new features in setuptools :/ Best, Erik