On Tue, Jun 2, 2015 at 3:41 AM, Robert Collins wrote:

On 2 June 2015 at 02:51, Erik Bray wrote:

...

On Sun, May 31, 2015 at 5:07 PM, AJ Friend wrote:

...

Hi,

I'm trying to write a new `setup.py` file for an extension module to wrap a C library (https://github.com/cvxgrp/scs).

The current `setup.py` file imports numpy. I'm trying to delay that import statement until setuptools has a chance to install numpy if it's not already installed. I'm trying to do that with this bit of code:

from setuptools.command.build_ext import build_ext as _build_ext class build_ext(_build_ext): def finalize_options(self): _build_ext.finalize_options(self) # Prevent numpy from thinking it is still in its setup process: __builtins__.__NUMPY_SETUP__ = False import numpy self.include_dirs += ext['include_dirs'] + [numpy.get_include()]

Running `python setup.py install` seems to work fine on my OSX machine, but when I run `pip install .` in the directory with `setup.py`, I get a clang error that it can't find one of the header files.

Any idea why that would be happening? Could it have anything to do with the relative path I'm giving for the include directories?

Also, I had trouble finding good documentation on subclassing build_ext. Does anyone know if setting self.include_dirs overwrites or appends to the include_dirs attribute of an Extension object defined later in setup.py?

For the curious, my current attempt at setup.py is athttps://github.com/ajfriend/scs/blob/setup2/python/setup.py. The original can be found in the same directory.

More generally, since I'm new to python packaging, I'm not sure how well or correctly I've written my `setup.py` file. Any feedback on doing things correctly would be appreciated.

Hi AJ,

For a lot of things in Python packaging there is not, sadly, One Right Way to Do It. Your setup.py looks okay though.

You may want to have a look at the get_numpy_include_path utility here: https://github.com/astropy/astropy-helpers/blob/7ee7e543641759ed1ee2b691bba1...

It's similar to what you're already doing, but maybe a little more 'robust'. In particular, I think the reload of the numpy module may be important.

It is, because what numpy is doing is importing from its own tree before setup has completed - which is in general unsafe.

And numpy may be in process depending on exactly what setuptools setup_requires easy-install code has done, which can lead to the symptoms you see. This should be better once we get pip interpreting setup_requires for you, but a perhaps shorter term fix would be to factor out the distutils support glue from numpy (which seems pretty self contained) into a out of package file - either adjacent to setup.py, or, if numpy is willing to make the dependency on setuptools a hard dep (which I recommend), then as a separate package which can be setup_require'd.

That would avoid the bootstrapping complexity, resulting in numpy not being imported until it has actually been installed, and things should be rosy for folk downstream of numpy from that point on.

That might not be a bad idea. This is more or less what we did for Astropy with astropy-helpers (this was done also so that other software distributions in the Astropy affiliated package ecosystem could take advantage of it). I think maybe splitting out much of the stuff in numpy.distutils into its own package (though unfortunately it would need a new name since I don't think we could make numpy into a namespace package) might help. Though I'm not sure how that would work for numpy.get_includes(). Would it just figure out where the include files *would* be installed to if one were installing Numpy into the currently active environment? Erik