[Distutils] Specify missing install-requires dependencies to develop egg?

[Matthew Brett]

Hi,

On Fri, Aug 26, 2011 at 6:05 AM, Jim Fulton <jim at zope.com> wrote:
> On Thu, Aug 25, 2011 at 9:06 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>> Hi,
>>
>> We have been working on using buildout with a develop package
>> depending on numpy, and ran into trouble because 'numpy' is not
>> specified in the package 'install_requires'.
>>
>> Specifically, we have something like this:
>>
>> [buildout]
>> parts =
>>    nibabel-target
>>    mypy
>>
>> [nibabel-target]
>> recipe = zc.recipe.egg:develop
>> setup = src/nibabel
>>
>> [mypy]
>> recipe = zc.recipe.egg:script
>> eggs =
>>    nibabel
>> interpreter = py
>>
>> 'nibabel' needs to import numpy to install, but does not specify
>> 'numpy' in its setup.py install_requires, because pip install of numpy
>> is fragile, and we wanted to error early in general for an attempted
>> pip install of nibabel, if numpy was not present.
>>
>> But of course the script above raises an error for the
>> 'nibabel-target' part, because of missing numpy:
>>
>> -------------
>> RuntimeError: Cannot import package "numpy" - is it installed?
>> While:
>>  Installing nibabel-target.
>>
>> An internal error occurred due to a bug in either zc.buildout or in a
>> recipe being used:
>> ------------
>>
>> What we would very much like to do, is to be able to specify the
>> missing install_requires for this package (and for others we are
>> thinking of working with), in the recipe - something like:
>>
>> [nibabel-target]
>> recipe = zc.recipe.egg:develop
>> setup = src/nibabel
>> install-requires =
>>    numpy
>>
>> but we couldn't immediately see a way of specifying such a dependency.
>>  A quick look at the Develop code in zc/recipe/egg/custom.py
>> suggested that there wasn't an easy way to add some needed egg onto
>> the path at this point.
>>
>> Is there anything we missed that we should have tried?
>>
>> If not, what would be the best way to solve our problem?  A modified recipe?
>
> Theoretically, you'd use setup_requires to specify packages needed by setup.
> I'm not sure if that works with buildout. I've heard reports that it
> does and reports that
> it doesn't.

Ah - yes - thanks - I had forgotten about setup_requires.

> Why does nibabel want numpy at setup time? If it's to get C incluse files, then
> you may be out of luck, as it's unlikely that buildout will install
> numpy in a way that
> will give nibabel what it wants.

As I was saying in my previous message, nibabel just needs numpy at
runtime.  However, various of our other packages need it at setup
time, to find C include files, or to use numpy.distutils.

When running these various failing buildouts, numpy got installed as
an egg into the ./egg directory.  If I put that egg onto the path with
PYTHONPATH in my bare virtualenv, and then rerun this basic original
buildout.cfg

[buildout]
parts =
    nibabel-target
    mypy

[nibabel-target]
recipe = zc.recipe.egg:develop
setup = src/nibabel

[mypy]
recipe = zc.recipe.egg:script
eggs =
    nibabel
interpreter = py

then it runs to completion.  With ./bin/py:

>>> import numpy
>>> numpy.get_include()
'/home/mb312/tmp/buildbotting/eggs/numpy-1.6.1-py2.6-linux-x86_64.egg/numpy/core/include'

- so I think the C includes would work.

Looking at the bugs pointed to by Wichert, it sounds like your
previous suggestion of a 'setup-requires' for custom and develop
recipes might be the way to go - that was what I was thinking of but
mis-naming in my previous mail.

Cheers,

Matthew