On 26-08-11 04:20, Matthew Brett wrote:

...

...

What we would very much like to do, is to be able to specify the

...

...

missing install_requires for this package (and for others we are thinking of working with)

The solution is quite simple: just add numpy to the list of eggs like this:

[mypy] recipe = zc.recipe.egg:script eggs = nibabel numpy interpreter = py That doesn't work for me. Rightly or wrongly I am guessing that the problem is that the dependency is not for the interpreter, but for running nibabel setup.py. In any case the failure remains with that addition:

Updating nibabel-target. Traceback (most recent call last): File "/tmp/tmpnKAmK6", line 11, in<module> execfile('/home/mb312/tmp/buildbotting/src/nibabel/setup.py') File "/home/mb312/tmp/buildbotting/src/nibabel/setup.py", line 40, in<module> package_check('numpy', NUMPY_MIN_VERSION) File "/home/mb312/tmp/buildbotting/src/nibabel/nisext/sexts.py", line 122, in package_check raise RuntimeError(msgs['missing'] % pkg_name) RuntimeError: Cannot import package "numpy" - is it installed? While: Updating nibabel-target.

"Updating nibabel-target": that's not the same part as that "mypy" target. Note that such an eggs attribute is only valid for that one part. So "[nibabel-target]" also should have an eggs attribute with numpy in it. Perhaps handier: add an "eggs" attribute to [buildout] with nibabel and numpy in it and use that: [buildout] .... eggs = nibabel numpy [mypy] ... eggs = ${buildout:eggs} ... [nibabel] ... eggs = ${buildout:eggs} ... DOes that work? Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"

Hi, On Thu, Aug 25, 2011 at 7:50 PM, Reinout van Rees wrote:

On 26-08-11 03:06, Matthew Brett wrote:

...

'nibabel' needs to import numpy to install

With your latest reply I re-read your original. Hm, this looks like a real problem: if your setup.py already imports from numpy (which your statement sounds like), numpy really needs to be there already.

Yes, that is what made me think that a new recipe might be the only way, where the recipe would put numpy on the path during the develop step.

The only thing I can think of right now is to have numpy installed globally and to use buildout 1.4.4 + osc.recipe.sysegg to depend on numpy. If you call the sysegg part right at the beginning, you might get it to work.

Yes, if numpy is already on the path, it works, but we wanted to test against several different numpy versions, and it seemed ugly and fragile to make a separate virtualenv for each python / numpy version pair. We'd much rather use the buildout mechanism to specify these, if possible.

Alternative is to re-work your setup.py. Do you really need to import numpy?

Actually, no, not in this case, it is only used as a crude dependency check. But, for other packages we have, we do need numpy.distutils for the setup.py - for example to find the numpy include directory for compiling C modules. The only reason I was offering this package as an example is that it is rather small and easy to install compared to the others. Thanks a lot, Matthew

On 08/26/2011 03:05 PM, Jim Fulton wrote:

Theoretically, you'd use setup_requires to specify packages needed by setup. I'm not sure if that works with buildout. I've heard reports that it does and reports that it doesn't.

I'm pretty sure it doesn't. There is an open bug for it: https://bugs.launchpad.net/zc.buildout/+bug/257555 . This bug looks related: https://bugs.launchpad.net/zc.buildout/+bug/110133 . Wichert.

Hi, On Fri, Aug 26, 2011 at 6:05 AM, Jim Fulton wrote:

On Thu, Aug 25, 2011 at 9:06 PM, Matthew Brett wrote:

...

Hi,

We have been working on using buildout with a develop package depending on numpy, and ran into trouble because 'numpy' is not specified in the package 'install_requires'.

Specifically, we have something like this:

[buildout] parts =    nibabel-target    mypy

[nibabel-target] recipe = zc.recipe.egg:develop setup = src/nibabel

[mypy] recipe = zc.recipe.egg:script eggs =    nibabel interpreter = py

'nibabel' needs to import numpy to install, but does not specify 'numpy' in its setup.py install_requires, because pip install of numpy is fragile, and we wanted to error early in general for an attempted pip install of nibabel, if numpy was not present.

But of course the script above raises an error for the 'nibabel-target' part, because of missing numpy:

------------- RuntimeError: Cannot import package "numpy" - is it installed? While:  Installing nibabel-target.

An internal error occurred due to a bug in either zc.buildout or in a recipe being used: ------------

What we would very much like to do, is to be able to specify the missing install_requires for this package (and for others we are thinking of working with), in the recipe - something like:

[nibabel-target] recipe = zc.recipe.egg:develop setup = src/nibabel install-requires =    numpy

but we couldn't immediately see a way of specifying such a dependency.  A quick look at the Develop code in zc/recipe/egg/custom.py suggested that there wasn't an easy way to add some needed egg onto the path at this point.

Is there anything we missed that we should have tried?

If not, what would be the best way to solve our problem?  A modified recipe?

Theoretically, you'd use setup_requires to specify packages needed by setup. I'm not sure if that works with buildout. I've heard reports that it does and reports that it doesn't.

Ah - yes - thanks - I had forgotten about setup_requires.

Why does nibabel want numpy at setup time? If it's to get C incluse files, then you may be out of luck, as it's unlikely that buildout will install numpy in a way that will give nibabel what it wants.

As I was saying in my previous message, nibabel just needs numpy at runtime. However, various of our other packages need it at setup time, to find C include files, or to use numpy.distutils. When running these various failing buildouts, numpy got installed as an egg into the ./egg directory. If I put that egg onto the path with PYTHONPATH in my bare virtualenv, and then rerun this basic original buildout.cfg [buildout] parts = nibabel-target mypy [nibabel-target] recipe = zc.recipe.egg:develop setup = src/nibabel [mypy] recipe = zc.recipe.egg:script eggs = nibabel interpreter = py then it runs to completion. With ./bin/py:

...

...

import numpy numpy.get_include() '/home/mb312/tmp/buildbotting/eggs/numpy-1.6.1-py2.6-linux-x86_64.egg/numpy/core/include'

- so I think the C includes would work. Looking at the bugs pointed to by Wichert, it sounds like your previous suggestion of a 'setup-requires' for custom and develop recipes might be the way to go - that was what I was thinking of but mis-naming in my previous mail. Cheers, Matthew