[Numpy-discussion] specifying numpy as dependency in your project, install_requires

[Nathaniel Smith]

On Sat, Sep 22, 2012 at 1:18 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 11:39 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> So the question is, how do we get a .egg-info? For the specific case
>> Ralf ran into, I'm pretty sure the solution is just that if you're
>> clever enough to do an in-place build and add it to your PYTHONPATH,
>> you should be clever enough to also run 'python setupegg.py egg_info'
>> which will create a .egg-info to go with your in-place build and
>> everything will be fine.
>
> That command first starts rebuilding numpy.

No, it just seems to run the config and source-generation bits, not
build anything. It also leaves the .egg-info in the source directory,
which is what you want.

> The correct one seems to be
> 'python setupegg.py install_egg_info'. This does install the egg_info file
> in site-packages, but it's still not working:
>
> $ python -c "import numpy as np; print(np.__version__)"
> 1.8.0.dev-d8988ab
> $ ls
> /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
> ...
> numpy-1.8.0.dev_d8988ab-py2.6.egg-info
> ...
> $ pip install -U --no-deps pandas
> Exception:
> Traceback (most recent call last):
> ...
> VersionConflict: (numpy 1.5.1
> (/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages),
> Requirement.parse('numpy>=1.6')).

The problem here is that you have numpy 1.5.1 installed in a directory
that appears on your PYTHONPATH *before* the directory that you
installed the .egg-info into. The .egg-info is supposed to go in the
same directory as the package; that way 'import numpy' and pip will
always find corresponding versions of numpy, no matter how you change
your PYTHONPATH.

It does look like you can also use: python setup.py install_egg_info -d .

This just uses standard distutils and skips running the
config/source-generation step. (I guess this is because the vanilla
distutils egg-info metadata is less thorough, and doesn't include a
list of installed files.)

> As long as you try to do anything with PYTHONPATH, I think
> pip/easy_install/setuptools are broken in a quite fundamental way.

I still see no evidence of this. Just put the .egg-info next to the
package and move on...

>> P.S.: yeah the thing where pip decides to upgrade the world is REALLY
>> OBNOXIOUS. It also appears to be on the list to be fixed in the next
>> release or the next release+1, so I guess there's hope?:
>> https://github.com/pypa/pip/pull/571
>
> Good to know. Let's hope that does make it in. Given it's development model,
> I'm less optimistic that easy_install will receive the same fix though ....

Yeah, easy_install is abandoned and bit-rotting, which is why people
usually recommend pip :-). But in this case, I thought that
easy_install already doesn't upgrade the world when it runs? Is there
something to fix here?

> Until both pip and easy_install are fixed, this alone should be enough for
> the advice to be "don't use install_requires". It's not like my alternative
> suggestion takes away any information or valuable functionality.

pandas, for example, requires several other packages, and I found it
quite convenient the other day when I wanted to try out a new version
and pip automatically took care of setting all that up for me. It even
correctly upgraded numpy, since the virtualenv I was using for testing
had inherited my system-installed 1.5.2, but this was the first
version of pandas that needed 1.6.

Python packaging tools make me feel grumpy and traumatized too but I
don't see how the solution is to just give up on computer-readable
dependency-tracking altogether.

-n