[Numpy-discussion] specifying numpy as dependency in your project, install_requires

Ralf Gommers ralf.gommers at gmail.com
Sat Sep 22 08:18:23 EDT 2012

On Fri, Sep 21, 2012 at 11:39 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Sep 21, 2012 at 9:42 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > Eh, just installing numpy with "python setup.py install" uses plain
> > distutils, not setuptools. So there indeed isn't an entry in
> > easy-install.pth.  Which some consider a feature:)
> I don't think this is correct. To be clear on the technical issue:
> what's going on is that when pip sees install_requires=["numpy"], it
> needs to check whether you already have the distribution called
> "numpy" installed. It turns out that in the wonderful world of python
> packaging, "distributions" are not quite the same as "packages", so it
> can't do this by searching PYTHONPATH for a "numpy" directory. What it
> does is search PYTHONPATH for a file named
> numpy-<version-number->.egg-info[1]. This isn't *quite* as dumb as it
> seems, because in practice there really isn't a 1-to-1 mapping between
> source distributions and installed packages, but it's... pretty dumb.
> Anyway. The problem is that Ralf installed numpy by doing an in-place
> build in his source tree, and then adding his source tree to his
> PYTHONPATH. But, he didn't put a .egg-info on his PYTHONPATH, so pip
> couldn't tell that numpy was installed, and did something dumb.
> So the question is, how do we get a .egg-info? For the specific case
> Ralf ran into, I'm pretty sure the solution is just that if you're
> clever enough to do an in-place build and add it to your PYTHONPATH,
> you should be clever enough to also run 'python setupegg.py egg_info'
> which will create a .egg-info to go with your in-place build and
> everything will be fine.

That command first starts rebuilding numpy. The correct one seems to be
'python setupegg.py install_egg_info'. This does install the egg_info file
in site-packages, but it's still not working:

$ python -c "import numpy as np; print(np.__version__)"
$ ls
$ pip install -U --no-deps pandas
Traceback (most recent call last):
VersionConflict: (numpy 1.5.1

As long as you try to do anything with PYTHONPATH, I think
pip/easy_install/setuptools are broken in a quite fundamental way.

The question is whether there are any other situations where this can
> break. I'm not aware of any. Contrary to what's claimed in the bit I
> quoted above, I just ran a plain vanilla 'python setup.py install' on
> numpy inside a virtualenv, and I ended up with a .egg-info installed.
> I'm pretty sure plain old distutils installs .egg-infos these days
> too.

You're right, that is the case.

> In that bug report Ralf says there's some problem with
> virtualenvs, but I'm not sure what (I use virtualenvs extensively and
> have never run into anything). Can anyone elaborate?

I haven't used them in a while, so I can't explain in detail now. Basic
numpy install into virtualenvs is working now AFAIK (which was quite
painful too), but I remember having problems when using them in combination
with PYTHONPATH too.

> [1] or several other variants, see some PEP or another for the tedious
> details.
> -n
> P.S.: yeah the thing where pip decides to upgrade the world is REALLY
> OBNOXIOUS. It also appears to be on the list to be fixed in the next
> release or the next release+1, so I guess there's hope?:
> https://github.com/pypa/pip/pull/571

Good to know. Let's hope that does make it in. Given it's development
model, I'm less optimistic that easy_install will receive the same fix
though ....  Until both pip and easy_install are fixed, this alone should
be enough for the advice to be "don't use install_requires". It's not like
my alternative suggestion takes away any information or valuable

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120922/6e8d71d9/attachment.html>

More information about the NumPy-Discussion mailing list