[Numpy-discussion] specifying numpy as dependency in your project, install_requires

[Nathaniel Smith]

On Fri, Sep 21, 2012 at 9:42 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> Eh, just installing numpy with "python setup.py install" uses plain
> distutils, not setuptools. So there indeed isn't an entry in
> easy-install.pth.  Which some consider a feature:)

I don't think this is correct. To be clear on the technical issue:
what's going on is that when pip sees install_requires=["numpy"], it
needs to check whether you already have the distribution called
"numpy" installed. It turns out that in the wonderful world of python
packaging, "distributions" are not quite the same as "packages", so it
can't do this by searching PYTHONPATH for a "numpy" directory. What it
does is search PYTHONPATH for a file named
numpy-<version-number->.egg-info[1]. This isn't *quite* as dumb as it
seems, because in practice there really isn't a 1-to-1 mapping between
source distributions and installed packages, but it's... pretty dumb.
Anyway. The problem is that Ralf installed numpy by doing an in-place
build in his source tree, and then adding his source tree to his
PYTHONPATH. But, he didn't put a .egg-info on his PYTHONPATH, so pip
couldn't tell that numpy was installed, and did something dumb.

So the question is, how do we get a .egg-info? For the specific case
Ralf ran into, I'm pretty sure the solution is just that if you're
clever enough to do an in-place build and add it to your PYTHONPATH,
you should be clever enough to also run 'python setupegg.py egg_info'
which will create a .egg-info to go with your in-place build and
everything will be fine.

The question is whether there are any other situations where this can
break. I'm not aware of any. Contrary to what's claimed in the bit I
quoted above, I just ran a plain vanilla 'python setup.py install' on
numpy inside a virtualenv, and I ended up with a .egg-info installed.
I'm pretty sure plain old distutils installs .egg-infos these days
too. In that bug report Ralf says there's some problem with
virtualenvs, but I'm not sure what (I use virtualenvs extensively and
have never run into anything). Can anyone elaborate?

[1] or several other variants, see some PEP or another for the tedious details.

-n

P.S.: yeah the thing where pip decides to upgrade the world is REALLY
OBNOXIOUS. It also appears to be on the list to be fixed in the next
release or the next release+1, so I guess there's hope?:
https://github.com/pypa/pip/pull/571