On Mon, Oct 26, 2015 at 10:31 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
Apparently it is not well known that if you have a Python project source tree (e.g., a numpy checkout), then the correct way to install it is NOT to type
python setup.py install # bad and broken!
but rather to type
pip install .
(I.e., pip install isn't just for packages on pypi -- you can also pass it the path to an arbitrary source directory or the URL of a source tarball and it will do its thing. In this case "install ." means "install the project in the current directory".)
These don't quite have identical results -- the main difference is that the latter makes sure that proper metadata gets installed so that later on it will be possible to upgrade or uninstall correctly. If you call setup.py directly, and then later you try to upgrade your package, then it's entirely possible to end up with a mixture of old and new versions of the package installed in your PYTHONPATH. (One common effect is in numpy's case is that we get people sending us mysterious bug reports about failing tests in files don't even exist (!) -- because nose is finding tests in files from one version of numpy and running them against a different version of numpy.)
But this isn't the only issue -- using pip also avoids a bunch of weird corner cases in distutils/setuptools. E.g., if setup.py uses plain distutils, then it turns out this will mangle numpy version numbers in ways that cause weird horribleness -- see [1] for a bug report of the form "matplotlib doesn't build anymore" which turned out to be because of using 'setup.py install' to install numpy. OTOH if setup.py uses setuptools then you get different weirdnesses, like you can easily end up with multiple versions of the same library installed simultaneously.
And finally, an advantage of getting used to using 'pip install .' now is that you'll be prepared for the glorious future when we kill distutils and get rid of setup.py entirely in favor of something less terrible [2].
So a proposal that came out of the discussion in [1] is that we modify numpy's setup.py now so that if you try running
python setup.py install
you get
Error: Calling 'setup.py install' directly is NOT SUPPORTED! Instead, do:
pip install .
Alternatively, if you want to proceed at your own risk, you can try 'setup.py install --force-raw-setup.py' For more information see http://...
(Other setup.py commands would continue to work as normal.)
I believe that this would also break both 'easy_install numpy', and attempts to install numpy via the setup_requires= argument to setuptools.setup (because setup_requires= implicitly calls easy_install). install_requires= would *not* be affected, and setup_requires= would still be fine in cases where numpy was already installed.
This would hopefully cut down on the amount of time everyone spends trying to track down these stupid weird bugs, but it will also require some adjustment in people's workflows, so... objections? concerns?
I gave it a shot the other day. Pip keeps a record of the path to the repo and in order to cleanup I needed to search out the file and delete the repo path. There is probably a better way to do that, but it didn't strike me as less troublesome than ` python setup.py install --local`. Chuck