[Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Nathaniel Smith]

Hi all,

Apparently it is not well known that if you have a Python project
source tree (e.g., a numpy checkout), then the correct way to install
it is NOT to type

  python setup.py install   # bad and broken!

but rather to type

  pip install .

(I.e., pip install isn't just for packages on pypi -- you can also
pass it the path to an arbitrary source directory or the URL of a
source tarball and it will do its thing. In this case "install ."
means "install the project in the current directory".)

These don't quite have identical results -- the main difference is
that the latter makes sure that proper metadata gets installed so that
later on it will be possible to upgrade or uninstall correctly. If you
call setup.py directly, and then later you try to upgrade your
package, then it's entirely possible to end up with a mixture of old
and new versions of the package installed in your PYTHONPATH. (One
common effect is in numpy's case is that we get people sending us
mysterious bug reports about failing tests in files don't even exist
(!) -- because nose is finding tests in files from one version of
numpy and running them against a different version of numpy.)

But this isn't the only issue -- using pip also avoids a bunch of
weird corner cases in distutils/setuptools. E.g., if setup.py uses
plain distutils, then it turns out this will mangle numpy version
numbers in ways that cause weird horribleness -- see [1] for a bug
report of the form "matplotlib doesn't build anymore" which turned out
to be because of using 'setup.py install' to install numpy. OTOH if
setup.py uses setuptools then you get different weirdnesses, like you
can easily end up with multiple versions of the same library installed
simultaneously.

And finally, an advantage of getting used to using 'pip install .' now
is that you'll be prepared for the glorious future when we kill
distutils and get rid of setup.py entirely in favor of something less
terrible [2].

So a proposal that came out of the discussion in [1] is that we modify
numpy's setup.py now so that if you try running

    python setup.py install

you get

    Error: Calling 'setup.py install' directly is NOT SUPPORTED!
    Instead, do:

        pip install .

    Alternatively, if you want to proceed at your own risk, you
    can try 'setup.py install --force-raw-setup.py'
    For more information see http://...

(Other setup.py commands would continue to work as normal.)

I believe that this would also break both 'easy_install numpy', and
attempts to install numpy via the setup_requires= argument to
setuptools.setup (because setup_requires= implicitly calls
easy_install). install_requires= would *not* be affected, and
setup_requires= would still be fine in cases where numpy was already
installed.

This would hopefully cut down on the amount of time everyone spends
trying to track down these stupid weird bugs, but it will also require
some adjustment in people's workflows, so... objections? concerns?

-n

[1] https://github.com/numpy/numpy/issues/6551
[2] https://mail.python.org/pipermail/distutils-sig/2015-October/027360.html

-- 
Nathaniel J. Smith -- http://vorpus.org