I think there's some pip bug somewhere discussing this ....
On Mon, May 29, 2017 at 9:19 PM, Paul Moore email@example.com wrote:
On 29 May 2017 at 08:05, Nathaniel Smith firstname.lastname@example.org wrote:
Right now pip doesn't really have a good way to expressing the latter.
'pip install directory/' is relatively unambiguously saying that I
want a local install of some potentially-locally-modified files, and while it might involve a temporary wheel internally there's no need to expose this in any way (and e.g. it certainly shouldn't be cached), so I think it's OK if this builds in-place and risks giving different results than 'pip install sdist.tar.gz'. (Also note that in the most common case where a naive user might use this accidentally, where they've downloaded an sdist, unpacked it manually, and then run 'pip install .', they do give the same results -- the potential for trouble only comes when someone runs 'pip install .' multiple times in the same directory.)
I think that the key thing here is that as things stand, pip needs a means to copy an existing "source tree", as efficiently as possible. For local directories (source checkouts, typically) there's a lot of clutter that isn't needed to replicate the "source tree" aspect of the directory - but we can't reliably determine what is clutter and what isn't.
Whether that copying is a good idea, in the face of the need to do incremental builds, is something of an open question - clearly we can't do something that closes the door on incremental builds, but equally the overhead of copying unwanted data is huge at the moment, and we can't ignore that.
Talking about a "build a sdist" operation brings a whole load of questions about whether there should be a sdist format, what about sdist 2.0, etc into the mix. So maybe we should avoid all that, and say that pip needs a "copy a source tree" operation. Backends SHOULD implement that by skipping any unneeded files in the source tree,but can fall back to a simple copy if they wish. In fact, we could make the operation optional and have pip fall back to copying if necessary. It would then be a backend quality of implementation issue if builds are slow because multi-megabyte git trees get copied unnecessarily.
Doesn't that just move the problem from pip to backends? It's still a choice between: (1) making no copy (good for in-place builds and also fine for pbr & co, but needs education or a "pip release" type command) (2) making a full copy like now including .git, .vagrant, etc. (super inefficient) (3) making an efficient copy (will likely still break pbr and setuptools-scm, and break in-place builds)
(This operation might also help tools like setuptools-scm that need
git information to work - the backend could extract that information on a "copy" operation and pit it somewhere static for the build).
If the backend can do it, so can pip right?
 As pip is currently the only frontend, "what pip needs right now" is the only non-theoretical indication we have of what frontends might need to do, so we should be cautious about dismissing this as "pip shouldn't work like this", IMO.
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig