[Distutils] A possible refactor/streamlining of PEP 517

[Donald Stufft]

> On Jul 4, 2017, at 3:22 AM, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> 
> On Tue, Jul 4, 2017, at 01:06 AM, Donald Stufft wrote:
>> 2) We have a VCS directory or “original development source” or whatever you want to call the thing you have before a sdist that typically gets into a sdist.
>>     - Works on both proposals for setuptools and flit (since both can go from a VCS to a sdist).
>>     - Thomas might have said he’d be unhappy if this case goes through a real sdist… I forget the specifics of that discussion now. 
> 
> Practical objection: besides it being a VCS checkout, you need the VCS tools available (e.g. git on $PATH). It's not hard to imagine cases where this doesn't hold, e.g. installing from a directory bind-mounted into a docker container. Between this and your case 3 (local directory not a VCS checkout), failures - while not common - won't be particularly rare.
> 

It occurs to me that your case here is actually a reason *not* to implement this hook. The goal of the hook is that the wheel built from the tree created by copying this file is the same as the wheel built from a sdist created from that same VCS directory. However if you require the VCS tools in order to decide what files to include in the sdist, then you also need those tools to decide what files to copy into the temporary directory. Otherwise you’ll get different outputs.

Which means that besides the unpacked sdist case, this actually breaks the from VCS case unless the prepare build files hook has all the same requirements as the build_sdist hook, in which case we lose the purpose of the hook to begin with.

Maybe the problem boils down to the fact we’re trying to treat VCS directories and unpacked Sdists the same and maybe we should just add a file which *only* gets added to a sdist (e.g. not a development install or a wheel) similar to the .dist-info/WHEEL file that just acts as a marker for “Hey, I’m in an sdist” and if we hit that, then pip just does the copytree implementation. That absolves build backends from needing to worry about the unpacked sdist case, but makes it still function in 100% of the cases (assuming the build backend produces a correct sdist). I think that actually works in *more* cases without introducing weirdness where you get different results if the tools aren’t available via prepare build files. The main downsides I see are:

(A) Docker bind mounts etc will require git etc to install when the sdist generation requires them, but I don’t think you can get around this as I said above.
(B) It removes a place that a hypothetical build backend could use to mark that a unpacked sdist has been modified (e.g. it could store a RECORD with hashes of all the files, and if someone modifies it, when it builds the sdist it could add a local version to the version to indicate it’s been modified).

However to my knowledge, nothing does (B) and (A) is unavoidable IMO.


—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170704/26c5b12c/attachment.html>