On Tue, Jul 4, 2017, at 06:24 PM, Donald Stufft wrote:
It occurs to me that your case here is actually a reason *not* to implement this hook. The goal of the hook is that the wheel built from the tree created by copying this file is the same as the wheel built from a sdist created from that same VCS directory. However if you require the VCS tools in order to decide what files to include in the sdist, then you also need those tools to decide what files to copy into the temporary directory. Otherwise you’ll get different outputs.
The function of the VCS is to identify extra files that are needed for an sdist that don't affect building a wheel (like docs and tests). It's easy for flit to identify everything necessary for building a wheel, but those things are not sufficient for a good sdist.
For *flit* that may be the case, but this isn’t a hook solely for flit. It’s not hard to imagine that some backends will require git etc to correctly copy those files across. I don’t think we should look at adding special hooks for each possible set of requirements that a backend project might have, but that’s exactly what this hook is. If your backend has the same constraints as flit, then yea maybe it will work fine, but otherwise it won’t. Importantly, the most widely used build tool right now could not use any of it’s VCS plugins with this hook which I think says something.
Maybe the problem boils down to the fact we’re trying to treat VCS directories and unpacked Sdists the same and maybe we should just add a file which *only* gets added to a sdist (e.g. not a development install or a wheel) similar to the .dist-info/WHEEL file that just acts as a marker for “Hey, I’m in an sdist”
"Hey, I'm in something which was once an sdist, but may no longer be clean. Or someone has copied the marker file into a directory that was never an sdist to make something work because for some reason there's a difference in behaviour for directories that pip thinks come from sdists.”
I don’t care about starting an arms race with people who are going to purposely circumvent the operations of pip to try and force a square peg into a round hole. I think approximately zero people will do this by accident, and a tiny fraction will do it on purpose, and when they file bug reports I’ll just close them as invalid.
and if we hit that, then pip just does the copytree implementation. That absolves build backends from needing to worry about the unpacked sdist case, but makes it still function in 100% of the cases (assuming the build backend produces a correct sdist).
It still doesn't deal with the cases where you're not coming from an sdist but you can't get the VCS info for whatever reason.
I don’t think those cases are meaningfully different from any other case where a prerequisite for building isn’t installed that the frontend can’t automatically install. You error out and tell people to install the prerequisite.
We had reached a compromise that we all seemed to be OK with - albeit not very keen on. I find it very frustrating that we seem to be rehashing the same arguments that got us there in the first place.
I don’t think it’s (entirely) rehashing. The discussion has made me realize that the purported cases covered by the hook aren’t actually going to be covered except in a narrow set of circumstances, which suggests that it’s not actually a good hook.