On 6 July 2017 at 07:45, Nathaniel Smith <njs@pobox.com> wrote:
On Wed, Jul 5, 2017 at 9:14 AM, Thomas Kluyver <thomas@kluyver.me.uk> wrote:
On Wed, Jul 5, 2017, at 05:08 PM, Paul Moore wrote:
is that flit doesn't handle scenarios like "I unpacked a sdist" or "I downloaded the project archive from github and unpacked that" well.
Flit handles these fine for everything *apart* from making an sdist. It can make a wheel, install the package, or symlink it as a development install. Hence why I'm so frustrated by the insistence that we must make an sdist when we have no need for an sdist.
It's not a compromise I'm entirely happy with, but all the other options that we came up with had bigger problems, IMO.
What do you think of the compromise in the draft that I posted at the beginning of this thread? The idea there is that flit would be responsible for providing the operations "build an sdist (or say that it can't)" and "build a wheel", and then if pip tries to build an sdist and flit tells it that it can't, it's pip's problem to figure out how it wants to handle that. Of course you'll still probably want to argue with the pip devs about how they handle this, and whether they can support in-place builds, etc., but at least those arguments stop being blockers for PEP 517.
Along those lines, I realised there's a variant of your "return NotImplemented" proposal that I actually like: we can actively encourage backends like flit that have additional requirements for building sdists to check for those external dependencies in `get_requires_for_build_sdist` and raise an exception if they're missing. That is, the "get_requires_*" hooks would have a dual responsibility: - fail outright if external dependencies are missing - otherwise report any Python level dependencies that the backend wants the frontend to install That would give us the following situation: - as long as the build environment has the relevant VCS tools available sdist-based builds for flit will "just work" - plenty of build environments are already going to have VCS tools routinely available anyway, and if they don't, adding them often won't be that big a deal - for the "unpacked sdist that the frontend doesn't know is an unpacked sdist" case, a backend like flit can use either PKG-INFO or else its own custom sdist marker file to detect unpacked sdist directories and just tar them back up to create a fresh sdist for the frontend to unpack
From a frontend evolution perspective, we'd then be anticipating one or the other of the following outcomes:
- pip gains a --build-strategy option to choose between 1) sdist based out-of-tree builds (the default); 2) copytree based out-of-tree builds; 3) in-place incremental builds - we eventually decide to revise the backend interface to add back a non-sdist based build tree preparation hook That frontend-centric `--build-strategy sdist|copytree|in-place` idea is starting to sound to me like it may be a better way to address both Thomas's concern about still being able to build a wheel when the requirements for creating an sdist aren't met ("--build-strategy copytree" or "--build-strategy in-place") and Nathaniel's concern about making it easy to share object files between successive builds ("--build-strategy in-place"). Recommending a frontend option like that also has a significant added benefit over the optional backend hook: it better abides by the principle of "In the face of ambiguity, refuse the temptation to guess" (with the ambiguity in this case being "What behaviour will people want if a frontend uses sdist-based out-of-tree wheel builds by default, but building the sdist fails in a situation where building the wheel directly would succeed?") Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia