On 6 July 2017 at 18:57, Thomas Kluyver <thomas@kluyver.me.uk> wrote:
Thank-you all for the discussion and the attempts to accommodate flit, but I'll bow out now. It's become clear that the way flit approaches packaging is fundamentally incompatible with the priorities other people have for the ecosystem.
While I can completely understand how the current debate over whether or not the prepare_input_for_build_wheel hook is necessary or not would make you feel that way, I hope I can convince you that we're really just quibbling over a genuinely trivial arcane technical detail that I'd never let get in the way of flit being a full-fledged participant in the Python packaging ecosystem. So I'll be completely clear: if *you* want that hook to be part of the API, that's sufficient reason for me to approve keeping it in the specification. However, I'll also make one last attempt at explaining why we suspect it may actually be redundant, especially if some additional configuration settings were added to pip. (I'll also note that I have an entirely selfish reason for ensuring that you're happy to continue active development on flit, which is that I've found it to be a delight to use for my own pure Python projects)
Namely, I see sdists as archival artifacts to be made approximately once per release, but the general trend is to make them a key part of the build pipeline.
Sort of.
From my point of view, the single most critical build path is:
1. Publisher uploads sdist to PyPI 2. Consumers and redistributors do their own from source builds targeting the binary format of their choice That's the path that enables the "publish my stuff once, let redistributors deal with getting that source code into arbitrary binary formats" dream: http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.ht... However, it's closely followed by the beginner-friendly: 1. Publisher uploads universal wheels or wheels for popular platforms to PyPI 2. Consumers use the appropriate wheel rather than doing their own builds Your relative priorities for those two paths are presumably the other way around, and that's fine. The only impact the current debate over whether or not to include a dedicated out-of-tree input preparation hook has on either of those paths relates to how much implicit testing the sdist building hook gets when developers do their own local builds (as if you're not using something like tox for your local testing, it's otherwise fairly easy to inadvertently publish sdists that don't actually include all the files they need to successfully build a wheel file). That is, the current point of contention is specifically about how we want tools to behave when we're starting with a source directory that: 1. Doesn't include VCS metadata (e.g. it's been exported as a tarball rather than cloned) 2. The build frontend doesn't want to use as the basis for an in-place build 3. The build frontend doesn't want to blindly copy into a separate build directory So just by way of those preconditions, we're already well outside the most common package installation workflows. Now, I'm personally entirely OK with backend developers saying that dealing with that scenario is entirely a frontend problem, since the problem is in large part *created* by the frontend's refusal to either do an in-place build, or just blindly copy the entire directory. It's a reasonable stance to take, and frontends and their users are in the best position to know how they want to handle it for their particular use case. That perspective is embodied in the hypothetical proposal to add a "--build-strategy" option to pip that would allow folks building wheels to choose between: - creating and unpacking an sdist and building a wheel from that - copying the directory tree and building a wheel from that - building a wheel directly from the original directory (Perhaps with a variant that tries to create and unpack the sdist first, and only if that fails falls back to copying the entire tree)
From a backend point of view, developers would then only need to worry about two cases:
- given a directory, either create an sdist, or fail, and report why it didn't work - given a directory, either create a wheel, or fail, and report why it didn't work The rest of the UX becomes a frontend developer concern. However, if you don't find that perspective compelling, and would strongly prefer to include a way to let frontends skip building an sdist just to do an out-of-tree wheel build, then I *don't* think that's a barrier to accepting the PEP. It would just be up to frontends how they wanted to prioritise that hook: - always use it in preference to build_sdist if present - use it as a fallback if build_sdist fails - never use it (for frontends that specifically want the sdist) The benefit I see to that approach is that it means that backend developers can clearly communicate the difference between what's required to make a source archive that's good enough to later build the wheel archive, and what's required to actually capture all the files that the publisher wants to publish.
Making a guerilla tool with no concern for integration was fun. It became frustrating as people began to use it and expected it to play well with other tools, so I jumped on PEP 517 as a way to bring it into the fold. That didn't work out, and a tool that doesn't play well with pip can only be an attractive nuisance at best, even if it technically complies with the relevant specs.
Flit is therefore deprecated, and I recommend anyone using it migrate back to setup.py packaging.
I stated this above, but it bears repeating: as the one currently offering to do the work for PEP 517, you *do* have the right to say that your willingness to make that contribution is contingent on the "prepare_input_for_build_wheel" hook being included in the design. While I see merit in the arguments raised against it, I'm personally OK with it, and ultimately, that's part of why we have the BDFL-Delegate system: so we don't lose potentially major contributions over details that ultimately don't matter all that much. The idea is for folks to end up mad at *me* for any such decisions that they don't like, rather than at the folks offering to volunteer their time to help improve the ecosystem for everyone (in this case, you). Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia