On 7 July 2017 at 20:30, Nathaniel Smith <njs@pobox.com> wrote:
I think this is a really interesting idea, but it makes me very nervous that we're starting design work on novel features when we still haven't finalized a basic build_wheel hook.
This isn't a novel feature, it's a revised approach to how we split the responsibility for supporting out of tree builds between frontends and backends, since folks aren't happy with the current "prepare_input/build_artifact" split.
PEP 517 was written in 2015...
And PEP 426 was written in 2012. Standards development tImelines can get looong when the status quo at least kinda sorta works, and nobody has commercial deadlines forcing them to push to standardize new interfaces before a genuine consensus has developed :)
This proposal creates substantial complications for build systems that default to doing in-place builds, which is almost all existing system from 'make' onwards. Legacy build systems often can't do out-of-tree builds at all (e.g. consider the case where you have a vendored C library whose build system you want to einvoke as part of your build). Is this a problem? The benefits are potentially large, but are they worth it if they increase the barrier to entry for new build systems? I'm not sure how to tell. And in the mean time, pip is still unconditionally calling copytree before even looking at the source tree...
Is it absolutely necessary to get this into the first PEP?
From flit's point of view, Tomas wants frontends to able to express a
If we don't explicitly include out-of-tree build support as a design concept from the start, then we increase the risk of folks creating in-tree only build backends that then break down later when out-of-tree support is introduced as an expectation. If anyone was negatively impacted by that, they could fairly accuse us of perpetrating a bait-and-switch in the standards definition process. And since distilling that support down to its core essence has been the single most controversial aspect of the PEP to date, getting to an approach that at least pip, flit, enscons, and a hypothetical PEP 517 adapter for setuptools can all live with is a fairly important point to resolve. The latest round of discussions have been enlightening, as they have allowed us to articulate that from pip's point of view, the key requirement is to be able to tell a backend not to include anything that wouldn't be included when building via an sdist. preference between two different failure modes: 1. The frontend wants the wheel build to *guarantee* it exactly matches going via the sdist path 2. The frontend wants a working wheel build more than it cares about matching the sdist path The "call build_sdist, then call build_wheel on the unpacked archive" approach gives us the first option, which is sufficient for pip's needs, but *only* including that path doesn't provide the latter capability. Up until now, we've attempted to provide the latter feature through various incarnations of the "prepare_wheel_input" hook, which have been consistently confusing as to when and how frontends should call them, and backends should deal with them. Daniel's latest suggestion was merely to ask "What if rather than making that a separate hook, we made it a parameter to the existing build_wheel hook?". That way, it's entirely up to the *backend* to decide how best to align the results of an out-of-tree build with what you'd get from going through the sdist path (with one of the main options being to move the input files and then do an in-place build in the new directory, and the other main option being to use any native out-of-tree build capabilities in the underlying build system).
From pip's point of view, it can still try the build_sdist hook implicitly *first* for source installations, and only fall back to setting "build_directory" on the build_wheel hook if the sdist creation fails.
So everybody gets what they're looking for from the API, and the only extra concept we have to explain in PEP 517 is the difference between in-place ("build_directory=None") and out-of-tree ("build_directory" set to a filesystem path) wheel builds. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia