On 14 July 2017 at 11:32, Thomas Kluyver <thomas@kluyver.me.uk> wrote:
On Fri, Jul 14, 2017, at 11:18 AM, Nathaniel Smith wrote:
OTOH this whole 11th hour discussion of forcing every build system to have in-tree and out-of-tree build support is solving some other problem.
It is not my intention to force build systems to support either of these.
- Where build systems only support out-of-tree builds, it should be trivial to create a temporary directory when build_directory is not passed. - Where build systems only support in-tree builds, they should copy the necessary files to build_directory and run an in-tree build there. This is more complex, but it appears to be non-negotiable that there is some way of building without affecting the source directory, so whatever the interface is, we need some way to do this.
It's "non-negotiable" in the sense that we (the pip developers) have in the past had to deal with issues being reported that turned out to be because a developer had unexpected files in their build directory that resulted in the build being incorrect. That's not a case of "not trusting build systems to do incremental builds", it's direct evidence that developers sometimes make mistakes and think it's pip's fault (until we explain their mistake to them). The plan for pip, to address this ongoing issue, is to provide out of tree builds, so that we can ensure that the developer is protected against such mistakes. The UI for out of tree builds is not decided finally yet (for example, whether it's the default, or if a flag is needed). The reason for this is because current tools *make it impossible* for us to implement this (with one proviso, it appears that no-one ever actually tried the build a sdist, then build from the sdist route - I thought we had and had hit issues caused by certain tools, but I was apparently wrong). So rather than back ourselves into a new corner where pip can't provide out of tree builds to solve this issue, we need some means of doing this in PEP 517. The option to do it via sdists, or via "copy tree" hooks, became messy and was a problem for flit, so we took a second look at it and agreed that we'd got hung up on not being willing to trust backends to do out of tree builds. The new proposal was to add a means for frontends to request an out of tree build, and trust backends to provide that. I'm happy with the current proposal. We shouldn't have a problem trusting backends, and now we have a means of asking for what we want, so everything is fine. If the issue here is that front ends shouldn't be allowed to even *want* to do out of tree builds, I'm not sure what I can further say to convince you. We have actual bugs which we can solve with out of tree builds, and we can't think of another way to solve them. (Unfortunately, I can't find the issue in pip where we have discussed implementing out of tree builds, so I can't give you a pointer to actual bug reports - and anyway, most if not all of them will have been worked around by the developer cleaning up his or her source tree). I hope this clarifies a little - I wasn't too keen on seeing the requirement described as "non-negotiable" (or of it being linked to incremental builds), when it's actually nothing other than an identified need that addresses a real world problem frontends have to deal with (of not-always-pristine build directories). Paul