On 10 July 2017 at 18:56, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Mon, Jul 10, 2017 at 7:13 PM, Thomas Kluyver <thomas@kluyver.me.uk> wrote:
On Mon, Jul 10, 2017, at 07:01 AM, Nick Coghlan wrote:
So I think we have pretty solid evidence that the reason the procedural "build directory preparation" hook wasn't sitting well with people was because that isn't the way build systems typically model the concept, while a "build directory" setting is very common (even if that "setting" is "the current working directory when configuring or running the build").
Hooray! :-)
Do we want to also provide a build_directory for the build_sdist hook? In principle, I don't think making an sdist should involve a build step, but I know that some projects do perform steps like cython code gen or JS minification before making the sdist. I think this was a workaround to ease installation before wheel support was widespread, and I'd be inclined to discourage it now, so my preference would be no build_directory parameter for build_sdist. Backends which insist on generating intermediates at that point can make a temp dir themselves.
No preference on yes/no for build_directory for build_sdist hook, but invoking Cython on .pyx files to generate C code rather than checking in generated C code is good practice. I don't think we want to go back to checking in generated code, nor do we want to store generated code in tmpdirs (because that loses the advantage of not having to regenerate when .pyx hashes are identical).
If the frontend offers a way to specify a particular build directory, then that kind of caching wouldn't be a problem when regenerating files as part of build_wheel - all the build files would be preserved between runs, not only those that made it into the final wheel file. That's a slightly different question than the one Thomas was asking, which is whether or not we want to support out-of-tree builds for sdist creation, and I don't believe we do, since we don't really *want* folks to be adding generated files to their sdist that they aren't keeping under source control - we'd prefer that such activities were postponed to "build_wheel" now that we have separate source and precompiled distribution formats. If publishers still prefer to go down that path, then they can either use mechanisms like .gitignore to cope with the consequences of doing it in-place, or else use ccache-style caching mechanisms that aren't dependent on exactly where you do the build, and instead depend on the hash of the input artifact (e.g. I'd be astonished if cython couldn't implement something like ccache natively for pyx file compilation). Either way, it won't impact of the experience of folks using pre-compiled wheel files, and for at least some use cases for sdists (e.g. Linux distributions), it's typically preferable to regenerate that kind of file anyway. Cheers, NIck. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia