[Distutils] Provisionally accepting PEP 517's declarative build system interface

[Paul Moore]

On 1 June 2017 at 23:14, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Thu, Jun 1, 2017, at 10:49 PM, Paul Moore wrote:
>> pip also needs a way to deal with "pip install <local directory>. In
>> this case, pip (under its current model) copies that directory to a
>> working area. In that area, it runs the build command to create a
>> wheel, and proceeds from there. In principle, there's little change in
>> a PEP 517 world. But again, see below.
>
> I still question whether the copying step is necessary for the frontend.
> Pip does it for setup.py builds (AIUI) because they might modify or
> create files in the working directory, and it wants to keep the source
> directory clean of that. Flit can create a wheel without
> modifying/creating any files in the working directory.

That's a very fair comment, and I honestly don't know how critical the
copy step is - in the sense that I know we do it to prevent certain
classes of issue, but I don't know what they are, or how serious they
are. Perhaps Donald does?

It's certainly true that setup.py based builds are particularly
unpleasant for the obvious "running arbitrary code" reasons. But I'm
not sure how happy I am simply saying "backends must ..." what? How
would we word this precisely? It's not just about keeping the sources
clean, it's also about not being affected by unexpected files in the
source directory. Consider that a build using a compiler will have
object files somewhere. Should a backend use existing object files in
preference to sources? What about a backend based on a tool designed
to do precisely that, like waf or make? What if the files came from a
build with different compiler flags? Sure, it's user error or a
backend bug, but it'll be reported to pip as "I tried to install foo
and my program failed when I imported it". We get that sort of bug
report routinely (users reporting bugs in build scripts as pip
problems) and we'll never have a technical solution to all the ways
they can occur, but preventative code like copying the build files to
a clean location can minimise them. (As I say, I'm speculating about
whether that's actually why we build in a temp location, but it's
certainly the sort of thinking that goes into our design).

Paul