[Distutils] A possible refactor/streamlining of PEP 517

[Nathaniel Smith]

I already responded to several of the overall points elsewhere in the
thread, but a few specific points:

On Mon, Jul 3, 2017 at 7:03 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I want prepare_wheel_metadata there as a straightforward way for
> backends to expose the equivalent of "setup.py egg_info". Losing that
> would be a significant regression relative to the status quo, since
> pip relies on it during the dependency resolution process

AFAIK this is a new argument that hasn't been made previously in any
of the discussions here. Is it a real problem that we've all missed?
This is important, because if pip really needs prepare_wheel_metadata
right now, then that's an argument that it needs to be a *mandatory*
hook, which is not the case in any version of PEP 517.

> To tighten that requirement up even further: if the backend's
> capabilities can't be accurately determined using
> "inspect.getattr_static", then the backend is not compliant with the
> spec. The build frontend/backend API is not a space where we want
> people to try to be clever - we want them to be completely dull and
> boring and use the most mundane code they can possibly come up with.

This does sound like a nice idea, but I mean, look at what people do
to distutils... I don't think there's much value in putting in
Strongly Worded Prohibitions that people will just ignore.

Also I don't think getattr_static is what you mean -- e.g. flit might
well write something like:

# flit/backend.py
if os.path.exists(".git"):
    def build_sdist(config_settings):
        ...

and that's totally getattr_static compliant.

>> - Add a TODO to decide how to handle backends that don't want to have
>>   multiple hooks called from the same process, including some
>>   discussion of the options.
>
> I don't think that's a TODO: I'm happy with the option of restricting
> frontends to "one hook invocation per subprocess call".
>
> It only becomes an open question in this revised draft by virtue of
> making the get_requires_* hooks mandatory, and I have a different
> resolution for that: keep those hooks optional, so that only backends
> that genuinely support dynamic build time dependencies will define
> them (others should either just get users to list any additional
> static build dependencies in pyproject.toml, or else list any
> always-needed static dependencies in the backend's own
> install_requires).

If you read the text, I pretty much came to the same conclusion :-). I
wanted to flag it as TODO though b/c AFAICT there hasn't been any
discussion of the trade-offs on the list, so the text I wrote is
raising new topics.

That said, if you have prepare_build_metadata and prepare_build_files,
then those *also* raise the issue of whether you really want to have
to spawn a new process. They ~double the number of process spawns,
which is significant overhead on Windows.

I suspect the right answer in any case is to default to running each
hook in a new process, and either now or later add a flag the backend
can provide to say that it's happy to run multiple hooks in the same
process. (Adding this later is totally backwards compatible if the
default is separate processes.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org