[Distutils] Finishing up PEP 517

[Nathaniel Smith]

On Fri, Jun 16, 2017 at 5:52 PM, Donald Stufft <donald at stufft.io> wrote:
> I’d probably want to spec this out as ext_{name-on-pypi}_* to remove a
> special case on pip in the PEP, to let others do experimentation as well.

Sure, like the [tool.*] escape hatch in pyproject.toml.

> However this solution is fine with me on both of the Non build_{sdist,wheel}
> hooks. Although I feel like it kind of defeats the purpose of the
> prepare_build_files hooks, because I think Thomas’ goal is to not have to
> support build_sdist from within another sdist, and if he depends on a pip
> extension (rather than it being an optional speed up like
> prepare_wheel_metadata) then we either break flit users if we decide it
> wasn’t worth it and we remove it, or we force other frontends to understand
> ext_pip_prepare_build_files and we get into the same kind of “everyone is
> just trying to emulate setuptools” bog that we’re trying to escape.

How I'm imagining it would work is:

Right now: pip just unconditionally uses shutil.copytree, and doesn't
even try to generate an sdist.

Eventually: someone writes the patch to pip to attempt to do the copy
step via generating an sdist. This is going to require some fallback
strategy for when building an sdist fails (because the legacy setup.py
command errors out, because the build_sdist hook is missing, because
the build_sdist hook errors out...); probably falling back on
shutil.copytree. For flit's purposes, this doesn't seem like a
disaster: flit projects don't have tens of megabytes of random build
artifacts lying around, and flit sdists in particular don't have tens
of megabytes of irrelevant VCS history. In fact, an unpacked flit
sdist is generally just as cheap to copy via shutil.copytree as any
other project would be to copy via build_sdist.

Eventually^2 (possibly part of the previous step): someone comes up
with a prototype ext_pip_prepare_build_files spec to optimize this
further, and it gets implemented in pip and flit. Experience shows
that the speedup is worthwhile, enough so that other build frontends
want to get in on the action.

Eventually^3: we write up a PEP that's just a copy/paste of the mature
pip behavior and tell other frontends to go for it.

But of course there are also other ways this could go. Like, I'm not
saying it's likely, but for all we know, between now and when pip gets
around to implementing the copy-via-sdist approach, Thomas might have
some brainstorm and realize that flit needs to implement
build-sdist-from-sdist for some other reason. Making predictions is
hard, especially about the future.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org