[Distutils] Provisionally accepting PEP 517's declarative build system interface

[Brett Cannon]

On Tue, 30 May 2017 at 11:40 Donald Stufft <donald at stufft.io> wrote:

>
> On May 30, 2017, at 2:17 PM, Brett Cannon <brett at python.org> wrote:
>
> Just to make sure I'm following this correctly, Donald is asking for:
>
>    - A way for pip to ask back-ends what files should be in an sdist from
>    a source checkout or to make an actual sdist
>       - Because sdists are a thing and so we should support them properly
>       - To make it so building wheels interact with just the files from
>       an sdist instead of skipping that and going straight from a source checkout
>    - Have this be a part of PEP 517 or at least a requirement for
>    back-ends to support so it doesn't get left out
>
> Am I missing anything?
>
>
>
> More or less. If I had my druthers we’d just add another mandatory method
> to the API, something like:
>
>     def build_sdist_tree(sdist_directory, config_settings) -> None:
>         …
>
> All this does is assemble the would be sdist tree into sdist_directory,
> handling any sdist creation time steps WITHOUT actually tar+gz’ing up this
> tree. My tests show that this is basically as fast as the copytree option
> in the simple case (which makes sense, that’s basically all it is) and is
> way faster than actually assembling the entire sdist tarball and all.
>

So the back-ends then need to provide a way for users to specify what files
to go into the sdist (e.g. the MANIFEST.in case). Obviously a fallback is
to just copy everything or just what is used to build the wheel (overly
board or overly selective, respectively), but I doubt tools will want to do
either in general and so they will need to come up with some solution to
specify extra files like READMEs and stuff.


>
> To handle creation, we can either have ``twine sdist`` or something like
> that which just calls the above API and then runs ``shutil.make_archive()``
> on the final directory.
>

Do we want to say that sdist creation is the front-end's job? If so that is
different from PEP 517 where the back-end packages zip up the wheel versus
simply providing all the data and files that make up a wheel as seems to be
suggested in the sdist case here.


>
> When building a wheel, pip can then just inside of this directory, call
> into the wheel metadata/wheel build API and ultimately install that wheel.
> It may even make sense for the build_wheel API to produce an “unpacked
> wheel” as well and again let something like ``twine wheel`` handle running
> ``shutil.make_archive()`` on it. Similar to the sdist case, this would
> further reduce the install time by avoiding the need to zip and then
> immediately unzip inside of pip.
>

If we're going to go down a route where back-ends simply provide the files
and the front-ends do the packaging then it makes sense to do that for
wheels as well in PEP 517, separating the back-ends to create the
components of the final artifact while the front-ends handle the bundling
up for all of it into its final form. This makes the back-ends purely a
compiler-and-metadata thing and the front-ends a manage-final-files thing.

-Brett


>
> One unrelated thing I just noticed, I don’t think the PEP states how pip
> is supposed to communicate to this API what it’s actually building. Is it
> just assumed to be ``os.curdir()``? Ideally I think we’d add another
> parameter to all of these functions that is the directory containing the
> “input” to each of these.
>
> —
>
> Donald Stufft
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170530/101f06ed/attachment.html>