On Tue, 30 May 2017 at 11:40 Donald Stufft email@example.com wrote:
On May 30, 2017, at 2:17 PM, Brett Cannon firstname.lastname@example.org wrote:
Just to make sure I'm following this correctly, Donald is asking for:
Am I missing anything?
More or less. If I had my druthers we’d just add another mandatory method to the API, something like:
def build_sdist_tree(sdist_directory, config_settings) -> None: …
All this does is assemble the would be sdist tree into sdist_directory, handling any sdist creation time steps WITHOUT actually tar+gz’ing up this tree. My tests show that this is basically as fast as the copytree option in the simple case (which makes sense, that’s basically all it is) and is way faster than actually assembling the entire sdist tarball and all.
So the back-ends then need to provide a way for users to specify what files to go into the sdist (e.g. the MANIFEST.in case). Obviously a fallback is to just copy everything or just what is used to build the wheel (overly board or overly selective, respectively), but I doubt tools will want to do either in general and so they will need to come up with some solution to specify extra files like READMEs and stuff.
To handle creation, we can either have
sdist or something like
that which just calls the above API and then runs
on the final directory.
Do we want to say that sdist creation is the front-end's job? If so that is different from PEP 517 where the back-end packages zip up the wheel versus simply providing all the data and files that make up a wheel as seems to be suggested in the sdist case here.
When building a wheel, pip can then just inside of
this directory, call
into the wheel metadata/wheel build API and ultimately install that wheel.
It may even make sense for the build_wheel API to produce an “unpacked
wheel” as well and again let something like
twine wheel handle running
shutil.make_archive() on it. Similar to the sdist case, this would
further reduce the install time by avoiding the need to zip and then
immediately unzip inside of pip.
If we're going to go down a route where back-ends simply provide the files and the front-ends do the packaging then it makes sense to do that for wheels as well in PEP 517, separating the back-ends to create the components of the final artifact while the front-ends handle the bundling up for all of it into its final form. This makes the back-ends purely a compiler-and-metadata thing and the front-ends a manage-final-files thing.
One unrelated thing I just noticed, I don’t think the
PEP states how pip
is supposed to communicate to this API what it’s actually building. Is it
just assumed to be
os.curdir()? Ideally I think we’d add another
parameter to all of these functions that is the directory containing the
“input” to each of these.