[Distutils] PEP 517: Open questions around artifact export directories

[Nick Coghlan]

On 16 June 2017 at 14:25, Nathaniel Smith <njs at pobox.com> wrote:
> On Thu, Jun 15, 2017 at 7:58 PM, Donald Stufft <donald at stufft.io> wrote:
>>
>>
>> On Jun 15, 2017, at 10:51 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> I can see some benefit to standardizing on a single format instead of
>> making every backend author learn e.g. the weird quirks required to
>> get unicode filenames correct in two different formats, and I'm not
>> sure why it would be a big deal to change the default for new tools
>> going forward given that all our infrastructure does support .zip
>> already.
>>
>>
>>
>> I believe it would be fairly disruptive for downstream redistributors like
>> Debian whose tooling is designed around the idea of a .tar.gz file, and whom
>> are forced to repack .zip files into .tar.gz files AIUI. I don’t really
>> personally care that much about the difference between .zip or .tar.gz for a
>> sdist, I just want to minimize people yelling at me unless it’s for a good
>> reason, and I don’t think this is a good reason :)
>
>
> You got me curious, and well, this is Debian we're talking about... judging
> from [1] it looks like at least someone writing their tooling is prepared
> for:
>
> \.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))
>
> OTOH I think PEP 517 will break everything else about their tooling, since
> it all assumes the presence of setup.py? Not that any of this makes an
> argument for .tar.gz or .zip in particular... that's the hard part about
> this question, it just doesn't matter very much :-).

Which is why the "current popularity" argument carries the day. It
doesn't really matter all that much (since Python can handle both
regardless of platform, and we don't care all that about the
ease-of-use of working with sdist's without unpacking them first), and
tar.gz is an order of magnitude more popular on PyPI (with no
likelihood of that ratio ever shifting significantly in zip's favour),
so tar.gz wins.

If current popularity doesn't feel persuasive, then we can also go
with "BDFL-Delegate fiat", since that also applies here :)

The previous discussion on this is covered in
https://www.python.org/dev/peps/pep-0527/, which changed PyPI to only
allow uploads of sdists as .tar.gz *or* .zip files (and only one or
the other for any given release). In that case, we didn't want to
disrupt the projects currently using zip sdists by deliberate choice
(rather than as the WIndows default), so we left them alone (but also
changed the tar.gz default to be platform independent).

For PEP 517 though, we're defining a *new* build interface, so anyone
adopting it is necessarily changing their publishing processes anyway.
I'm OK with having some folks resent me for making "use tar.gz sdists"
a hard requirement for PEP 517 adoption if the pay-off is to make it
easier for tool developers to adapt to PEP 517 (since they only need
to handle that tar.gz sdist format, leaving zip sdists as purely a
setuptools implementation detail) and (given time) for the overall
ecosystem to see increased consistency in the format of published
sdists.

For wheel files, by contrast, we *do* care about supporting random
access, so tar was never even considered as an option. That means that
if folks are looking for a post hoc rationalisation of a decision that
was really made for hysterical raisins [1], then "wheels are designed
to support efficient random access, but sdists arent, so wheels use
zip, while sdists use tar.gz" works.

Cheers,
Nick.

[1] courtesy of a former teammate, "hysterical raisins" = historical
reasons for design decisions that don't necessarily make all that much
sense when considered outside their particular historical context :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia