[Distutils] Deprecating little used file types/extensions on PyPI?

[Nick Coghlan]

On 23 August 2016 at 00:46, Donald Stufft <donald at stufft.io> wrote:
>
>> On Aug 22, 2016, at 7:14 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> Thanks, that's good info that shows I was clearly being unduly
>> pessimistic about toolchain compatibility. It means the only salient
>> technical difference we're aware of between the two formats is that
>> the distutils and setuptools default settings on .tar.gz currently
>> result in smaller archives than the default settings for .zip.
>
> I’m less worried about the Linux toolchains and I’m more worried about
> the adhoc toolchains created by all the various publishers on PyPI. It’s
> not a wide stretch to imagine release scripts that hard code in an
> assumption on either .tar.gz _or_ .zip and picking one or the other will
> inevitably break these people (albeit with a fairly simple fix in the
> typical case). Picking the lesser used format just increases the number
> of people who might end up having their release scripts or other misc
> scripts broken because of it.

Yeah, that's fair. To summarise where we're at right now:

- the change to always use .tar.gz by default has been made in
distutils for 3.6, so if we want to do something else (like continue
allowing both .tar.gz and .zip and keep the Windows default as .zip),
we need to make that clear before the final beta in November
(*reverting* a behaviour change like that is permitted within the beta
period - otherwise the beta period wouldn't be particularly useful,
since we couldn't respond to community feedback)

- folks seem to be broadly in favour of standardising on ".zip"
whenever "formally define a new iteration of the sdist format" makes
it to the top of the collective todo list

- however, if we were to consolidate on a single sdist format *right
now* (regardless of whether that's .tar.gz or .zip), the concern is
that we risk breaking ad hoc automation and other tooling without a
compelling sales pitch to the folks that need to update their
toolchains about how the change will make *their* lives better

Furthermore, at a language ecosystem level, we don't particularly want
to inflict more disruption on either *nix users *or* Windows users
that are sufficiently engaged with the Python community to be
publishing their own software on PyPI - we still have that everpresent
"change overload" concern, especially for folks that have been
affected by both Python 3 *and* the packaging tooling changes in
recent years.

So I think the next step would be to summarise the changes to release
file hosting support and the permitted extensions in a PEP, with
details along the following lines:

Permitted formats:

- sdist (*.zip, *.tar.gz)
- bdist_wheel (*.whl)
- bdist_egg (*.egg)
- bdist_wininst (*.exe)
- bdist_msi (*.msi)

Dropped formats:

- bdist_dumb
- bdist_rpm (*.rpm)
- bdist_dmg (*.dmg)

Possible future design changes:

- defining sdist 2.0 (as an explicitly defined zip-based format)
- defining a successor to the egg format for the non-wheel use cases
- encouraging use of Windows Package Management over manual installer execution

The rationale for why the Windows formats get to stay when the other
platform specific formats are being dropped is implied by that last
line: we're expecting users on other platforms to be more comfortable
with using platform specific tooling to manage platform specific
formats (e.g. the system package manager on Linux, homebrew on Mac OS
X).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia