[Distutils] PEP 527 - Removing Un(der)used file types/extensions on PyPI

Donald Stufft donald at stufft.io
Tue Aug 23 17:17:31 EDT 2016

> On Aug 23, 2016, at 5:05 PM, Daniel Holth <dholth at gmail.com> wrote:
> I would be sad to see .zip go. I would rather the rule be '0 or 1 sdists'. For my own selfish reasons I am trying to generate sdists with SCons, and it happens to expect the tar command but can already generate zips cross platform, so I will need to patch SCons' archiver to comply with the PEP. Sure, it only affects 2 packages, but they are mine.

It will be 0 or 1 sdists, having two sdists for a single version is not going to be a thing. However, standardizing around a single extension makes implementing that easier. It also makes it easier for people consuming things from PyPI because there is less variation in what they’re going to get. As time goes on, if we standardize on a single format, all of the older formats will end up just being old cruft that’s laying around, that nobody really installs or uses so  it doesn’t really matter if the tooling handles them or not.

> ZIP is interesting from the required C extensions angle because you can compress individual members with LZMA for example, breaking the 'only requires zlib' constraint; zipimport would not like those either. If the package already required Python 3.6 would this be a problem? Or the sdist could require the go compiler, and not work, but at least the packaging tool would not be the failure point.

I already plan to disallow uploads of wheels and sdists (if we keep .zip sdists) that use any storage methods besides ZIP_STORED and ZIP_DEFLATED. At least .tar.xz tells you on the tin if your current Python can support extracting it, compressing something in a zip file with LZMA is just a russian roulette of “will this extract or will it not?”. As far as whether or not depending on Python 3.6 means it’s no really an issue, that doesn’t solve the problem because the lzma module is an *optional* module, so any particular instance of Python 3.6 may or may not have it available.

> Wheel comes with a converter for bdist_wininst. They are conceptually very similar, both zip format, but bdist_wininst prepends an .exe installer.
> On Tue, Aug 23, 2016 at 4:13 PM Ian Cordasco <graffatcolmingov at gmail.com <mailto:graffatcolmingov at gmail.com>> wrote:
> On Tue, Aug 23, 2016 at 2:03 PM, M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>> wrote:
> > On 23.08.2016 18:46, Donald Stufft wrote:
> >> Since it seemed like there was enough here for a proper PEP I went ahead and
> >> write one up, which is now PEP 527. The tl;dr of it is that:
> >>
> >> * Everything but sdist, bdist_wheel, and bdist_egg get deprecated.
> >
> > -1 on removing bdist_wininst and bdist_msi. If PyPI is supposed
> > to retain the status of the main website to go search for Python
> > package downloads, it needs to be able to provide ways of hosting
> > all distribution types which are supported by distutils, including
> > ones which target platform configuration management system such as
> > the Windows one.
> >
> > The number of downloads is really irrelevant for this kind of
> > argument. Since the PEP proposes to keep the existing uploads
> > around, I also don't follow the argument of reduced maintenance.
> > PyPI will still have to host and support downloading those file
> > types.
> >
> > To me, all this sounds a lot like eventually turning PyPI into a
> > pip package index, which no longer serves the original intent of
> > a Python package index. I think that's taking a wrong turn in the
> > development of such an index.
> >
> > IMO, we should aim to reunite separate indexes such as the
> > one used for conda or the win32 index maintained by
> > Christoph Golke back into PyPI, not create even more
> > separation by removing platform specific formats.
> I disagree. As it is, tools like Twine only introspect the more common
> file formats to upload them to PyPI and has not had a single user
> complain about it. Given that Twine is advertised as the preferred
> method to upload to PyPI, you might expect that this would have been
> requested already. Alas it has not. No one using modern tooling for
> package management is uploading these file types.
> Even if the statistic of downloads (which actually is valid) doesn't
> sway you, maybe this will. Alternatively, maybe you will help maintain
> support for these file types in Warehouse?
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org <mailto:Distutils-SIG at python.org>
> https://mail.python.org/mailman/listinfo/distutils-sig <https://mail.python.org/mailman/listinfo/distutils-sig>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

Donald Stufft

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160823/a6ff91aa/attachment-0001.html>

More information about the Distutils-SIG mailing list