[Distutils] greater compression on pypi?

Daniel Holth dholth at gmail.com
Mon Aug 22 10:29:23 EDT 2016

Some obvious ideas about how to enable greater compression for pypi, should
anyone be motivated enough to do so.

1. If it's a zip, nested zips like so,


The metadata is easy to get to, and everything else requires a second
unpack operation. data.zip is stored, and only compressed by the outer
.zip. This could be done in a backwards compatible way.

Wheel could be revised to put everything except *.dist-info inside a zipped
*.data directory.

2. Sign the uncompressed data

Check hashes and signatures against the .tar file instead of .tar.gz when
doing pip install ... #sha256=nnn. For zip, check against a hash of all the
hashes of the uncompressed members.

3. Go crazy

pypi is now free to re-compress without additional input from the
publisher. Both .gz and .lzma versions etc. could be offered.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160822/73e5b5d6/attachment.html>

More information about the Distutils-SIG mailing list