[Distutils] API CHANGE - Migrating from MD5 to SHA2, Take 2

[Donald Stufft]

Starting a new thread with more explicit details at Richard’s request.
Essentially the tl;dr here is that we'll switch to using sha2 (specifically
sha256).


Simple API
----------

Drop the #md5= from the PyPI hosted tarballs and replace it with #sha256, the
~60 or so externally hosted files which are using #md5 links will be fetched
(one time) verified, and have their #md5= hash replaced with a computed
#sha256= hash.

Impact:
  - pip: Will work with no issues, pip has supported sha256 since 1.2, and
         < 1.2 will install without a hash just fine.
  - setuptools: Will work with no issues, setuptools has supported sha256 since
                0.9 and < 0.9 will install without a hash just fine.
  - distribute: Doesn't support sha256, will intall without a hash just fine.
  - buildout: Uses setuptools/distribute to do the downloads I believe.
  - z3c.pypimirror: Appears to use MD5 hashes, but appears it won't error out
                    if they do not exist.


JSON / XMLRPC API
-----------------

Keep the md5_sum field, add an additional sha256_sum, suggest that applications
switch to using sha256 for verification.

Impact:
  - bandersnatch: bandersnatch will continue to use the md5_sum field from the
                  JSON (and previously XMLRPC) and should be updated to using
                  sha256 in the future.


Web UI
------

Simply replace any use of MD5 with SHA256, no clients are expected to access
anything here so this should be perfectly fine.


Other Clients
-------------

  - pep381client: Doesn't do anything special with the hash, will continue to
                  work.
  - devpi: ??? Unsure, I don't follow the code which fetches from PyPI so I
           can't determine where it gets the md5sum from and what it will do if
           it doesn't exist. It does have some handling of md5 though.


List of clients to look at taken from http://d.stufft.io/image/402r1s442m2r,
which is generated by looking at what is downloading the files from PyPI.


---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA