On 04/09/2013 11:52 PM, Trishank Karthik Kuppusamy wrote:
I have finished generating the /simple metadata and they are about 52MB --- not too far off from my estimate of 59MB. Remember: this is the worst-case size for simple metadata.
Okay, so we have finished generating the TUF metadata for a complete (if not the latest) set of PyPI packages. Summary of the largest metadata, assuming the worst case of a key per package on PyPI: release.txt: 11MB /simple metadata: 52MB /packages metadata: 96MB All in all, the metadata sums to about 159MB. With the data being 45GB, that works out to the metadata size being 0.35% of the data size. Remember: this is the worst case for the metadata, where every PyPI package has its own key, and there is a role for every possible target subdirectory. The metadata is also uncompressed JSON. As we have said before, we think we can do better (e.g, by reusing keys for multiple packages), and we are working on it. Simultaneously, we are testing a TUF-enabled version of pip against a TUF-secured PyPI mirror.