Sept. 4, 2018
4:40 p.m.
On Tue, Sep 4, 2018 at 11:33 AM Jeremy Stanley <fungi@yuggoth.org> wrote:
Yes. If you haven't tried running a mirror of PyPI lately you're likely not to have noticed, but the various nightly builds for tensorflow seem to be the majority of the data on PyPI now. I'm sure it's a very neat and useful tool, but we basically had to switch from mirroring PyPI in our CI system to using a caching proxy because of this.
Side note: PyPI now provides a list of the largest packages by total filesize: https://pypi.org/stats/ Depending on what mirror you're using, you may be able to exclude these packages from your mirror if you don't need them, e.g. for bandersnatch: https://github.com/pypa/bandersnatch/blob/master/docs/filtering_configuratio...