On Tuesday, September 4, 2018, Dustin Ingram <
di@python.org> wrote:
On Tue, Sep 4, 2018 at 11:33 AM Jeremy Stanley <fungi@yuggoth.org> wrote:
>
> Yes. If you haven't tried running a mirror of PyPI lately you're
> likely not to have noticed, but the various nightly builds for
> tensorflow seem to be the majority of the data on PyPI now. I'm sure
> it's a very neat and useful tool, but we basically had to switch
> from mirroring PyPI in our CI system to using a caching proxy
> because of this.
Side note: PyPI now provides a list of the largest packages by total
filesize: https://pypi.org/stats/
Bandwidth & download counts might be helpful here too.
Depending on what mirror you're using, you may be able to exclude
these packages from your mirror if you don't need them, e.g. for
bandersnatch: https://github.com/pypa/bandersnatch/blob/master/docs/filtering_configuration.md#blacklist-filtering-settings
A list of these as such might be helpful for maintenance of mirrors.