[Distutils] does pypi or red-dove have a better firehose API than "download all the packages"?

[Vinay Sajip]

Daniel Holth <dholth <at> gmail.com> writes:

> Is there an API for "all the metadata for everything" that doesn't
> require one web request per package version? Maybe something like an
> rdiff-backup of a database?

Well, the red-dove.com metadata works at three levels:

http://www.red-dove.com/pypi/projects/projects.json

gives a list of all PYPI project names, then e.g.

http://www.red-dove.com/pypi/projects/W/wheel/project.json

gives all the version numbers for wheel with download URLs, MD5 hashes (and
importantly, dependencies) and e.g.

http://www.red-dove.com/pypi/projects/W/wheel/package-1.0.0a2.json

will give all the particulars of that particular wheel version, including
dependencies, sources, build and test options, package data etc.

So, the middle level gives a useful overview of a project sufficient for
dependency resolution (it's what distlib and distil use).

The data is kept reasonably fresh by periodically looking at what's changed
recently on PyPI and updating the metadata for new releases etc.

Regards,

Vinay Sajip