[Distutils] does pypi or red-dove have a better firehose API than "download all the packages"?

[David Wilson]

Would something like http://pypi.h1.botanicus.net/static/dump.txt.gz be
useful to you? (warning: 57mb expanding to 540mb). Each line is a
JSON-encoded dict containing a single package release.

for line in gzip.open('dump.txt.gz'):
    dct = json.loads(line)
    ....

etc

The code for it is very simple, would be willing to clean it up and turn it
into a cron job if people found it useful.

Note the dump above is outdated, I only made it as a test.




On 15 May 2013 21:12, Daniel Holth <dholth at gmail.com> wrote:

> Yeah, I've been using the "run bandersnatch" API, but the local
> storage requirement is a bit hefty.
>
> On Wed, May 15, 2013 at 4:11 PM, Donald Stufft <donald at stufft.io> wrote:
> > Nvm missed the one web request requirement. No I don't think so.
> >
> > On May 15, 2013, at 4:07 PM, Daniel Holth <dholth at gmail.com> wrote:
> >
> >> Is there an API for "all the metadata for everything" that doesn't
> >> require one web request per package version? Maybe something like an
> >> rdiff-backup of a database?
> >> _______________________________________________
> >> Distutils-SIG maillist  -  Distutils-SIG at python.org
> >> http://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130516/7a06823f/attachment.html>