[Distutils] Mirroring PyPI JSON Locally

[Nick Coghlan]

On 21 August 2017 at 00:51, Wes Turner <wes.turner at gmail.com> wrote:
> IIUC, the task is still to:
> Download transitive portions of a [linked data] graph as JSON[LD] (optimally
> without iteratively downloading and decompressing package archives in order
> to retrieve their platform-dependent dependency edge metadata from a
> setup.py that is executed with filesystem privileges).

While I'm still generally negative on the idea of native reliance on
JSON-LD, I'll note one thing that has changed since I last looked at
it: I now see some potential concrete practical benefits to adopting
it, rather than purely theoretical ones. In particular,
https://github.com/scienceai/jsonld-vis now exists, and there wasn't
anything like that around at the time of previous discussions.

However, that's still only of potential interest for PEP 426, which in
turn still isn't needed for any of our practical near term objectives
(not even the "dependencies without downloads" one - if we were to
prioritise that, we'd likely go for something closer to the way client
side dependency resolution already works, such as extracting the
METADATA file from uploaded wheels and making it available for
download in addition to the full wheel archives).

So for this thread, Paul's right: a PEP-503-style document describing
the current PyPI JSON API would likely be a reasonable thing to write,
as it would allow for more complete emulations of the current
production PyPI service, including checking that Warehouse replicates
that part of the API correctly.

Anything more than that is still on the "No, not until some point
after Legacy PyPI has been shut down" list.

Cheers,
Nick.

P.S. Some of the tools mentioned at
http://www.seoskeptic.com/structured-data-markup-validation-testing-tools/
may also prove useful if we go down the JSON-LD path. However, that is
the only reason I'd ever support us going down that path: useful
functionality that we get for free by virtue of adopting an
established convention. I've wrangled volunteer contributors to open
source projects for long enough now to know that "because it's the
right thing to do" simply doesn't cut it as a motivational tool -
there's need to be some kind of actual benefit to the folks doing the
work :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia