[Distutils] PEP 503 - Simple Repository API

[Wes Turner]

On Sep 5, 2015 11:12 AM, "Donald Stufft" <donald at stufft.io> wrote:
>
> On September 5, 2015 at 5:43:58 AM, M.-A. Lemburg (mal at egenix.com) wrote:
> >
> > Hmm, if the installer will build the URL itself, why is there even
> > a need for a top-level index page ?
> >
> > I mean for the occasional human reading the page it will certainly
> > make sense to have such a page, but for the API this doesn't
> > appear to be essentially needed.
> >
> > Or is the idea to have the package manager scan the index for package
> > hosted on that index prior to asking for the package it would like
> > to install ?
>
> The latest versions of pip won't use it, setuptools and older versions of
pip
> will use it though. The versions of pip/setuptools that would use it, use
it as
> a fallback. They don't pre-normalize the name before requesting the URL
so they
> just used whatever the user typed. This comes from when a project like
"Django"
> was at /simple/Django/ on PyPI but if a user typed ``pip install django``
it
> would first fetch /simple/django/ and if that 404'd it would fall back
onto
> /simple/ and look for these links. On PyPI this rarely happened because
PyPI
> redirects /simple/anything-that-normalizes-to-the-name/ to the correct
URL but
> it's useful for static repositories that don't have something to redirect
it
> in front.
>
> I've tried to make it so that all of the SHOULD and MUST directives can be
> implement by a standard Nginx/Apache/whatever web server with static files
> while maintaining compatability with older installers.
>
> >
> > Would it help the package manager to more easily detect the links
> > that point to distribution files instead of e.g. documentation or
> > other resources ?
> >
> > setuptools uses rel="download" for this:
> >
> > https://pythonhosted.org/setuptools/easy_install.html#package-index-api
>
> This is actually for the link spidering that PEP 470 removed, links
marked with
> either rel="download" or rel="homepage" would be fetched (unless they
looked
> installable) and searched for additional links before PEP 438/470 started
to
> deprecate/remove them. Both setuptools and pip only need a simple page
that has
> links that point to files on the, see for example the /simple/ page for
> requests: https://pypi.python.org/simple/requests/
>
> >
> > Could we perhaps also add optional features like:
> >
> > * Distribution link elements MAY include a data-gpg-sig=""
> > attribute to provide a GPG signature of the linked file
> >
> > This could later be extended to more meta data, such as platform
> > tags, distribution file types, license info, mirror locations,
> > documentation, help strings, etc.

RDFa and JSON-LD would make this graph constraint solution solving possible.

>
> I actually forgot to mention the GPG signatures, currently the assumption
is
> that if a GPG signature exists it will live at the same location as the
file
> with a .asc on the end, so if the file is /packages/Django-1.0.tar.gz
then the
> GPG signature will be located at /packages/Django-1.0.tar.gz.asc. I'll
add this
> to the PEP.
>
> I don't want to add more features to the API, particularly not in this
PEP. My
> longer term plan is to work on a a new API for installers to use which
will be
> easier to work with. The current API is great for it's simplicity and the
fact
> it can be implemented on the server side with nothing more than a
directory
> structure full of files and python -m http.server. The plan in my head is
to
> add a new repository API which can handle the more complex cases AND
which will
> most likely be JSON based to simplify parsing of it. The simple API would
not
> be deprecated, it would just be up to the repository which "version" of
the API
> they use. For people hosting their own repositories, if they have a
simple case
> they will be able to get away with the simple API, but the more complex
API
> would offer benefits like being able to access the metadata information
without
> downloading the file.
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150907/40f00600/attachment.html>