[Distutils] Specification for a local PyPI simple index

[PJ Eby]

On Mon, Sep 9, 2013 at 10:54 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> Is the spec at http://peak.telecommunity.com/DevCenter/EasyInstall#package-index-api
> still the definitive version of what must be provided for a local PyPI
> index (for use by something like pip)? Or is there a more up to date
> version anywhere?

That spec is for setuptools 0.6.  Later versions should have changed
this documentation (in the PyPA repository) if they changed the
protocol, but I don't know if anybody's actually keeping tabs on that.

> For example, are MD5 signatures still the only supported version? I
> thought we were moving away from MD5.

Hm.  Yeah, a quick glance at https://bitbucket.org/pypa/setuptools/src
shows the docs unchanged, so whoever added the non-MD5 support forgot
to check the docs for references to md5.


> And while I haven't really
> followed the offsite hosting changes, are the
> rel="homepage"/rel="download" links still as stated? (I think I'd want
> rel="download" on everything as I only expect to provide URLs for
> actual package content).

The meaning of re="downloadl" values is, "spider this page for
download links", not "this is a link to download".    Links to
download are identified by inspecting a link, not retrieving it.  The
only reason the rel tags exist is to mark URLs as spiderable.


> Also, how definitive is item 7, which states that the root URL must
> result in a page containing all projects, but it can be omitted if
> case-insensitive safe_name() matching of projects is implemented?

It's definitive for easy_install.  The only reason easy_install
retrieves the root URL is if a requested package isn't found; the
reason it does this is to catch alternative spellings due to
case-sensitivity and/or differences in punctuation folding.  If you
can interpret easy_install's initial GET as a package requirement
string (w/case- and punctuation-insensitivity via
pkg_resources.safe_name()) rather than as an exact string match, then
failure to match would produce the same failure to match on a full
package listing, so there's no point having the full listing appear.
It's strictly a fallback intended for "dumb" package indexes that
simply consist of a directory tree and a web server providing
directory listings.  (I think it can even work with an FTP site, but
it's been a while since I worked on that code.)

> The
> reason I ask is that providing a full listing will be somewhat costly
> in my application, but providing case-insensitive matching should be
> doable (actually, I'm not sure yet what's feasible, but I want to know
> whether it's worth my time even investigating).

I don't know what pip does, but I assume that it's probably true of
all package managers that either their targeted request succeeds or
fails, and then they either request the full listing or they don't.
So...  the only possible way not providing the full list would be if
some (foolish) package manager always began by requesting a full
package listing.

It's possible there are tools that wish to obtain a full listing and
use the base URL for that...  but AFAICT it would be a foolish thing
to do if you're just trying to install packages.