[Distutils] Specification for a local PyPI simple index

[Donald Stufft]

On Sep 9, 2013, at 11:15 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 9 September 2013 16:11, Donald Stufft <donald at stufft.io> wrote:
>> It might match what easy_install looks for but it's not up to date. I think it's
>> still written as if the simple index had not been created. That being said
>> it'll probably mostly work. One of my steps I want to do with warehouse
>> is actually get this codified.
> 
> Sigh. That's what I thought :-(
> 
> Particularly as I don't actually want it to work with easy_install,
> but rather with pip. OK, I guess I'll just have to go with what that
> page and reverse-engineer from there. Warehouse (or at least something
> with an actual *spec*) can't come too soon for me...
> 
> Thanks for the clarifications on the other points, too.
> 
> Paul


It's not that complicated.

Assuming an --index-url of https://pypi.python.org/simple/

``pip install foo`` will look at

1. https://pypi.python.org/simple/foo/
2. https://pypi.python.org/simple/

``pip install foo==1.0`` will look at

1. https://pypi.python.org/simple/foo/1.0/
2. https://pypi.python.org/simple/foo/
3. https://pypi.python.org/simple/

It will stop once it finds a page that doesn't error or 404 (so typically it only looks at /foo/).

Once it has a page it looks for html links and essentially categorizes them into "installable" and "not installable". It's installable if it "looks" like a package url (the url looks like it's a file that matches the naming scheme). Then it looks at any rel = "download" or rel = "homepage" and if they are not installable links it will also fetch their HTML and do the same process of looking for "installable" links.

Once it has a list of all the installable links for a page it selects the highest version one matching the spec we have and installs it.

If there is a meta tag for api version and that version is >= 2 then that triggers the new external stuff which is opt in for pip 1.4 and opt out in pip 1.5. This allows an index server to specify "this file is hosted by me" by adding a rel="internal" to links to self hosted files. In pip 1.5 (and opt in for 1.4) pip will only install internal files (for servers with an api version metatag) unless the proper flags were passed to the command.

There is a gotcha where setuptools (and pip still, though I want to deprecate it) also looks for a an extract string that looks like http://peak.telecommunity.com/DevCenter/EasyInstall#backward-compatibility which can be inside HTML comments. 

Does that help?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130909/53784e94/attachment.sig>