[Python-Dev] Packaging documentation and packaging.pypi API

[Carl Meyer]

Hi Alexis,

On 06/20/2012 10:57 AM, Alexis Métaireau wrote:
> Le mer. 20 juin 2012 18:45:23 CEST, Paul Moore a écrit :
>> Thanks - as you say, it's not so much the actual problem as the
>> principle of what the packaging API offers that matters here. Although
>> it does make a good point - to what extent do the packaging APIs draw
>> on existing experience like that of pip? Given that tools like pip are
>> used widely to address real requirements, it would seem foolish to
>> *not* draw on that experience in designing a stdlib API.
> 
> IIRC, pip relies nly onthe XML/RPC API to get information about the
> distributions from the cheeseshop. the code that's in packaging.pypi was
> built with the implementation in setuptools in mind, so we keep
> compatibility with setuptools "easy_install".

No, this is not accurate. Pip's PackageFinder uses setuptools-compatible
link-scraping, not the XMLRPC API, and it is the PackageFinder that is
used to actually find distributions to install. I think PackageFinder is
pretty much equivalent to what packaging.pypi is intended to do.

Pip does have a separate "search" command that uses the XMLRPC API -
this is the only part of pip that uses XMLRPC. I consider this a bug in
pip, because the results can be inconsistent with actual installation
using PackageFinder, and "search" can't be used with mirrors or private
indexes (unless they implement the XMLRPC API). The "search" command
should just use PackageFinder instead.

> That is, this leverages one question more on my side: was/is pip
> intended to be used as a library rather than as a tool / are there some
> people that are actually building tools on top of pip this way?

Pip's internal APIs are not documented, and they aren't the cleanest
APIs ever, but some of them (particularly PackageFinder and
InstallRequirement/RequirementSet) can be used without too much
difficulty, and some people are using them. Not a lot of people, I don't
think; I don't have hard numbers. I haven't seen much in the way of
public reusable tools built atop pip, but I've talked with a few people
building internal deployment tools that use pip as a library.

Carl