[Distutils] What pip uses setuptools for

Ian Bicking ianb at colorstudy.com
Sat Mar 28 15:52:15 CET 2009

During the discussion at the open space about distutils, it was noted
several tools use Setuptools.  Right now I think we all agree
(probably PJE too) that Setuptools implements more than we can move
into the standard library.  Since these tools represent clear use
cases, figuring out how to get these tools to not depend on Setuptools
will, I think, factor out many of the most important APIs.  And it so
happens that pip uses just a few things, so factoring out this
functionality seems feasible:

I call setup.py egg_info.  This is currently awkward, because where
the egg-info information goes is hard to figure out.  But it works,
and gives me parseable metadata.  (Note, all calls to setup.py are
done in a subprocess; I think some tools call setup.py in-process, but
subprocesses seemed safer to me and I don't see much of a downside,
certainly not performance.)

I install using Setuptools, which installs the egg-info metadata, and
also records the files that were written.  (I write these to a
temporary location, rewrite the file paths, and put the list of files
into the package's egg-info).

I install using setup.py develop, using Setuptools.  This calls
egg_info, and also handles fixing up a .pth file.  I use --no-deps,
because I never want any of these installation processes to install

I've copied some code out of setuptools.command.egg_info for getting
the svn revision.  I'm fine maintaining this code (since there's also
code for other version control systems).

The uses of pkg_resources:

I look around at all the packages' dependency_links.txt, as this gives
pip a hint of where things came from, and this lets me better
implement pip freeze.  Some people hate dependency_links, but I don't
at all.

I get a list of all the installed packages (with versions), for pip
freeze.  It is very helpful that people use -r1234 for svn checkouts,
as I detect this and use this to create sensible frozen versions.
Handling version numbers for unreleased packages is important.

I use pkg_resources to parse versions, test for which is better, and
test for inclusion (whether a version satisfies a requirement).  Oh,
and I parse requirements.  I would like if there was a distinction
between stable and unstable versions included in the version

I have some code to normalize project names.  The code is awkward, and
kind of uses pkg_resources.  I'd like this handled more nicely.  I
think Setutools' case insensitivity is useful, as is some other forms
of normalization.

More information about the Distutils-SIG mailing list