[Distutils] [venv] question about future pip capabilities

Ian Bicking ianb at colorstudy.com
Thu Dec 10 20:30:47 CET 2009

On Thu, Dec 10, 2009 at 11:18 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> Questions concerning pip:
> * Is there a roadmap or timeline concerning installing from windows
> exe or msi files? I didn't see anything at bitbucket.

There isn't a timeline, as we don't have anyone actively contributing
with respect to Windows.

> * Is there a roadmap or timeline concerning extras? I didn't see
> anything at bitbucket.

There isn't really; extras would be nice, but there hasn't been much
demand for them.  pip requirement files serve a similar purpose though
in a somewhat different way.

> * Could anyone please expand on the comment that pip is incompatible
> with some packages that customize distutils or setuptools in their
> setup.py files, and that it maybe doesn't work on windows?

Maybe the Windows statement is exaggerated.  Unfortunately we don't
have a good Windows testing setup, and I don't even know what that
would look like.  Some buildbot (or buildbot-like-thing, e.g., Hudson)
running on a Windows machine would be really helpful.

pip itself relies only on the command line and some expected behavior
of setup.py files, it doesn't really care how this is implemented.  To
install a package it does:

python -c 'import setuptools; __file__="setup.py"; execfile(__file__)'
install --single-version-externally-managed --record=/tmp/XXX

The first part installs the setuptools monkeypatching for packages
that just use distutils.  So you at least have to make something
compatible with that monkeypatching (i.e., it can't rely on setuptools
not being imported).  Then it uses those two command-line switches
(and I'm pretty sure that's it; you can see a complete record of what
pip does with pip install -vv).  Those are provided by Setuptools (or
Distribute) and not by Distutils, but pip doesn't care how they are
implemented so long as you accept those options.

Also to be compatible with pip you have to create a proper
Package.egg-info/ directory (to record that the package is installed)
and use that --record option to write a list of all the files that are
written to disk, so they can be removed when the package is removed.

> Questions for pip and Distribute:
> * How closely coupled are these projects?

They are not closely coupled at all.

> * I have a bit of time after hours, how can I help?

If you are a Windows developer, Windows help for pip would be great.
Installing Windows binaries would be the biggest help; I suspect this
means unzipping the .exe or .msi and moving the files into place
manually (since there is no setup.py), and maybe even generating that
.egg-info/ file.  I suspect that if you install a Windows installer
directly that pip (and Setuptools/Distribute/pkg_resources) might not
be able to see that the package is installed.  But I've never tried.
It would be nice if these installation techniques were compatible with
each other, which means the installer creating .egg-info files (or
shipping with EGG-INFO, similar to how Windows binary eggs work).

In theory Windows binary eggs could be installed similarly, but I
don't think there's as much enthusiasm for that among Windows users,
and .exe or .msi packages should be mostly equivalent (except that
lack of metadata).  But I don't consider myself a good judge of Window
users' preferences, so maybe I'm wrong.

A shorter time commitment is to download pip and run the tests on
Windows and report problems, as it's unfortunately quite possible
we've introduced Windows regressions.

Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker

More information about the Distutils-SIG mailing list