[Distutils] pip on windows experience

[Paul Moore]

On 23 January 2014 11:48, Thomas Heller <theller at ctypes.org> wrote:
> After having used and built bdist_wininst installers for years (well, I
> should say decades) I have recently started playing with pip.

Thanks for trying this - it's really useful to have a newcomer's
perspective to the pip experience.

> However, does it work in real life?
>
> There are no wheels for important packages like pywin32, numpy, lxml,
> at least not on PyPi.  'pip install' using the source packages fails to
> build for most but the simplest stuff; probably because distutils has
> been extended too much, or because header files or libraries are
> missing, or whatever...

This is the key point - pip install from source fails on Windows for
pretty much any package with C code and non-trivial dependencies.
Building any of the above packages is murder on Windows, and it's 100%
critical to the user experience that you shouldn't have to.

> Is there a solution to this?  I've seen that the wheel tool can convert
> bdist_wininst installers into wheels - does this work for the packages
> I mentioned above?  Do we have to build or convert to wheel those
> packages, and setup a central place where we store them and make them
> available to our developers?

The good news is that "wheel convert XXX.exe" works on all the above
for creating a wheel from the wininst installer. The official numpy
installer uses some complex magic to select the right binaries based
on your CPU, and this means that the official numpy "superpack"
wininst files don't convert (at least I don't think they do, it's a
while since I tried). But happily, Christoph Gohlke hosts a huge list
of readymade wininst installers for hard-to-build projects, and the 3
you mention are all there. He's very good about building for latest
Pythons, too (3.4 is already there for many packages). Anyone working
on Windows who doesn't know his site
(http://www.lfd.uci.edu/~gohlke/pythonlibs/) should check it out.

So, to summarise, yes, you can get wheels for pretty much everything
you need by using wheel convert on wininst installers. You do need to
manually download, run wheel convert, and host the wheels locally (a
simple directory is fine though).

I'd love to see some or all of these projects host wheels themselves
on PyPI, or someone like Christoph host a pypi-style index of wheels,
so we could just point pip at that index as well as PyPI (note: I
don't know if pip works when a package is split across 2 index sites,
that's something I should check, but if it doesn't, I'd like to see
that added). But that's in the future for now - we probably need more
uptake of wheels before the demand becomes sufficient to persuade
anyone to do this.

> BTW: It seems the pip 1.4.1 doesn't download the wheel built with the
> most recent setuptools/wheel version, instead it downloads and tries
> to build from the source package.

Pip 1.4 doesn't use wheels by default - you should add "--use-wheel"
to the install command (or put it in your pip.ini). "--use-wheel is
the default from pip 1.5 onwards.

Hope this helps,
Paul.