Hi,
I keep forgetting in discussions that pip doesn't support eggs. Their main usecase is installing binary python eggs on windows, I'd say. I use linux and OSX myself, so my windows experience is limited.
Question: is pip's compile-from-source character a problem on windows? Does everyone use a clicky-click .exe installer for binary packages instead of pip? Is easy_install still more popular?
Note, I'm not talking about installing pure python packages, but more stuff like mapnik, pil, matplotlib and psycopg2: packages that have C bindings.
I'm asking because I'm writing a Django book and I don't want to get clobbered for advising pip when it won't install a mysql or postgres binding without major work. So I'd love some feedback from windows users!
(I'm mostly using buildout myself, but I'm only describing that near the end of my book).
Reinout
Hi,
If I were telling a new Windows Python user what to do to get started, I would tell them to use easy_install because easy_install will pick up both installers and binary eggs and do the right thing with those and virtualenvs.
pip on Windows is a decent into misery. Giving Windows users reliable instructions on how to set up an appropriate compiler is fraught with peril. Probably most new Python users on Windows have never done C or set up a build environment; they probably don't even know what a compiler is or why they would need one. Even if they do know about compilers, getting the right Visual Studio version(express or otherwise) is a bit of a problem because Python does not use the current version of VS and which version you need changes with Python versions. If you get a working compiler, then you need to track down the C dependencies of the module you are building.
The click installers are going to be a problem if you do any recommendation of virtualenv. The recent (or soon to be released) versions have --no-site-packages as the default and so, the click installers (that will install to global site-packages directory) won't show up without the user explicitly giving whatever the arg is for --with-site-packages when the virtualenv is created.
I heard rumours of plans for pip to support binary packages of some kind on Windows, but I don't know the details or current status of that.
-Chris
On Sun, Apr 15, 2012 at 7:50 PM, Reinout van Rees reinout@vanrees.orgwrote:
Hi,
I keep forgetting in discussions that pip doesn't support eggs. Their main usecase is installing binary python eggs on windows, I'd say. I use linux and OSX myself, so my windows experience is limited.
Question: is pip's compile-from-source character a problem on windows? Does everyone use a clicky-click .exe installer for binary packages instead of pip? Is easy_install still more popular?
Note, I'm not talking about installing pure python packages, but more stuff like mapnik, pil, matplotlib and psycopg2: packages that have C bindings.
I'm asking because I'm writing a Django book and I don't want to get clobbered for advising pip when it won't install a mysql or postgres binding without major work. So I'd love some feedback from windows users!
(I'm mostly using buildout myself, but I'm only describing that near the end of my book).
Reinout
-- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.**nl/http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
______________________________**_________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/**mailman/listinfo/distutils-sighttp://mail.python.org/mailman/listinfo/distutils-sig
Hi,
On Sun, Apr 15, 2012 at 8:07 PM, Chris Lambacher chris@kateandchris.net wrote:
Hi,
If I were telling a new Windows Python user what to do to get started, I would tell them to use easy_install because easy_install will pick up both installers and binary eggs and do the right thing with those and virtualenvs.
pip on Windows is a decent into misery.
Seconded; our scientific packages use numpy (C) and scipy (C, C++ Fortran, big) and have C and sometimes Fortran extensions, so pip is more or less useless for windows, and a real PITA for anything else.
Best,
Matthew
On 16-04-12 06:17, Matthew Brett wrote:
Hi,
On Sun, Apr 15, 2012 at 8:07 PM, Chris Lambacherchris@kateandchris.net wrote:
Hi,
If I were telling a new Windows Python user what to do to get started, I would tell them to use easy_install because easy_install will pick up both installers and binary eggs and do the right thing with those and virtualenvs.
pip on Windows is a decent into misery.
Seconded; our scientific packages use numpy (C) and scipy (C, C++ Fortran, big) and have C and sometimes Fortran extensions, so pip is more or less useless for windows, and a real PITA for anything else.
Thanks Chris and Matthew for the info! Especially about the upcoming no-site-packages default.
I'm really going to have to add a section on debugging site.path to my book... :-)
Reinout - happy to be using buildout ;-)
Hi Chris,
On 04/15/2012 09:07 PM, Chris Lambacher wrote:
If I were telling a new Windows Python user what to do to get started, I would tell them to use easy_install because easy_install will pick up both installers and binary eggs and do the right thing with those and virtualenvs.
Agreed.
pip on Windows is a decent into misery. Giving Windows users reliable instructions on how to set up an appropriate compiler is fraught with peril. Probably most new Python users on Windows have never done C or set up a build environment; they probably don't even know what a compiler is or why they would need one. Even if they do know about compilers, getting the right Visual Studio version(express or otherwise) is a bit of a problem because Python does not use the current version of VS and which version you need changes with Python versions. If you get a working compiler, then you need to track down the C dependencies of the module you are building.
The click installers are going to be a problem if you do any recommendation of virtualenv. The recent (or soon to be released) versions have --no-site-packages as the default and so, the click installers (that will install to global site-packages directory) won't show up without the user explicitly giving whatever the arg is for --with-site-packages when the virtualenv is created.
The argument is spelled --system-site-packages.
I heard rumours of plans for pip to support binary packages of some kind on Windows, but I don't know the details or current status of that.
Support for installing binary packages on Windows has been a "patches welcome" situation for as long as I've used pip (3-4 years?) - all that's lacking is someone motivated to provide a good patch :-)
Carl
On 16 April 2012 20:08, Carl Meyer carl@oddbird.net wrote:
I heard rumours of plans for pip to support binary packages of some kind on Windows, but I don't know the details or current status of that.
Support for installing binary packages on Windows has been a "patches welcome" situation for as long as I've used pip (3-4 years?) - all that's lacking is someone motivated to provide a good patch :-)
Speaking as someone who's looked at this, the problem seems to me to be that pip has a *lot* of baked in assumptions that it's working with source distributions, and it's very hard to see how to fit a binary install mechanism in place. It would be (relatively) easier to add a new "pip bininstall" subcommand, but once you start looking at that, it's not easy to see what it gives you over "easy_install".
Personally, I'm waiting to see where the Python 3.3 "packaging" solution fits into all this - as I understand it, it currently hits the same spot as pip (source-install only) but it's less mature. It may be that it will be easier to integrate binary installers into the packaging infrastructure but there's still a lot of debate and no real code in this area (I produced a patch, but it foundered over debates about whether the binary format I proposed was "the right way" to do it).
I'd love to see a group of people getting together to really work through the options for binary installers that work with pip/virtualenv/packaging, but the support seems pretty fragmented, and the fact that eggs are a "good enough" solution tends to stall any casual efforts in this direction.
Paul.
On 04/16/2012 03:56 PM, Paul Moore wrote:
On 16 April 2012 20:08, Carl Meyer carl@oddbird.net wrote:
Support for installing binary packages on Windows has been a "patches welcome" situation for as long as I've used pip (3-4 years?) - all that's lacking is someone motivated to provide a good patch :-)
Speaking as someone who's looked at this, the problem seems to me to be that pip has a *lot* of baked in assumptions that it's working with source distributions, and it's very hard to see how to fit a binary install mechanism in place. It would be (relatively) easier to add a new "pip bininstall" subcommand, but once you start looking at that, it's not easy to see what it gives you over "easy_install".
I'm happy to consult with anyone working on this. I think this particular framing conflates UI issues (what is the command named?) with internal code structure unnecessarily. It's true that much of pip's code is for working with sdists and support for installing binary packages would require some significant new and separate code paths internally (in InstallRequirement and, to a lesser extent, PackageFinder), but the actual user-facing commands (e.g. InstallCommand) are a relatively small part of the code; I don't think adding a separate command would actually save much work over extending the "install" command. But I could be wrong - were there specific issues that you saw as problematic for adding support to "install"?
Personally, I'm waiting to see where the Python 3.3 "packaging" solution fits into all this - as I understand it, it currently hits the same spot as pip (source-install only) but it's less mature. It may be that it will be easier to integrate binary installers into the packaging infrastructure but there's still a lot of debate and no real code in this area (I produced a patch, but it foundered over debates about whether the binary format I proposed was "the right way" to do it).
I'd love to see a group of people getting together to really work through the options for binary installers that work with pip/virtualenv/packaging, but the support seems pretty fragmented, and the fact that eggs are a "good enough" solution tends to stall any casual efforts in this direction.
Agreed that focusing on support in d2/packaging is probably a better use of time for the long term. I'm hopeful that at some point pip can lose a lot of code and become a much thinner wrapper around d2/packaging, with added support for things like installing direct from VCS, requirements files, etc.
Carl
On 16 April 2012 23:17, Carl Meyer carl@oddbird.net wrote:
I'm happy to consult with anyone working on this. I think this particular framing conflates UI issues (what is the command named?) with internal code structure unnecessarily. It's true that much of pip's code is for working with sdists and support for installing binary packages would require some significant new and separate code paths internally (in InstallRequirement and, to a lesser extent, PackageFinder), but the actual user-facing commands (e.g. InstallCommand) are a relatively small part of the code; I don't think adding a separate command would actually save much work over extending the "install" command. But I could be wrong - were there specific issues that you saw as problematic for adding support to "install"?
It was pretty much what you said - the InstallRequirement changes would be fairly extensive to factor out the reusable parts from the bits that only relate to source installs, and the finder changes (which I hadn't located, but knew would need to be addressed) needed to locate binary distributions as well as source ones. Also, pip reuses setuptools to build source packages, but because easy_install doesn't support --single-version-externally-managed, you can't do that for a binary, so you have to replicate all that logic within pip.
Thanks, for the offer, though - if the packaging initiative doesn't come up with anything concrete for 3.3, I might look at this again. Although as I mentioned, I find that in practice, easy_install is good enough for most cases where I need binaries, and hand-unpicking the packages works for the rest (MSI and custom installers).
Paul.
participants (5)
-
Carl Meyer -
Chris Lambacher -
Matthew Brett -
Paul Moore -
Reinout van Rees