[Distutils] Are there any plans to move to pip/wheels in buildout?
jim at zope.com
Wed Dec 3 15:01:32 CET 2014
On Tue, Dec 2, 2014 at 8:09 AM, Piotr Dobrogost
<p at lists-2014.dobrogost.net> wrote:
> On Tue, Dec 2, 2014 at 1:53 PM, Reinout van Rees <reinout at vanrees.org> wrote:
>> The alternative is, like Marius said in his reply, to use virtualenv + pip +
>> makefiles or so. And you'd have to add something that replaces the recipes.
>> For that, I really like buildout :-)
> It seems like building/installing python packages should be moved from
> buildout's core into plain recipe so that different tools for this
> task could be used.
That's how buildout works today. There's nothing stopping someone from
building a recipe today that uses pip + virtualenv.
You might lose some features with this, like caching of built packages
but that may not be important to you. <shrug> It looks like pip has
no way to cache builds as buildout does, possibly because it's
allergic to eggs. But it does support download caches. Yay! And it
should be possible to "cache" builds in wheels, though I think this
would still be slower than what buildout does with eggs. But you'd
probably gain some things, like a much shorter path to search at run
Buildout still wants to download packages for it's own use. I suppose
buildout could run in it's own virtualenv with recipes installed by
It remains that the API to pip would be via subprocess calls or
through a hideous distutils command interface.
> Even if this is possible I guess it's really major
> change (and paradigm shift to some extent).
There would be be no change needed to buildout to provide a recipe that
built using virtualenv and pip.
At some point, somethings going to have to change:
- As you noted, Python packaging is changing, and eventually
the changes will break buildout.
Many users, like myself, download source releases exclusively, and
to us, wheels are irrelevent. However, users on Windows or who use
systems without build tools for some reason, will experience
breakage as soon as people stop creating traditional binary releases
in favor of wheels.
- Implementing and maintaining packaging code isn't really all that
interesting to me and, I'd hazard to guess, to anyone else on the
buildout team. Buildout has more packaging code than I ever wanted
to write. It does largely because it needed policies and features
not provided by easy_install. (This is not meant to disparage
easy_install, which was a ground-breaking tool.)
Buildout needs to build on something else. For the time being, it
makes sense to continue to use setuptools, as this requires very
little effort. Eventually, this will be something else and the
transition is bound to be bumpy.
In the long run, I think keeping buildout alive with a manageble
effort is worth transition pain. I think buildout offers a lot of
benefit for a lot of projects, and I'd hate to see people use
P.S. I think it's a fallacy to assume that lack of activity or commits
(buildout has had both recently) necessarily indicates that a
project is dead. Lack of activity often just means that something
doesn't require change. I prefer to spread my work over many
small projects and it's not uncommon for projects I use every day
to go years without change.
I realize that activity is an easy metric, especially when
looking at projects with which you aren't familiar. I often use
it myself, but it's important to keep it's limitations in mind.
More information about the Distutils-SIG