[Distutils] Are there any plans to move to pip/wheels in buildout?

[Jim Fulton]

On Tue, Dec 2, 2014 at 8:09 AM, Piotr Dobrogost
<p at lists-2014.dobrogost.net> wrote:
> On Tue, Dec 2, 2014 at 1:53 PM, Reinout van Rees <reinout at vanrees.org> wrote:
>>
>> The alternative is, like Marius said in his reply, to use virtualenv + pip +
>> makefiles or so. And you'd have to add something that replaces the recipes.
>> For that, I really like buildout :-)
>
> It seems like building/installing python packages should be moved from
> buildout's core into plain recipe so that different tools for this
> task could be used.

That's how buildout works today. There's nothing stopping someone from
building a recipe today that uses pip + virtualenv.

You might lose some features with this, like caching of built packages
but that may not be important to you. <shrug>  It looks like pip has
no way to cache builds as buildout does, possibly because it's
allergic to eggs. But it does support download caches. Yay!  And it
should be possible to "cache" builds in wheels, though I think this
would still be slower than what buildout does with eggs.  But you'd
probably gain some things, like a much shorter path to search at run
time.

Buildout still wants to download packages for it's own use.  I suppose
buildout could run in it's own virtualenv with recipes installed by
pip.

It remains that the API to pip would be via subprocess calls or
through a hideous distutils command interface.

> Even if this is possible I guess it's really major
> change (and paradigm shift to some extent).

There would be be no change needed to buildout to provide a recipe that
built using virtualenv and pip.

At some point, somethings going to have to change:

- As you noted, Python packaging is changing, and eventually
  the changes will break buildout.

  Many users, like myself, download source releases exclusively, and
  to us, wheels are irrelevent.  However, users on Windows or who use
  systems without build tools for some reason, will experience
  breakage as soon as people stop creating traditional binary releases
  in favor of wheels.

- Implementing and maintaining packaging code isn't really all that
  interesting to me and, I'd hazard to guess, to anyone else on the
  buildout team.  Buildout has more packaging code than I ever wanted
  to write.  It does largely because it needed policies and features
  not provided by easy_install. (This is not meant to disparage
  easy_install, which was a ground-breaking tool.)

  Buildout needs to build on something else. For the time being, it
  makes sense to continue to use setuptools, as this requires very
  little effort. Eventually, this will be something else and the
  transition is bound to be bumpy.

In the long run, I think keeping buildout alive with a manageble
effort is worth transition pain.  I think buildout offers a lot of
benefit for a lot of projects, and I'd hate to see people use
something worse.

Jim

P.S. I think it's a fallacy to assume that lack of activity or commits
     (buildout has had both recently) necessarily indicates that a
     project is dead. Lack of activity often just means that something
     doesn't require change.  I prefer to spread my work over many
     small projects and it's not uncommon for projects I use every day
     to go years without change.

     I realize that activity is an easy metric, especially when
     looking at projects with which you aren't familiar. I often use
     it myself, but it's important to keep it's limitations in mind.

--
Jim Fulton
http://www.linkedin.com/in/jimfulton