[Distutils] Why I like eggs (or similar mechanisms) and my thoughts on future of buildout

[Wes Turner]

On Saturday, August 20, 2016, Nathaniel Smith <njs at pobox.com> wrote:

> On Aug 20, 2016 11:00 AM, "Jim Fulton" <jim at jimfulton.info
> <javascript:_e(%7B%7D,'cvml','jim at jimfulton.info');>> wrote:
> >
> [...]
> > With buildout, I chose to use eggs differently.  I simply generate
> scripts with dependencies explicitly listed in the Python path.  This is
> very easy to reason about.  It makes it easy to see what's installed by
> looking at the generated path.  This is similar to the way Java uses class
> paths to assemble jar files, which is fitting given that the design of eggs
> took at least some inspiration from jar files.
> >
> > I'm not a fan of Java, but I think class paths + jar files was something
> it got right.  Each program has it's own distinct environment.  If you need
> to add/remove/update a dependency, just change the path.  Want to share
> installed components between programs?  It's easy because only one
> installed component needs to be stored and different programs point to it
> via their paths.
>
> Wheels are a pretty simple and straightforward format. They've got some
> metadata, and then the are a set of directories with labels attached: "this
> directory needs to go on sys.path", "this directory has scripts that should
> have shebang fixups applied and then go on $PATH", etc. (The spec is not
> long: https://www.python.org/dev/peps/pep-0427/)
>
> My first guess at how buildout could handle wheels elegantly would be:
>
> - we have one piece of code that abstracts the operation of "please give
> me a wheel for package X version Y that works on this system". This then
> contacts pypi, builds sdists if it has to, whatever. It could be in the
> form of an invocation of pip, or it could be a call into some common
> library that gets factored out of pip and used by both pip and buildout,
> whatever.
>
> - pip uses this code to get a wheel, and once it has a wheel, it installs
> it into an environment in the same way it does today.
>
> - buildout *also* uses this code to get a wheel, and once it has a wheel,
> it does its own thing: maybe its own dependency resolution, and definitely
> its own install strategy, where it unpacks the wheel's directories into its
> own preferred layout, does sys.path munging on the wheel's scripts, etc.
>
> AFAICT there's nothing in the wheel format that ties you to the
> virtualenv-style directory layout.
>
> The disadvantage is that someone would have to do the work to factor out
> that get-a-wheel code, and to write/maintain custom wheel installation code
> in buildout, instead of just invoking setuptools. The advantage would be
> the now you aren't dependent on setuptools...
>
> > The buildout developers have discussed options for the future. We know
> there's a reckoning coming, but so far, thankfully, we've been able to put
> it off, but we don't want to be a burden on the rest of the Python
> community. (Seriously, thank you for not breaking us. :) )
>

I may be optimistic, but should adding buildout support for wheel be more
complicated than shelling out to pip with the correct, uhm, --prefix etc?

... recipes for buildout & pip, omelette symlinks:
http://www.mail-archive.com/distutils-sig@python.org/msg24964.html
Re: [Distutils] Future of setuptools and buildout
<http://www.mail-archive.com/search?l=distutils-sig@python.org&q=subject:%22Re%5C%3A+%5C%5BDistutils%5C%5D+Future+of+setuptools+and+buildout%22&o=newest>

Re: buildout and pip and wheel

"Add support for installing
wheels"https://github.com/buildout/buildout/issues/144






> As a heads up, there is unfortunately another source of breakage heading
> down the pipe: the PEP 516/517 plans for making it easy to use
> non-distutils-based build systems are heavily tied to wheels, so if/when
> the spec gets finished and projects start adopting alternative build
> systems, you may see an increasing proliferation of projects that cannot
> build eggs or be easy_install'ed.
>
> (Ref: https://www.python.org/dev/peps/pep-0517/)
>
> It's not clear when this will happen exactly -- I'm in the midst of
> dealing with some medical stuff, so haven't had much/any time for
> distutils-sig stuff recently, but sooner or later this is the direction we
> want to go.
>
> -n
>

So these will all still install with pip?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160820/d956e612/attachment-0001.html>