[Distutils] Future of setuptools and buildout

[Paul Moore]

On 16 August 2016 at 17:58, Leonardo Rochael Almeida
<leorochael at gmail.com> wrote:
> Questions:
>
> 1. Is the future outlined above considered likely?

I can't speak for setuptools, but certainly pip is intended to be the
canonical installer for Python. It's not impossible for competing
tools like the download/install features in setuptools to remain, but
like you say, I'd expect the trend to be against them.

> 2. If this future arrives, what package should buildout be ported to use?

That's really for buildout to decide.

> It should provide:
>
>  * The ability to locate packages and their versions, resolve their
> dependencies, and download them, whether they're sdists or wheels (or eggs,
> while we still have them)

pip will do this (no support for eggs exists or will be added, but
sdists and wheels yes). Note that pip should be run as a command line
tool - we don't support use of pip's internal API from an existing
Python process. If an in-process API is important, though, it wouldn't
be impossible to discuss adding that.

>  * Install each package separately into it's own directory that, if added to
> `sys.path`, allows this same package (or perhaps another one) to read
> information from entry points so that buildout can locate its own extensions
> and install console scripts.

That's not an installer feature. So pip doesn't provide anything
specific for that. Rather, that's a runtime support facility that is
tied to Python's import mechanisms. Setuptools (or rather,
pkg_resources?) may well continue to provide that, but laying out the
files present in a wheel in the format you need to support your
requirements would have to be added - maybe setuptools would be
interested in providing such a facility, I don't know. But it's not
exactly hard, at the most basic level you're probably only talking
about unzipping the wheel and writing a bit of metadata based on
what's specified in the wheel. Heck, a "wheel to egg" function
shouldn't be that hard to write, and then you could simply use the
existing runtime egg support code.

However, I don't believe that mandating that a single package should
provide both requirements is necessary, or helpful. The two features
are different - one of the features of setuptools that many people
(myself included) disliked, was the fact that it mixed "install time"
and "run time" features in one package, making it hard to avoid having
install time capabilities in a runtime-only environment.

Having said this, if someone were to propose, design and standardise a
*runtime* format for whatever you're hinting at as "install into its
own directory ... read information", and get it accepted as an
installation layout standard, then it might make sense to add support
to pip for installing files using that layout (analogous to the
"--target" flag for pip install). Runtime support would still have to
come from elsewhere, though. The design ideas for that spec may even
be based on the ideas developed for egg. But the key focus is moving
away from adhoc, implementation-defined standards, to properly
specified and documented standards. So step 1 would be to get such a
standard accepted, *then* look at proposing pip add it as an
alternative installation layout. (And I'd recommend not calling that
standard "egg" format - like it or not, the historical baggage
associated with that name will make it harder than you'd like to get
adoption of a new standard).

Hope this helps,
Paul