[Distutils] [Python-Dev] How we can get rid of eggs for 2.6 and beyond

[Alexander Michael]

On Sat, Mar 22, 2008 at 10:02 AM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> > It seems to me that this discussion is being undermined by not
>  > acknowledging the many use cases up front. There is no rationale
>  > because there are too many tacit rationales.
>
>  I honestly, really, cannot imagine what those are. Explicit is
>  better than implicit.

Exactly. I was attempting to suggest that we be explicit about "the
problem" being solved so it was easier for everyone interested to
verify that their "needs" (i.e. use case) will be met and that the
proposed solution solves the problem. Since the topic of the thread is
how to get rid of eggs in 2.6, the question (to me, at least) is how
to enable the functionality provided by setuptools to be provided
outside the standard library in a way that does not create pressure
for the entire python community to be assimilated into egg-ified
zombies.

Why do we have the current pressure for egg-ification? The main
pressure appears to arise from a strong desire by the python community
for a simple installation tool like CPAN that downloads and installs
the package of interest *as well as* any dependencies you don't
already have installed. Of course, it is the desire for the
dependencies to be discovered, downloaded, and installed in a manner
that honors the current state of your python environment that creates
all of the problems.

My participation may be unwanted or unwarranted due to my lack of
education/understanding, but I like to live dangerously in the hopes
of actually being helpful. With that preamble, here's my attempt at an
explicit rationale for a database of installed packages (A.K.A. The
New PEP 262):

"""
Rationale
=========
It is often necessary during the course of managing a python
installation for an administrator to determine the following things:

1. If the current installation state satisfies the requirements of a
new package being considered for installation.
2. If it is safe for the administrator to upgrade or remove a package,
and if so, how (e.g. use a system-level package management tool).
3. What files to remove in order to uninstall the package (if a
system-level package management tool was not used to install it).
4. If the current installation was modified in-place or custom
configured in an way, so that such changes can be noted before
upgrading.

Furthermore, many administrators want to do as much of this as
possible with automated tools, without manually inspecting README
files, INSTALL files, or the installed code itself to determine the
list of dependencies and installed versions, so there is a desire to
be able to make the above determinations programmatically.

Current efforts to provide these capabilities without standard library
support have resulted in many users being forced to use non-standard
package management tools because other users desired these
capabilities. This proposal is motivated by a desire to provide the
minimum required infrastructure so that both segments of the python
community can peacefully coexist without getting in each others way
(i.e. the ability to "opt in" to python-based non-system-level package
managers).

Proposal
========
The proposal is to provide in the standard library the following capabilities:

1. List the installed packages, along with the version and dependency
list for each package.
2. Query the ownership of a currently installed package (standard
library, system-level package management tool, etc.).
3. List the files installed be specific package.
4. Recall the original message digest for each installed file to
determine if the file has been modified.

...

Alternatives
============
There are at least three alternatives to providing a database of
installed packages that could also potentially enable administrators
to accomplish the same ultimate goal of installing new packages and
their dependencies without breaking or interfering with the "system"
installation.

Switchable Environments
-----------------------
Eliminate the need to be careful by providing a mechanism for the
creation of and installation into what amounts to user-selectable
site-packages directories that each "inherit" from what is currently
called site-packages (something like an eggless virtualenv with a
python comannd-line option to choose the environment). Need to upgrade
a package provided by the standard library or your system-level
package manager? No Problem! Of course, this won't help you manage
your sand-boxed environment, but you can have lots of them for very
little cost (it's not a whole new python installation) so just create
a new one each time you want to change something.

Ubiquitous System Packages
--------------------------
Eliminate the need to *not* use your OS's system-level package manager
by creating the necessary infrastructure so that python packages can
easily be distributed in the major system-level packager formats (RPM,
DEB, MSI, MPKG, etc.) for all publicly released python packages. A
windows-based developer who releases their pure-python package should
be able to create RPMs, DEBs, etc. automatically (using distutils, for
instance) without access to a computer running the appropriate OS
(python setup.py bdist_all upload).

Remote Hosted Installation Database
-----------------------------------
Eliminate the requirement that your installation be self-describing by
maintaining the information in PyPI necessary for one to figure out
what versions of what things are installed in your system (either by
recipes or a table of message digest values for the directories) in
addition the dependencies and list of installed files for each version
of each package thats ever been released.

"""

Regards,
Alex