[Python-Dev] How we can get rid of eggs for 2.6 and beyond

[Phillip J. Eby]

At 02:31 AM 3/22/2008 +0100, Martin v. Löwis wrote:
>>I'm making the assumption that the author(s) of PEP 262 had good 
>>reason for including what they did, rather than assuming that we 
>>should start the entire process over from scratch.
>
>The objections to the PEP remain the same as they were then,
>though: In the requirements, it says "we need", without saying
>why we need. It then goes on saying "we want" (rephrased)
>"to duplicate APT and RPM", without saying why we want that,
>or why that brings us closer to what we need.
>
>IOW, the PEP is lacking a rationale.

Ok, well, I have a rationale for it: make it possible to get rid of 
eggs as a mechanism for supporting easy_install.  Many people 
(yourself included) have criticized eggs as an installation 
mechanism, and this is an alternative that gets rid of .egg files and 
directories in that case, and most of the need for .pth file usage as well.


>If there was a chance that the infrastructure being developed
>actually helps these tools, *that* would be a reasonable goal,
>IMO.

Yes, I'm of course primarily interested in Python-specific tools such 
as virtualenv, easy_install, buildout, and the as-yet-unwritten 
uninstallers, package listers, etc., that can usefully read or write such data.


>However, I'm extremely skeptical that this can ever succeed
>to the degree that whoever provides RPMs, .debs, or MSI
>files will actually use such data, as they will find that
>the data are incomplete, and they have to redo all of it,
>anyway.

The data isn't for them to use to meet their use cases, it's for them 
to *provide* so that Python tools don't stomp on, uninstall, or 
otherwise interfere with files installed by the system.  In other 
words, for system packagers, it's a communication from the system to 
Python, rather than the other way around.  Even though the distutils 
will build the file in the bdist, the system packaging tools would be 
free to generate their own file listing and signatures and such.


>Do you also envision the objective of PEP 262, then? I.e.
>to provide a database of installed packages, in .../install-db?

In each directory relative to a given sys.path directory, yes.  That 
is, installing a distutils distribution to any directory would result 
in a file being added to an install-db within that 
directory.  (Assuming we use the proposed implementation model of PEP 
262, which at the moment I don't see any substantial obstacle to.)



>>And as I said, I'll be happy if all we do is get the distutils to 
>>abide by the spec for 2.6, even if it means we don't get an 
>>uninstall tool.  That can always be installed later using Guido's 
>>bootstrap tool.  :)
>
>I'm even more skeptical here. If the assumption is that the package
>database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI
>should then *not* write to that package database, as they also write
>to a different database, out of the scope of the PEP, and this is
>what uninstallation should use.

I probably should have brought this up, in fact, I think I mentioned 
it in a previous thread, but I would like to see PEP 262 add a way to 
say "this is a system-installed package, *don't touch*".  The idea 
again is not to do the job of the native packaging system, but rather 
to ensure that Python-specific tools (e.g. easy_install and friends) 
do not interfere or conflict with it.

A big problem in the early development of easy_install, even using 
eggs, was that there was no way to tell whether it was safe to delete 
or overwrite an existing file or directory that was already installed 
on the system.  A mechanism like this would allow tools like 
easy_install to say, "oh, your system packager has a conflicting 
package here, you need to use that tool to sort this out if you 
really want to do something here.  I'm not going to touch 
that."  Without something like this, there is no way to tell the 
difference on many systems between a system package and something the 
user has put there with "sudo python setup.py install".