[Distutils] [Python-Dev] How we can get rid of eggs for 2.6 and beyond

[M.-A. Lemburg]

On 2008-03-21 14:47, Phillip J. Eby wrote:
> So, to accomplish this, we (for some value of "we") need to:
> 
> 1. Hash out consensus around what changes or enhancements are needed 
> to PEP 262, to resolve the previously-listed open issues, those that 
> have come up since (namespace packages, dependency specifications, 
> canonical name/version forms), and anything else that comes up.
> 
> 2. Update or replace the implementation as appropriate, and modify 
> the distutils to support it in Python 2.6 and beyond.  And "support 
> it" means, "ensure that 'install' and *all* bdist commands update the 
> database".  The bdist_rpm, bdist_wininst, and bdist_msi commands, 
> even bdist_dumb.  (This should probably also include the add/remove 
> programs stuff in the Windows case.)

The bdist commands don't need to touch that database in any way,
since they don't install anything, nor do they upload things
anywhere. They simply package code and put the result into
the dist/ subdir. That's all.

What you probably mean is that the installers, pre/post-scripts,
etc. run when installing one of those packages should update
the database of installed packages.

Note that there are several package formats which do not execute
any code when installing them - the user simply unzips them in
some directory. These packages won't be able to register themselves
with a database.

I guess the only way to support all of these variants is
to use a filesystem based approach, e.g. by placing a file
with a special extension into some dir on sys.path.
The "database" logic could then scan sys.path for these
files, read the data and provide an interface to it.

All bdist formats would then have to include these files.

distutils already writes .egg-info files when running
python setup.py install, so perhaps that's a start (though
I'd prefer a three letter extension such as ".pkg").

.egg-info files currently only include the package meta-data
(the PKG-INFO section from PEP 262).

We'd have to add a list of files making up the package (FILES
section in PEP 262) and also some extra information about any
extra files the package creates that can safely be removed in
the uninstall process (e.g. .pyo and .pyc files, temporary files,
database files, configuration data, registry entries, etc.) -
this is currently not covered in PEP 262.

I don't think the REQUIRES and PROVIDES sections from the
PEP 262 are needed. That info can easily go into the PKG-INFO
section.

A separate FILES section also doesn't seem to be necessary -
we could just add one or more entries or the format:

CreatesDir abc/
CreatesFile abc/xyz1.py
CreatesDir abc/def/
CreatesFile abc/def/xyz2.py
CreatesFile abc/def/xyz3.py
CreatesFile abc/def/xyz4.ini

(BTW: wininst writes such a file for the uninstall process)

So to keep things simple, the rfc822 approach defined in
PEP 241 would easily cover everything needed and we could
trim down the PEP 262 format to a simple rfc822 header
list.

In other words: the .egg-info files already provide the basis
and only need to be extended with a list of created files,
directories (and possibly other resources) as well as a list
of resources which may be removed even if not installed
explicitly such as byte-code files, etc.

> 3. Create a document for system packagers referencing the PEP and 
> introducing them to what/why/how of the standard, in case they 
> weren't one of the original participants in creating this.

This should probably be a new PEP defining all the bits and
pieces making up the installation "database".

> It will probably take some non-trivial work to do all this for Python 
> 2.6, but it's probably possible, if we start now.  I don't think it's 
> critical to have an uninstall tool distributed with 2.6, as long as 
> there's a reasonable way to bootstrap its installation later.

BTW: There's a simple uninstall command in mxSetup.py that we
could contribute to distutils. It works much in the same
way as the install command... except that it removes all the
files it would have installed.

Using pre-built packages, this works without having to rebuild
the package just to be able to determine the list of things
that need to be removed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 21 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611