[Distutils] Re: cleaning up packages

[Randall Hopper]

Thanks for the reply.

Greg Ward:
 |Randall Hopper:
 |>      With DistUtils, what's the command to clean up all the kibbles and
 |> bits in the Python tree associated with a package of a given name?
...
 |Punted on for now

Ok.

 |the immediate concern is building, distributing, and installing Python
 |modules and applications, and -- even more important -- generating
 |"smart" installers that do all of the above work (uninstall,
 |list-what's-installed, etc.) for us.

I'm a little confused here.  Your latter points (uninstall, list what's
installed) is what I'm referring to, and if I interpret correctly, you list
this in the "immediate concerns" category.

 |I have no huge desire to reinvent the RPM wheel.  The guys at Red Hat
 |did a great job, and from what I've heard the Debian and Free BSD folks
 |have done a little better still.  However, the sad truth is that most
 |commercial Unices are saddled with abysmally awful package management
 |systems, and hardly any free software developers bother to support, eg.,
 |Sun's pkgtool (or whatever it's called).

Right.  SGI has it's own as well.

 |Is it worth expending effort to support what is obviously a dying breed?
 |In the Unix world, Linux and *BSD are clearly the way of the future; all
 |of the commercial Unices save Solaris are gasping for breath, ...

Here's why I think it is worth the effort.

Clearly folks "must" deinstall old versions of module distributions before
installing new ones.  Neglecting this task risks picking up old,
incompatible bits from prior versions that may render the module
distribution unusable.  Case in point: Numeric changing it's include and
lib directory names every other version (numeric/multiarray.so vs.
Numeric/multiarray.so), or a .so or .py module that gets moved between
submodules in a package.

Some deinstaller is needed, whether OS-supported or Python-supported.
(<grin>Either that, or we adopt the MSWin strategy of users completely
reinstalling Python when the blue-screen frequency goes high. :-)

And here is why I think the OS-supported package managers won't work for
the Python world.

We have two important classes of Python users that install module
distributions:

  1) Those with administration privilege      (sysadmins, home users)
  2) Those without                            (commercial/government users)

For those with admin privilege, the FreeBSD, Redhat, Debian, Sun, SGI,
MSWin, <<insert-favorite-OS-here>> package managers for the respective OS
may be just fine.  We've got the headache of supporting N different ways
for folks to deinstall packages from their Python tree based on the OS
(assuming someone will support building dists for all N) which'll doubtless
become an FAQ, but it works.  For folks to build these OS-specific
packages, we've also got to figure out N different times how each version
of each module dist sprinkles files throughout the Python tree (when
Distutils could tell us that).  And for those users on OSs without package
managers, they'll have to cobble together their own ad-hoc solution to keep
track of how to clean up module dists.

But the real problem is for users wiithout admin privilege (frequent in the
commercial and government worlds).  For them, the system-provided package
manager isn't a realistic option.  Assuming these users can find a
pre-built package for their OS (SGI, Sun, etc.), and that the local admin
compiled/installed Python to live in the same absolute path as the module
distribution wants to live in, these users have to get their administrator
to install it.  This can be a time-consuming task (or neigh impossible in
some cases! -- think Dilbert sysadmins and government beaurocracy).  More
than likely, these developers have their own writable Python tree installed
locally off their $HOME so they can do their own install/uninstall and
actually get work done.

These non-admin users will always be going the "setup.py install" route and
building module dists themselves.  They will be upgrading, and the system
package manager is not going to help them track and clean up old versions.

I may be underestimating, but from what's already provided to Distutils in
the setup.py files (name, version), it looks like a simple "keep a
files-per-module-dist" list doesn't appear that it'd be a whole lot of work:


        <module-dist-name>-<version> [TAB] <module-filename>
        <module-dist-name>-<version> [TAB] <module-filename>  
        <module-dist-name>-<version> [TAB] <module-filename>
        ...

and in Python pseudocode, the module deletion is just:

        while read line: if line[0] == THE_MODULE: os.remove( line[1] )

I share your concern about expanding the number of file formats supported
by DistUtils.  What about one of these many simple database packages that
Python already supports?  Just some simple text-file DB is all we need.
Just: "give me all the files for this package" | xargs rm -f.

If someone wanted to go all-out we could implement empty directory removal
and module dist dependencies by adding other columns to the table (text or
SQL DB table; doesn't matter), but IMHO this is a relatively low priority.
This simple "module dist deletor" doesn't need to be fast either.  If
there's a need, some budding Python hacker will come along later and
improve performance with much praise from the community.

-- 
Randall Hopper
aa8vb@yahoo.com