[Distutils] PEP 376 for Distutils

[P.J. Eby]

At 01:33 PM 2/24/2009 +0100, Tarek Ziadé wrote:
>Philip wrote:
> > So, the uninstallation code should simply not remove file(s) that 
> are referenced by more than one manifest in the target directory -- 
> a relatively simple, future-proof safeguard, that doesn't require 
> any specific knowledge of "namespace packages" per se.
> >
>
>Sounds good. Although, it requires scanning the files again which is
>not optimal, but the last point of this mail might be an idea to
>adress this.

Uninstallation isn't exactly a performance-critical activity.  It's 
trivial to read targetdir/*/RECORD and build up a dictionary of 
file->package and package->file links; this is Python, after all.  ;-)


>2009/2/24 Joachim König <him at online.de>:
> > An other option could be to put the egg-info dir into the package 
> itself, e.g.
> >
> > zlib/
> >   __init__.py
> >   egg-info/
> >       PKG-INFO
> >       MANIFEST
> >       RECORD
> >  ...
>
>This would require setuptools and pip to change the way they look for
>the packages,

More precisely, it would require pkg_resources to become ridiculously 
slow, by massively increasing the number of directories that need to 
be read at runtime to determine what packages are present.


>Indeed. Having an index file would make things a whole lot simpler.

For *whom*?  Certainly not for system packaging tools (rpm, deb, et al).

A design goal should be to allow system packaging tools to install a 
static file footprint: i.e., independent files with predefined 
content, and no post-processing steps.  You can't do that with a 
shared file, which is why setuptools uses a .pth hack to install 
namespace packages when building packages for rpm et al.


>I am wondering then if this is not an evolution of the .pth files.
>
>Although I find that having as many .pth file as we want is not robust.
>It make things slow down when you have too many of them.
>
>So, I'd be in favor of a new, unique, optimized, index file,
>with a set of functions located in pkgutil to read and write in it
>
>This index file could also index the namespace information,
>in order to speed up the work needed to uninstall a package that
>shares a namespace.

So, .pth files are bad...  let's make more?  In fact, let's make a 
new thing that'll have its own, new bugs?  So that we can have 
uninstalls that take only 1/10th of a second instead of 1/2 a second?

The standard to beat in this area, I believe, is PEP 262.  At the 
very least, if you're making any major changes in direction from that 
PEP, the rationale for those differences should be documented.  (And 
consolidated index files are explicitly rejected by that PEP, with 
good reason.)