Ronald Oussoren wrote:

On 24 Feb, 2009, at 16:20, P.J. Eby wrote:

...

...

Indeed. Having an index file would make things a whole lot simpler.

For *whom*? Certainly not for system packaging tools (rpm, deb, et al).

A design goal should be to allow system packaging tools to install a static file footprint: i.e., independent files with predefined content, and no post-processing steps. You can't do that with a shared file, which is why setuptools uses a .pth hack to install namespace packages when building packages for rpm et al.

What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip.

Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files.

But that is already implemented via file/dir permissions. By your reasoning, we should also have something which warns users not to install to the system directory. These ideas are a duplication of functionality -- this functionality is implemented by the disabling write permissions of non-sysadmins into system directories. Or do you propose users put some stuff into their system directories not managed by their package managers and other stuff managed by the package managers?

At 02:57 PM 2/24/2009 -0500, P.J. Eby wrote:

At 04:45 PM 2/24/2009 +0100, Ronald Oussoren wrote:

...

What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip.

Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files.

The absence of a RECORD file could suffice for that, I think. Any file that isn't positively listed in *exactly* one RECORD file shouldn't be removed.

I'm not super-set on that position, just noting that the absence of a listing is sufficient to detect something that "shouldn't" (as opposed to "mustn't") be deleted. The only contradicting use case is the deletion of packages that were installed via an older distutils version.

Hm. This is also relevant for *install* operations... which should not overwrite (without warning) a file that's not listed in any RECORD files, even if it has the exact same content. (As that would mean a later uninstall would silently delete it.)

On Tue, Feb 24, 2009 at 11:48:13AM -0800, Andrew Straw wrote:

Ronald Oussoren wrote:

...

What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip.

Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files.

But that is already implemented via file/dir permissions.

Not all systems are managed by experienced sysadmins and on many single user systems `sudo' is all to easy to invoke.

By your reasoning, we should also have something which warns users not to install to the system directory. These ideas are a duplication of functionality -- this functionality is implemented by the disabling write permissions of non-sysadmins into system directories.

As pointed out by PJE simply removing the RECORD file should do the trick.

Or do you propose users put some stuff into their system directories not managed by their package managers and other stuff managed by the package managers?

It's interesting to point out what seems to be planned for Debian: http://lists.debian.org/debian-devel/2009/02/msg00431.html Quoting just the relevant part: """ Local installation path ----------------------- When installing Python modules using distutils, the resulting files end up in the same location wether they are installed by a Debian package, or by a local user or administrator, unless the installation path is overwritten on the command line. Compare this with most software based on autoconf, where an explicit prefix has to be provided for the packaging, while the default install installs into /usr/local. For new Python versions packaged in Debian this will change so that an installation into /usr (not /usr/local) requires an extra option to distutils install command (--install-layout=deb). To avoid breaking the packaging of existing code the distutils install command for 2.4 and 2.5 will just accept this option and ignore it. For the majority of packages we won't see changes in the packaging, provided that the python packaging helpers can find the files in the right location and move it to the expected target path. A second issue raised by developers was the clash of modules and extensions installed by a local python installation (with default prefix /usr/local) with the modules provided by Debian packages (/usr/local/lib/pythonX.Y/site-packages shared by the patched "system" python and the locally installed python. To avoid this clash the directory `site-packages' should be renamed to `dist-packages' in both locations: - /usr/lib/pythonX.Y/dist-packages (installation location for code packaged for Debian) - /usr/local/lib/pythonX.Y/dist-packages (installation location for locally installed code using distutils install without options). The path /usr/lib/pythonX.Y/site-packages is not found on sys.path anymore. About the name: Discussed this with Barry Warsaw and Martin v. Loewis, and we came to the conclusion that using the same directory name for both locations would be the most consistent way. This change should make the request to conditionalize the inclusion of /usr/local/lib/pythonX.Y/site-packages into sys.path obsolete. If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages pointing to dist-packages. """ This should address the concern of sysadmins making mistakes when adding and removing python distributions (packages/modules). In other words, maybe it's not python's problem but the OS distribution's problem. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

2009/2/25 P.J. Eby :

At 10:03 PM 2/25/2009 +0000, Floris Bruynooghe wrote:

...

It's interesting to point out what seems to be planned for Debian: http://lists.debian.org/debian-devel/2009/02/msg00431.html

Quoting just the relevant part:

""" Local installation path -----------------------

When installing Python modules using distutils, the resulting files end up in the same location wether they are installed by a Debian package, or by a local user or administrator, unless the installation path is overwritten on the command line.  Compare this with most software based on autoconf, where an explicit prefix has to be provided for the packaging, while the default install installs into /usr/local.  For new Python versions packaged in Debian this will change so that an installation into /usr (not /usr/local) requires an extra option to distutils install command (--install-layout=deb).  To avoid breaking the packaging of existing code the distutils install command for 2.4 and 2.5 will just accept this option and ignore it.

I wonder why they don't just use the sitewide distutils.cfg file, which would let them configure user-installed packages to go to somewhere else, e.g.:

[install] prefix = /usr/local

(And of course the build tools could specify different options.)

There might be an issue with this while building python itself. I remember having issues failing to build python because I had [install] home=/home/flub in my .pydistutils.cfg. When python is building and installing it's standard library it uses distutils and having that in my configuration file messed everything up and the builds failed (this was both when building the upstream tarball as well as rebuilding the debian package). This is all IIRC however, I'd have to try this again to remind me of the exact issues (thinking about it now this is probably because python's makefile doesn't invoke setup.py with enough options to be sure to overwrite the configuration file). So you'd have to build-conflict against python, which would be annoying at best, but you also need python to build python these days (again IIRC, might be possible to get round this) so you're stuck. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org