Hello,

As mentioned on python-dev, I would like to do a new round of work on PEP 376 before Pycon,

We had a short off-list conversation with Guido on PEP 376, I'd like to share here so we can discuss the point, it's about the REQUESTED file:

Guido's question: I have a question about REQUESTED. It seems that the absence of this file means that the package is installed as a dependency of another package, so that it can be removed if that other package is removed . But since there is no record of the requesting package, it would seem problematic to find out when a dependent package can be removed, since there could possibly be multiple packages depending on it. Maybe it would be better to keep track of a list of packages that depend on a given package, adding a special "root" or "explicit" line for packages that were explicitly requested by the user?

My answer: Distutils doesn't provide a feature where dependencies gets installed or removed. It works at the package level only, and provides tools to install or uninstall one package at a time.

So we didn't want to provide a record of a full dependency graph through this file because we have considered that this work is up to a full-featured install/uninstall tool that wants to install and remove dependencies. This tool can calculate this graph when working, as it does when installing dependencies, since everything is available in the metadata once we have PEP 345 -- with the "requires-dist" field.

IOW: 1/ an orphaned package is a package that doesn't have a REQUESTED file, *and* that is not used by another package anymore. So this can only be known by a uninstaller tool that deals with dependencies.

2/ a requested package is a package that has the REQUESTED file

Now for Distutils, the REQUESTED file is useful to mark the packages that were explicitely wanted. For example to develop against them. vs things that where installed as a dependency.

Of course the caveat is that a full-featured uninstall tool might be slow to compute the full dependency graph when trying to remove dependencies, because it will have to take a look at all other REQUESTED packages.

Installers/uninstallers that are built on the top of Distutils could probably create their own dependency records inside each .egg-info placeholders, to speed up the work. But I am not sure it's such a big deal to look at all metadata files when performing an uninstallation. It's ok for this operation to be kind of slow I guess. And I am also not sure how this could work if the user uses different kind of installers.

This REQUESTED file is at the boundary of what we want to provide in Distutils. Going further seemed dangerous at this point because as we said last year at the summit, we want to make distutils a toolkit that provides some standards and a common ground for installers out there, rather than a full-featured package manager. IOW: we don't want to tell the tools out there how they should deal with the dependencies installation / uninstallation process, but give them tools to deal efficiently with one package at a time.

I realize this is not clear enough in the PEP. Let me know if what I've explained makes sense to you. If so I'll work on adding details on these points. If not, it means we need more work on this.

So Guido wonders if adding this file is really necessary since higher-level installers/uninstallers will have to build the graph anyways.

I think it's still useful, because it points the root packages that can be removed safely without breaking the system -- even if leaving orphaned packages behind.

Any opinion ?

Regards, Tarek