[Distutils] PEP 376 -- new round

[Tarek Ziadé]

Hello,

As mentioned on python-dev, I would like to do a new round of work on
PEP 376 before Pycon,

We had a short off-list conversation with Guido on PEP 376, I'd like
to share here so we can discuss the point,
it's about the REQUESTED file:

Guido's question:
 I have a question about REQUESTED. It seems that the absence of this
 file means that the package is installed as a dependency of another
 package, so that it can be removed if that other package is removed .
 But since there is no record of the requesting package, it would seem
 problematic to find out when a dependent package can be removed, since
 there could possibly be multiple packages depending on it. Maybe it
 would be better to keep track of a list of packages that depend on a
 given package, adding a special "root" or "explicit" line for packages
 that were explicitly requested by the user?

My answer:
 Distutils doesn't provide a feature where dependencies gets installed
 or removed. It works at the package level only, and provides tools to
install or
 uninstall one package at a time.

 So we didn't want to provide a record of a full dependency graph
 through this file because we have considered that this work is up to
a full-featured
 install/uninstall tool that wants to install and remove dependencies.
 This tool can calculate this graph when working, as it does when
 installing dependencies, since everything is available in the metadata
 once we have PEP 345 -- with the "requires-dist" field.

 IOW:
 1/ an orphaned package is a package that doesn't have a REQUESTED file,
 *and* that is not used by another package anymore. So this can only be
 known by a uninstaller tool that deals with dependencies.

 2/ a requested package is a package that has the REQUESTED file

 Now for Distutils, the REQUESTED file is useful to mark the packages
 that were explicitely wanted. For example to develop against them.  vs
 things that where installed as a dependency.

 Of course the caveat is that a full-featured uninstall tool might be
 slow to compute the full dependency graph when trying to remove
 dependencies, because it will have to take a look at all other
 REQUESTED packages.

 Installers/uninstallers that are built on the top of Distutils could
 probably create their own dependency
 records inside each .egg-info placeholders, to speed up the work. But
 I am not sure it's such a big deal to look at all metadata files when
 performing an uninstallation. It's ok for this operation to be kind of
 slow I guess. And I am also not sure how this could work if the user
uses different kind
 of installers.

 This REQUESTED file is at the boundary of what we want to provide in
 Distutils. Going further seemed dangerous at this point because as we
 said last year at the summit, we want to make distutils a toolkit that
 provides some standards and a common ground for installers out there,
 rather than a full-featured package manager. IOW: we don't want to
 tell the tools out there how they should deal with the dependencies
 installation / uninstallation process, but give them tools to deal
 efficiently with one package at a time.

 I realize this is not clear enough in the PEP. Let me know if what
 I've explained makes sense to you. If so I'll work on adding details
 on these points. If not, it means we need more work on this.



So Guido wonders if adding this file is really necessary since higher-level
installers/uninstallers will have to build the graph anyways.

I think it's still useful, because it points the root packages that
can be removed
safely without breaking the system -- even if leaving orphaned packages behind.

Any opinion ?

Regards,
Tarek

-- 
Tarek Ziadé | http://ziade.org