At 02:54 PM 7/3/2009 +0100, Paul Moore wrote:

Eggs are fundamentally a PEP 302 zip file format. There are some extra bits of metadata for setuptools/easy_install in there (as I understand things) but essentially they are zip files. When you say "decoupling the egg format", I assume you mean "decoupling the egg metadata" - which is fine, but to properly decouple, you need API level access to the metadata. PEP 376 offers read-only access, but as you rightly point out, it is only for filesystem data (and some form of zip file, which appears to be limited in some way, as it isn't PEP 302 based, and the actual format isn't defined anywhere). The basic point here is that PEP 376 needs to define precisely how pkgutil.get_distributions() scans sys.path looking for ".egg-info directories". What does it do for sys.path entries that don't correspond to filesystem directories? (Note - these may or may not be zip files. Even if they are zip files, an earlier entry on sys.path_hooks could have taken precedence. At the very least, you should only process path entries as zip files if their importer - in sys.path_importer_cache or via an explicit path hook scan - is a zipimporter object.). To be honest, this is a major can of worms. But if PEP 376 is not going to support PEP 302, then it must state that fact explicitly, to avoid giving people false expectations - particularly with Brett's importlib in Python 3.1, which will make it far easier for people to experiment with new packaging formats such as the ones Lukasz mentions above. And it MUST fail gracefully in the face of unsupported importer types.

Well, we could always resurrect PEP 365, since pkg_resources already has documented extensible support for arbitrary importers. That solves backward *and* forward compatibility. Then PEP 376's uninstall facilities could be implemented using pkg_resources' existing metadata query features. The primary downside to that, of course, is that it brings in the matter of version specifications and dependencies... which appear to be a contentious topic. (Note that Tarek is proposing to drop the PEP 386 proposal to standardize a much more restrictive scheme than seutptools' version parser, precisely because of the controversy.)