[Python-Dev] PEP 376

[Tarek Ziadé]

On Fri, Jul 3, 2009 at 7:02 PM, P.J. Eby<pje at telecommunity.com> wrote:
> At 02:54 PM 7/3/2009 +0100, Paul Moore wrote:
>>
>> Eggs are fundamentally a PEP 302 zip file format. There are some extra
>> bits of metadata for setuptools/easy_install in there (as I understand
>> things) but essentially they are zip files. When you say "decoupling the egg
>> format", I assume you mean "decoupling the egg metadata" - which is fine,
>> but to properly decouple, you need API level access to the metadata. PEP 376
>> offers read-only access, but as you rightly point out, it is only for
>> filesystem data (and some form of zip file, which appears to be limited in
>> some way, as it isn't PEP 302 based, and the actual format isn't defined
>> anywhere). The basic point here is that PEP 376 needs to define precisely
>> how pkgutil.get_distributions() scans sys.path looking for ".egg-info
>> directories". What does it do for sys.path entries that don't correspond to
>> filesystem directories? (Note - these may or may not be zip files. Even if
>> they are zip files, an earlier entry on sys.path_hooks could have taken
>> precedence. At the very least, you should only process path entries as zip
>> files if their importer - in sys.path_importer_cache or via an explicit path
>> hook scan - is a zipimporter object.). To be honest, this is a major can of
>> worms. But if PEP 376 is not going to support PEP 302, then it must state
>> that fact explicitly, to avoid giving people false expectations -
>> particularly with Brett's importlib in Python 3.1, which will make it far
>> easier for people to experiment with new packaging formats such as the ones
>> Lukasz mentions above. And it MUST fail gracefully in the face of
>> unsupported importer types.
>
> Well, we could always resurrect PEP 365, since pkg_resources already has
> documented extensible support for arbitrary importers.  That solves backward
> *and* forward compatibility.  Then PEP 376's uninstall facilities could be
> implemented using pkg_resources' existing metadata query features.
>
> The primary downside to that, of course, is that it brings in the matter of
> version specifications and dependencies...  which appear to be a contentious
> topic.  (Note that Tarek is proposing to drop the PEP 386 proposal to
> standardize a much more restrictive scheme than seutptools' version parser,
> precisely because of the controversy.)

I don't think we can add pkg_resources as-is because it does much more than
providing the query features, which is beyond distutils scope.

I think that part of the PEP 376 code at the end will be very similar
to the subpart of pkg_resources
that has the query features (as it is now with the current prototype)

And what is nice with Paul's proposal on changing PEP 302 is that
setuptools will
be able to propose its EggFormats on the top of the pkgutils/distutils
changes and make its
pkg_resource code lighter.