[Distutils] Metadata files, dist-info/egg-info and migration paths

[Daniel Holth]

On Mon, Mar 4, 2013 at 4:46 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> This is peripherally related to PEP 426, to the extent that PEP 426
> specifies that the distribution metadata goes in the dist-info
> directory defined by PEP 376. The dist-info directory conceptually
> replaces the old de-facto standard egg-info directory. But neither PEP
> 376 nor PEP 426 mention anything about what should happen to the
> *other* files that currently reside in egg-info. These are basically
> setuptools extensions, for things like namespace packages, entry
> points, zipped egg support, etc. As far as I am aware, recent versions
> of distribute look for the setuptools metadata files in the dist-info
> directory if it's present. So for projects using distribute, moving
> all of the metadata files to dist-info makes sense. But there's no
> release of setuptools that supports this, so what should happen there?
>
> The issue is with built distributions. Core python distutils still
> writes an egg-info directory (and that won't change till 3.4, and only
> distribute supports Python 3, so no issue there). Setuptools and
> distribute both write to egg-info, so there's no compatibility issue
> there either. But the wheel format uses dist-info internally, and
> installation is defined as only producing a dist-info directory (by
> unpacking the one in the wheel).
>
> There are two questions here that bear discussion. First of all, when
> creating a wheel, should builders put custom metadata files from the
> existing egg-info data into the dist-info directory. I would suggest
> that yes, they should, as otherwise that data is lost completely - in
> particular, setuptools entry points (and hence executable wrappers)
> fail without the entry_points.txt file. There is some support in
> distlib for replacement functionality in some of these areas (exports,
> the EXPORTS file and script wrappers) but this is at an early stage
> and there's no migration path defined yet that I'm aware of.
>
> The more difficult question is what should happen when a wheel is
> installed. At the moment, tools write out the dist-info directory and
> that's it. That works fine for projects using distribute, or ones that
> don't use setuptools-style metadata. But projects using setuptools
> under Python 2 won't be able to see the metadata. Should we require
> that in order to use wheels, distribute should be used (or a suitably
> patched setuptools, should that become available)? Or should wheel
> installers write a legacy egg-info directory for use by setuptools
> (I'd suggest that this should only happen on Python 2, and even then
> probably only if a specific "legacy" flag was set).
>
> I have no real knowledge of what to do here - my suggestions above are
> relatively uninformed, and in particular I have little knowledge of
> what is common among people still using Python 2. What do the experts
> think?
>
> Paul.

Hmmmmm.

You have to use distribute >= 0.6.28 or a currently unavailable
suitably patched setuptools. It would be fascinating or horrifying to
convert the other way, from .dist-info to .egg-info. The installer is
not currently smart or perverted enough to do that.

Wheels are currently made by converting .egg-info to .dist-info. The
wheel converter should preserve any file it doesn't understand.
Besides the common metadata, a few distributions register custom
metadata. In distribute the pkg_resources "get distribution
metadata(filename)" API works identically for dist-info and egg-info
directories.

HTH,

Daniel