Metadata files, dist-info/egg-info and migration paths
This is peripherally related to PEP 426, to the extent that PEP 426 specifies that the distribution metadata goes in the dist-info directory defined by PEP 376. The dist-info directory conceptually replaces the old de-facto standard egg-info directory. But neither PEP 376 nor PEP 426 mention anything about what should happen to the *other* files that currently reside in egg-info. These are basically setuptools extensions, for things like namespace packages, entry points, zipped egg support, etc. As far as I am aware, recent versions of distribute look for the setuptools metadata files in the dist-info directory if it's present. So for projects using distribute, moving all of the metadata files to dist-info makes sense. But there's no release of setuptools that supports this, so what should happen there? The issue is with built distributions. Core python distutils still writes an egg-info directory (and that won't change till 3.4, and only distribute supports Python 3, so no issue there). Setuptools and distribute both write to egg-info, so there's no compatibility issue there either. But the wheel format uses dist-info internally, and installation is defined as only producing a dist-info directory (by unpacking the one in the wheel). There are two questions here that bear discussion. First of all, when creating a wheel, should builders put custom metadata files from the existing egg-info data into the dist-info directory. I would suggest that yes, they should, as otherwise that data is lost completely - in particular, setuptools entry points (and hence executable wrappers) fail without the entry_points.txt file. There is some support in distlib for replacement functionality in some of these areas (exports, the EXPORTS file and script wrappers) but this is at an early stage and there's no migration path defined yet that I'm aware of. The more difficult question is what should happen when a wheel is installed. At the moment, tools write out the dist-info directory and that's it. That works fine for projects using distribute, or ones that don't use setuptools-style metadata. But projects using setuptools under Python 2 won't be able to see the metadata. Should we require that in order to use wheels, distribute should be used (or a suitably patched setuptools, should that become available)? Or should wheel installers write a legacy egg-info directory for use by setuptools (I'd suggest that this should only happen on Python 2, and even then probably only if a specific "legacy" flag was set). I have no real knowledge of what to do here - my suggestions above are relatively uninformed, and in particular I have little knowledge of what is common among people still using Python 2. What do the experts think? Paul.
On Mon, Mar 4, 2013 at 4:46 PM, Paul Moore
This is peripherally related to PEP 426, to the extent that PEP 426 specifies that the distribution metadata goes in the dist-info directory defined by PEP 376. The dist-info directory conceptually replaces the old de-facto standard egg-info directory. But neither PEP 376 nor PEP 426 mention anything about what should happen to the *other* files that currently reside in egg-info. These are basically setuptools extensions, for things like namespace packages, entry points, zipped egg support, etc. As far as I am aware, recent versions of distribute look for the setuptools metadata files in the dist-info directory if it's present. So for projects using distribute, moving all of the metadata files to dist-info makes sense. But there's no release of setuptools that supports this, so what should happen there?
The issue is with built distributions. Core python distutils still writes an egg-info directory (and that won't change till 3.4, and only distribute supports Python 3, so no issue there). Setuptools and distribute both write to egg-info, so there's no compatibility issue there either. But the wheel format uses dist-info internally, and installation is defined as only producing a dist-info directory (by unpacking the one in the wheel).
There are two questions here that bear discussion. First of all, when creating a wheel, should builders put custom metadata files from the existing egg-info data into the dist-info directory. I would suggest that yes, they should, as otherwise that data is lost completely - in particular, setuptools entry points (and hence executable wrappers) fail without the entry_points.txt file. There is some support in distlib for replacement functionality in some of these areas (exports, the EXPORTS file and script wrappers) but this is at an early stage and there's no migration path defined yet that I'm aware of.
The more difficult question is what should happen when a wheel is installed. At the moment, tools write out the dist-info directory and that's it. That works fine for projects using distribute, or ones that don't use setuptools-style metadata. But projects using setuptools under Python 2 won't be able to see the metadata. Should we require that in order to use wheels, distribute should be used (or a suitably patched setuptools, should that become available)? Or should wheel installers write a legacy egg-info directory for use by setuptools (I'd suggest that this should only happen on Python 2, and even then probably only if a specific "legacy" flag was set).
I have no real knowledge of what to do here - my suggestions above are relatively uninformed, and in particular I have little knowledge of what is common among people still using Python 2. What do the experts think?
Paul.
Hmmmmm. You have to use distribute >= 0.6.28 or a currently unavailable suitably patched setuptools. It would be fascinating or horrifying to convert the other way, from .dist-info to .egg-info. The installer is not currently smart or perverted enough to do that. Wheels are currently made by converting .egg-info to .dist-info. The wheel converter should preserve any file it doesn't understand. Besides the common metadata, a few distributions register custom metadata. In distribute the pkg_resources "get distribution metadata(filename)" API works identically for dist-info and egg-info directories. HTH, Daniel
Paul Moore
There are two questions here that bear discussion. First of all, when creating a wheel, should builders put custom metadata files from the existing egg-info data into the dist-info directory. I would suggest that yes, they should, as otherwise that data is lost completely - in particular, setuptools entry points (and hence executable wrappers) fail without the entry_points.txt file. There is some support in distlib for replacement functionality in some of these areas (exports, the EXPORTS file and script wrappers) but this is at an early stage and there's no migration path defined yet that I'm aware of.
distlib could be changed to use entry_points.txt as the filename for now. The file format is the same.
The more difficult question is what should happen when a wheel is installed. At the moment, tools write out the dist-info directory and that's it. That works fine for projects using distribute, or ones that don't use setuptools-style metadata. But projects using setuptools under Python 2 won't be able to see the metadata. Should we require that in order to use wheels, distribute should be used (or a suitably patched setuptools, should that become available)? Or should wheel installers write a legacy egg-info directory for use by setuptools (I'd suggest that this should only happen on Python 2, and even then probably only if a specific "legacy" flag was set).
I agree with Daniel here - I don't see any point in writing to .egg-info, and if people can't use distribute, they'll have to wait for setuptools to get compatibility. Presumably, the only reason for not using distribute would be that there's some bug in it which is not in setuptools, and if it's important enough, surely people will contribute the fix, or if they can't, stick with the status quo? I think it makes sense to assist people in migrating from old packaging standards to new ones by reading e.g. old formats, as wheel and wheeler.py do, but if we persist in writing old formats, then it doesn't seem like any migration is actually happening. Regards, Vinay Sajip
On 4 March 2013 23:17, Vinay Sajip
I agree with Daniel here - I don't see any point in writing to .egg-info, and if people can't use distribute, they'll have to wait for setuptools to get compatibility. Presumably, the only reason for not using distribute would be that there's some bug in it which is not in setuptools, and if it's important enough, surely people will contribute the fix, or if they can't, stick with the status quo?
That;'s generally my view. I'm looking at changing virtualenv to install the default packages (setuptools/distribute and pip) from wheels. In doing that, pip's console script entry points will only work if distribute is used, so in effect this means making virtualenv always use distribute. This certainly won't be for a while yet, but I want to be sure it's not going to enrage a horde of dedicated setuptools users... Paul.
participants (3)
-
Daniel Holth
-
Paul Moore
-
Vinay Sajip