At 07:15 PM 7/6/2009 +0100, Paul Moore wrote:

My point was that distributions which use setuptools in their setup.py, *even if there's no runtime dependency on setuptools* end up with non-standard .egg-info's. There is no good reason for this, from my POV as a package user. So if setuptools is breaking otherwise entirely standard packages, so that they won't work with core distutils mechanisms, that's wrong. Setuptools is refusing to play nicely with other packaging systems.

Now at the moment, this is a moot point, because the stdlib doesn't have any package discovery mechanisms.

Not so; it has .egg-info. (See footnote below.)

The PEP 302 hooks I'm writing provide the "plugin" ability for alternative storage formats. But there's no provision for alternative *metadata* formats. What's the point of a standard if people don't follow it?

I'm confused. Are you saying pkgutil will recognize .egg-info, or not?

The only practical difference between the existing formats and the proposed new one (AFAICR) is the extension of the directory name, and the presence or absence of the RECORD file.

My POV is that it would be a shame to discard compatibility with existing installed software, just for a bikeshedding of the directory name. Especially if it means that any tools that currently generate .egg-info's (and I'm not 100% positive setuptools is the only case) will need to grow options for whether to generate the old name, possibly based on what Python version... except you can install new distutils into old Python... etc.

If this were strictly a matter of, "from Python 3.x on, generate .pydist", it's a lot less of a problem for me and everybody else, because Python 3 is effectively going to force a fork (or at least a branch) of setuptools anyway.

For Python 2.x, having *two* names for .egg-info (which is what I understand the proposal to effectively be) is just asking for trouble, IMO.

And my concern is that currently these compatibility issues are being handwaved rather than considered up front.

To reiterate, summarize, and attempt to make as concrete as I can:

I'm fine with a rename, *if and only if* it can be done in a way that never requires a tool to generate BOTH files for compatibility reasons. AFAICT, this can only be done by making the extension dependent upon the Python version, and this version distinction must be made *even if* the distutils themselves are backported to an earlier Python version. (Alternatively, this could be resolved by *not* backporting the PEP 376-compatible distutils/pkgutil to 2.x.)

Currently, there is no official Python 3 release of setuptools, so compatibility can (IMO) be considered moot there. What's more, it is easy to conditionally determine whether to use '.egg-info' or '.pydist' based on Python version. What is not easy and likely to be confusing for everyone, is having the possibility for *both* to exist in the same system, *without* also having bidirectional interoperability.

So, I'm +1 for no change (obviously), and +0 for "always .pydist in 3.x and always .egg-info in 2.x", whether the latter part is achieved by making distutils/pkgutil use a version-dependent extension, or by refusing to backport distutils/pkgutil to 2.x. I'm -1 for having .pydist ever appear on a 2.x Python, either via stdlib or backporting.

(These comments also apply for any name other than '.pydist'.)

Footnote: there has been some discussion that implies that somehow .egg-info was hijacked into the stdlib, and that it should therefore not be supported. This is incorrect: at Pycon the year this happened (2005 IIRC), Guido *blessed setuptools for inclusion in the stdlib* in his keynote. Yes, *all* of setuptools, including easy_install, eggs, the whole nine yards.

A few months later, when I actually checked in the code, there were protests from a few people on Python-Dev, some of which I agreed were valid. I reverted the checkin, floating instead a compromise proposal to add .egg-info support to distutils (i.e. the install_egg_info command) to allow setuptools and other 3rd party tools to detect the presence of distutils-installed packages.

At the time, the assumption was that eggs and setuptools would still, eventually, make it into the stdlib -- at the time, Guido had not yet changed his mind about that. So the ".egg-info" name was not controversial, to the best of my recollection, and the inclusion did not violate any of the expressed concerns of the three persons who objected to including setuptools whole-hog.

So, from my POV at least, dropping .egg-info is dropping backward compatibility on a feature that was *intentionally* added to the stdlib -- with approval from Guido and non-protest from Python-Dev -- specifically to provide that compatibility.

That doesn't obligate it to stay... but I suggest that it *does* mean Python 3, not 2, is the appropriate place for dropping it (if it is indeed to be dropped)... and that it is not appropriate to do so in 2.x on merely a handwave by persons not involved in the previous discussions. (And of course Guido can Pronounce otherwise; I'm just saying I don't think anything less than such a pronouncement is required to remove it from 2.x. And again, I'm fine with making a clean break of it in 3.x, since setuptools is already broken there anyway.)