Why are egg-info and related pth files required for rpm packages?
I recently installed Fedora 7 and looked at /usr/lib/python2.5/site-packages. The directory has numerous Python packages installed but no egg-info and few .pth files. Of the three .pth files in my installation, only one has a content different from the name of the package (pointing to a subdirectory of site-packages with a different name). Setuptools has the advantage of simplifying preparation of rpms because it has find_packages(). However, it produces egg-info and .pth files as part of the rpm's. I can understand the need for these files when packaging into eggs for installation in places other than python/site-packages or when packaging for non-Linux operating systems. But why are they needed when installing as rpms on Linux systems into site-packages? I think this issue applies to most Linux packaging, which is usually either rpm or deb. Stan Klein
At 12:39 PM 9/4/2007 -0400, Stanley A. Klein wrote:
I recently installed Fedora 7 and looked at /usr/lib/python2.5/site-packages. The directory has numerous Python packages installed but no egg-info and few .pth files. Of the three .pth files in my installation, only one has a content different from the name of the package (pointing to a subdirectory of site-packages with a different name).
Setuptools has the advantage of simplifying preparation of rpms because it has find_packages(). However, it produces egg-info and .pth files as part of the rpm's.
I can understand the need for these files when packaging into eggs for installation in places other than python/site-packages or when packaging for non-Linux operating systems. But why are they needed when installing as rpms on Linux systems into site-packages?
I think this issue applies to most Linux packaging, which is usually either rpm or deb.
The .egg-info files or directories are required in order to contain project-level metadata. Without them, the fact that the packages are installed can't be detected in a cross-platform way, and other projects *simply won't function without them*. For example, Chandler i18n support is provided via the contents of .egg-info, as is plugin registration. A wide range of plugin-supporting libraries and tools absolutely require .egg-info, including TurboGears, Buffet, Trac, Chandler and setuptools itself. Setuptools can't even build itself unless its own .egg-info is present! So .egg-info is absolutely indispensable, regardless of installation method. As for .pth files, the only .pth files that should be generated by setuptools are the ones needed to support namespace packages. When you have multiple projects spanning a namespace package, each of those projects would contain "somepackage/__init__.py" in its naive layout. But this would cause conflicts between the RPMs, so setuptools uses uniquely-named .pth files to work around the absence of an __init__.py. So, these "Project-version-nspkg.pth" files are also indispensable, as the packages involved won't be importable without them. However, the .pth files you described don't sound like ones generated by setuptools. Note, by the way, that as of Python 2.5, *all* distutils-generated packages include .egg-info; they just use a single file instead of a directory. This makes it easy to detect what Python packages are installed on a system, as long as the platform maintainers don't remove this file.
At 12:39 PM 9/4/2007 -0400, Stanley A. Klein wrote:
I recently installed Fedora 7 and looked at /usr/lib/python2.5/site-packages. The directory has numerous Python packages installed but no egg-info and few .pth files. Of the three .pth files in my installation, only one has a content different from the name of the package (pointing to a subdirectory of site-packages with a different name).
I have some egg-info files in my site-packages. Those are from setuptools using packages. I noticed the seemingly useless .pth files
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: the other day and am puzzled by their presence. More info on those later in this post. [...]
The .egg-info files or directories are required in order to contain project-level metadata. [...] So .egg-info is absolutely indispensable, regardless of installation method.
Absolutely agreed.
As for .pth files, the only .pth files that should be generated by setuptools are the ones needed to support namespace packages. When you have multiple projects spanning a namespace package, each of those projects would contain "somepackage/__init__.py" in its naive layout. But this would cause conflicts between the RPMs, so setuptools uses uniquely-named .pth files to work around the absence of an __init__.py. So, these "Project-version-nspkg.pth" files are also indispensable, as the packages involved won't be importable without them.
However, the .pth files you described don't sound like ones generated by setuptools.
I looked into this briefly when attempting to get rid of .pth's and eggs to diagnose the earlier bug (thanks again for the quick patch and release!) The packages I've looked at so far are all being generated by distutils and have a C component. I haven't had a chance to delve deeper.
Note, by the way, that as of Python 2.5, *all* distutils-generated packages include .egg-info; they just use a single file instead of a directory. This makes it easy to detect what Python packages are installed on a system, as long as the platform maintainers don't remove this file.
I'm sorry to say that this is not true on Fedora 7's python2.5. There's a patch that disables generating egg-info files for distutils. I've started talking with the python maintainer in Fedora to find out why the patch exists and if it can be removed but he needs some time to find out why the patch was added in the first place. (A note in the spec files implies that the patch was added so as not to generate egg-info for python core libraries and it might not have been meant to affect distutils as a whole. I have to figure out if even that level of meddling is going to prove bothersome and make a recommendation. If you can think of some cases where that would be bad, please reply so that I can include them in our discussion.) - -Toshio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFG3chuX6yAic2E7kgRAs5nAJwPYy3Qh5udkOujX6Hz5VoemUyoOACcDwxe vJLPiVHaB38bbhRrvw0j/+c= =sXCC -----END PGP SIGNATURE-----
At 02:04 PM 9/4/2007 -0700, Toshio Kuratomi wrote:
Note, by the way, that as of Python 2.5, *all* distutils-generated packages include .egg-info; they just use a single file instead of a directory. This makes it easy to detect what Python packages are installed on a system, as long as the platform maintainers don't remove this file.
I'm sorry to say that this is not true on Fedora 7's python2.5.
Well, as I said, "as long as the platform maintainers don't remove" it. :) I didn't know of anybody specifically removing it, but given the long and colorful history of distributions hacking up the way Python is installed, I expected that *somebody* would be unable to resist tinkering. :)
There's a patch that disables generating egg-info files for distutils. I've started talking with the python maintainer in Fedora to find out why the patch exists and if it can be removed but he needs some time to find out why the patch was added in the first place.
(A note in the spec files implies that the patch was added so as not to generate egg-info for python core libraries and it might not have been meant to affect distutils as a whole. I have to figure out if even that level of meddling is going to prove bothersome and make a recommendation. If you can think of some cases where that would be bad, please reply so that I can include them in our discussion.)
If you mean the Python-2.5.egg-info and wsgiref.egg-info, they are both definitely supposed to be there. The latter is there so that packages that depend on a specific version of wsgiref can do so without needing to check what Python version they're on. I really would have liked to include .egg-info for *all* the 2.5 stdlib add-ons including ctypes and ElementTree, but there wasn't time to co-ordinate that before the release.
participants (3)
-
Phillip J. Eby -
Stanley A. Klein -
Toshio Kuratomi