At 12:11 AM 3/4/2009 -0600, skip@pobox.com wrote:

So I have rather casually installed a bunch of packages from PyPI using easy_install. It now contains 23 eggs, all of which get used a bit, but nowhere near as often as the standard lib or our internal stuff. Unsuccessfully stat()ing in all those egg directories was really starting to hammer our NFS server (we often start up hundreds of Python processes at a time), so I commented out the second import line at the end which serves to leave the eggs at the end of sys.path. Now every time I install a package I have to quickly edit easy-install.path to remove the import line the install process adds.

I wonder though... Is there some alternate way to install stuff from PyPI which reduces the number of directories on sys.path? Maybe some post-processing step which takes the current set of eggs, creates a single directory into which they are all unfolded? That single directory could then be added to sys.path.

You could use Ian Bicking's 'pip' installer. It can't install from eggs, but it does do source installs this way. There is a very straightforward mapping from zipped eggs to "flat" files, btw. For a given .egg file or directory, the mapping is simply to unzip it, renaming its EGG-INFO subdirectory to the original egg's filename with '-info' appended. E.g. "foobar-1.2-py2.5.egg/EGG-INFO/" becomes "foobar-1.2-py2.5.egg-info/". Everything else remains exactly the same. It wouldn't be hard to make a tool that does this; in fact, the setuptools.archive_util module has an 'extract_archive' function that works on directories as well as zipfiles, and can be passed a filter function that can change the extraction location for any given file. Something like: def flatten_egg(egg_filename, extract_dir): eggbase = os.path.filename(egg_filename)+'-info' def file_filter(src, dst): if src.startswith('EGG-INFO/'): src = eggbase+s[8:] dst = os.path.join(extract_dir, *src.split('/')) return dst return extract_archive(egg_filename, extract_dir, file_filter) Of course, this will only work if 'egg_filename' is a valid .egg file or directory; the results are otherwise pretty unpredictable. This also doesn't check for overwriting other files, deal with permissions, etc. Ergo, making this production-ready is left as an exercise for the implementer. ;-)