In a nutshell, here is what I want to be able to do:
Python Eggs are great, but I have a major problem with setuptools: Where I work, I am not allowed to alter PYTHONPATH or to install into site-packages. I can add paths to sys.path at runtime, but normally we load modules via sys.meta_path. We have a large database of modules by many different developers, and we have per-project configuration files which select versions of these modules to be accessible via sys.meta_path. I'd like to continue to select our own modules via the configuration files, but I'd like to load Eggs from PYPI via the procedure above.
This may seem impossible, but I have a fairly simple solution: Put an Egg importer on sys.meta_path. I'll outline the solution:
First, realize that no site.py or .pth files will be processed in this usage model.
Second, when a version range is specified via pkg_resources.require(), register that specification into a global Egg importer (attached to the setuptools module). Also, append that importer to the end of sys.meta_path, if that has not already been done. This Egg importer should follow the protocol of PEP 302.
Third, when a module registered as an Egg is imported, Python will first look for an importer on sys.meta_path. If not intercepted by another meta_path importer, eventually our new Egg importer will be tried. The Egg importer should search for the Egg by scanning sys.path, in order. It should look not only for Eggs, but also for directories which might contain Eggs. It should accept the first Egg discovered which matches the registered criteria and load it. Otherwise, it should throw an ImportError exception.
Some things to note about this approach:
I hope that everyone takes this request seriously. The current system is almost completely broken for me. The only way I can use eggs is to add each one to sys.path explicitly at runtime. That's a huge PITA and a big maintenance problem. I'm very disappointed by the whole PYPI system at the moment. Something that could be extremely easy is instead very difficult.