[Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[zooko]

On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> zooko wrote:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

>> Here is a simple proposal:  make the standard Python "import"   
>> mechanism notice eggs on the PYTHONPATH and insert them (into the   
>> *same* location) on the sys.path.
>> This eliminates the #1 problem with eggs -- that they don't  
>> easily  work when installing them into places other than your site- 
>> packages  and that if you allow any of them to be installed on  
>> your system then  they take precedence over your non-egg packages  
>> even you explicitly  put those other packages earlier in your  
>> PYTHONPATH.  (That latter  behavior is very disagreeable to more  
>> than a few prorgammers.)
>
> Sorry if I'm out of the loop and there's some subtlety here that  
> I'm disregarding, but it doesn't appear that either of the issues  
> you mention is a actually problem with eggs.  These are instead  
> problems with how eggs get installed by easy_install (which uses  
> a .pth file to extend sys.path).  It's reasonable to put eggs on  
> the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))  
> instead of using easy_install to install them.

Yes, you are missing something.  While many programmers, such as  
yourself and Lennart Regebro (who posted to this thread) find the  
current eggs system to be perfectly convenient and to Just Work, many  
others, such as Glyph Lefkowitz (who posted to a related thread) find  
them to be so annoying that they actively ensure that no eggs are  
ever allowed to touch their system.

The reasons for this latter problem are two:

1.  You can't conveniently install eggs into a non-system directory,  
such as ~/my-python-stuff.

2.  If you allow even a single egg to be installed into your  
PYTHONPATH, it will change the semantics of your PYTHONPATH.

Both of these problems are directly caused by the need for eggs to  
hack your site.py.  If Python automatically added eggs found in the  
PYTHONPATH to the sys.path, both of these problems would go away.

I am skeptical that the current proposals to define a new database  
for installed packages will fare any better than the current eggs  
scheme does in this respect.

This issue is important to me, because the benefits of eggs grow  
superlinearly with the number of good programmers who use them.  They  
are a tool for re-using source code -- a tool for cooperation between  
programmers.  To gain the greatest benefits at this point we do not  
need to add new features to eggs, we need to make them more palatable  
to more good programmers.

I am skeptical that prorgammers are going to be willing to use a new  
database format.  They already have a database -- their filesystem --  
and they already have the tools to control it -- mv, rm, and  
PYTHONPATH.  Many of them already hate the existence the  
"easy_instlal.pth" database file, and I don't see why a new database  
file would be any different.

My proposal makes the current benefits of eggs -- clean, easy code re- 
use among programmers -- more compatible with their current tools --  
mv, rm, and PYTHONPATH.  It is also forward-compatible with more  
sophisticated proposals to add features like uninstall and operating  
system integration.

By the way, since I posted my proposal two weeks ago I have pointed a  
couple of Python hackers who currently refuse to use eggs at the URL:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

They both agreed that it made perfect sense.  I told one of them  
about the alternate proposal to define a new database file to contain  
a list of installed packages, and he sighed and rolled his eyes and  
said "So they are planning to reinvent apt!".

Regards,

Zooko