[Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 10:01 AM 4/8/2008 -0700, zooko wrote:

>On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> > zooko wrote:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
> >> Here is a simple proposal:  make the standard Python "import"
> >> mechanism notice eggs on the PYTHONPATH and insert them (into the
> >> *same* location) on the sys.path.
> >> This eliminates the #1 problem with eggs -- that they don't
> >> easily  work when installing them into places other than your site-
> >> packages  and that if you allow any of them to be installed on
> >> your system then  they take precedence over your non-egg packages
> >> even you explicitly  put those other packages earlier in your
> >> PYTHONPATH.  (That latter  behavior is very disagreeable to more
> >> than a few prorgammers.)
> >
> > Sorry if I'm out of the loop and there's some subtlety here that
> > I'm disregarding, but it doesn't appear that either of the issues
> > you mention is a actually problem with eggs.  These are instead
> > problems with how eggs get installed by easy_install (which uses
> > a .pth file to extend sys.path).  It's reasonable to put eggs on
> > the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))
> > instead of using easy_install to install them.
>
>Yes, you are missing something.  While many programmers, such as
>yourself and Lennart Regebro (who posted to this thread) find the
>current eggs system to be perfectly convenient and to Just Work, many
>others, such as Glyph Lefkowitz (who posted to a related thread) find
>them to be so annoying that they actively ensure that no eggs are
>ever allowed to touch their system.
>
>The reasons for this latter problem are two:
>
>1.  You can't conveniently install eggs into a non-system directory,
>such as ~/my-python-stuff.

Wha?

>2.  If you allow even a single egg to be installed into your
>PYTHONPATH, it will change the semantics of your PYTHONPATH.

Only in the same way that manually putting an egg on the front of 
PYTHONPATH can be considered to "change the semantics" of your PYTHONPATH.


>Both of these problems are directly caused by the need for eggs to
>hack your site.py.  If Python automatically added eggs found in the
>PYTHONPATH to the sys.path, both of these problems would go away.

And add new ones.


>I am skeptical that the current proposals to define a new database
>for installed packages will fare any better than the current eggs
>scheme does in this respect.

The purpose for the installation database is to allow easy_install to 
eschew the use of .egg files or directories for anything other than 
multi-version installs.  Thus, no need to add those .egg files or 
directories to the head of the PYTHONPATH.  Conflicts would be 
handled at install time rather than runtime, in other words.


>I am skeptical that prorgammers are going to be willing to use a new
>database format.  They already have a database -- their filesystem --
>and they already have the tools to control it -- mv, rm, and
>PYTHONPATH.  Many of them already hate the existence the
>"easy_instlal.pth" database file, and I don't see why a new database
>file would be any different.

PEP 262 does not propose a database file -- it proposes the inclusion 
of a metadata file for each installed distribution.


>My proposal makes the current benefits of eggs -- clean, easy code re-
>use among programmers -- more compatible with their current tools --
>mv, rm, and PYTHONPATH.  It is also forward-compatible with more
>sophisticated proposals to add features like uninstall and operating
>system integration.

Actually, your current proposal doesn't work, unless you at least 
have some way to indicate which *version* of an egg should be 
automatically added to sys.path -- and some way to change 
that.  Otherwise, you might as well use the -m option to 
easy_install, and require() the eggs at runtime.  (Which needs 
neither .pth files nor site.py hacking.)

Meanwhile, my understanding is that the people who dislike eggs, 
dislike them because when they install a setuptools-based package, 
it's installed as an egg by default.  The installation database 
proposal (and by the way, people really should read and understand 
PEP 262, including the open issues, before trying to compete with 
it), will allow setuptools-based packages to install the 
"old-fashioned" way by default.  That is, not as eggs.  Similarly, 
easy_install would be able to skip installing .eggs unless you wanted 
multi-version support.

So, people who don't like eggs would never see them, since the only 
way you'd ever get them would be via easy_install -m, and they would 
never use it.


>By the way, since I posted my proposal two weeks ago I have pointed a
>couple of Python hackers who currently refuse to use eggs at the URL:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
>They both agreed that it made perfect sense.  I told one of them
>about the alternate proposal to define a new database file to contain
>a list of installed packages, and he sighed and rolled his eyes and
>said "So they are planning to reinvent apt!".

No, we're planning to make it possible for easy_install not to 
overwrite things that would break your system, and allow distutils 
and setuptools to uninstall what they installed.  That's a 
considerably less ambitious goal, by far.  :)