[Distutils] setuptools and pkg registration

[Phillip J. Eby]

At 11:37 PM 4/10/2008 +0200, Jeroen Ruigrok van der Werven wrote:
>Phillip,
>
>I am not sure (haven't verified) if my email ever got through to the list,
>but with FreeBSD's Perl installation an additional module gets installed
>that registers every CPAN installed module as a pkg in FreeBSD's pkg system.
>
>Given the recent bruhaha about the setuptools and eggs (hey, from a
>programmer and sysadmin point of view I like 'm) I wonder if this would be
>easier for the discussion if you could dump a prototype on the table.
>
>If it would help I can see if I can cook up something.

At this point, it has become clear to me that the package database 
isn't really necessary for other systems.  easy_install could simply 
not uninstall things that it doesn't find listed in its *own* metadata.

Really, the only technical hurdle remaining to solving egg-free 
installation is setting up a namespace package's __path__ correctly, 
in a situation where:

1. The user has system packages contributing to the namespace in a 
"site" directory,
2. There is no __init__.py in the namespace under that site directory,
3. There are non-egg packages on PYTHONPATH that contribute to the 
same directory, and
4. Two versions of the same module within that namespace have been 
installed, one in site-packages and the other on PYTHONPATH

Actually, there are various other criteria involved, such as whether 
pkg_resources is imported or not, and whether the setuptools site.py 
hack is employed in the PYTHONPATH directory.  For example, if 
pkg_resources were imported (which normally happens when 
setuptools-installed scripts are run), it would be possible to have 
it do the sorting, and thus ensure that there's never a problem like that.

What it would not fix, is the circumstance where you just pop into 
the Python interpreter with your PYTHONPATH set, if you have the 
above situation.  And the above situation is going to occur anytime 
you're using PYTHONPATH to get a later version of something that uses 
a namespace package.

So, the other alternative is to keep the 'site.py' hack in effect, 
but either change the contents of that site.py or add extra 
information to a .pth file so that the namespaces get sorted at that 
point.  There would still be a hacked site module at that point, but 
at least upgrading the site.py would let us get rid of the need to 
load any .pth files from the directory, put eggs ahead of other things, etc.

For that matter, the site.py wouldn't be needed unless you were 
installing things with namespace packages in it.  (Unfortunately, 
looking at site-packages to see if the hack is needed won't help, 
since you might install a conflicting system package *later*.)