[Distutils] [buildout] path usage and proposal

[David Lyon]

Good Evening Jim,

On Mon, 6 Jul 2009 11:48:25 -0400, Jim Fulton <jim at zope.com> wrote:

> ..
> Paths are also used by running scripts to search for packages.  When
> buildout generates a script, it generates a search path based on the
> working set.  Note that it adds the working set locations to the base
> path that the script starts with.

True

> For existing behavior, we can work smarter by searching the base path
> + the current install paths for installed distribution and passing an
> empty search path the index used to look for uninstalled distributions.

That's already how it works.

> Note that not every distribution found on the base path is
> importable.  In particular, an egg may be found on the base path, but
> the egg itself must be added to the Python path to be importable.
> The base path is used for 2 things:
> 
> - A place to look for installed distributions
> - A place to import things from at run time.

Yes, and the importing is done via .PTH files...

> It would be cleaner to separate these uses, although, for usability,
> it might be simpler to keep them together.  There's probably little
> harm in including a path in sys.path that *only* includes eggs.

The only way to enforce that is to change the interpretor itself..

Within site.py... otherwise.. enforce it with some sort of packaging
tool.... lol

Are you suggesting something like a directory "SITE-EGGS" ?

> So hear's a proposal:
> 
> 1. Add an include-site-packages option, defaulting to true.  If false:
>    set the base path to those paths not added by running site.py.
>    We'll get this by evaling the output of:
> 
>      python -Sc "import sys; print (repr(sys.path))"
> 
>    The base path will be used for searching for distrbutions *and* for
>    computing the run-time path.  The script preamble will become
>    someting like::
> 
>       sys.path[:] = ... # set sys.path to the base path
>       for path in ...: # iterate over working set using expression we  
> use now
>           if path not in sys.path:
>               sys.path.insert(0, path)
> 
>    Note:
>    a. We completely replace sys.path
>    b. If a distro's location was already in the base path, we don't
>       insert it.  This will help avoid accidently putting a base path
>       ahead of egg-supplied paths.

It seems clunky....

> 2. Add a search-path option.  By default, it will be the base path.
>    It can be manipulated in 2 ways:
> 
>    - It can be set.  If this is done, the base path will be ignored.
> 
>    - It can be incremented, for example, to cherry-pick the foo package
>      on ubuntu:
> 
>        search-path += /usr/share/pycentral/foo/site-packages
> 
..

> Thoughts? Questions?

All the functionality you are asking for already exists within python
already... afiak there's already provision to add extra paths and do all
these extra things.

Under linux, there is this great place to put packages .. 
/usr/local/pythonx.x/.. and under windows there's also a spot in
\Documents and Settings\User\My Documents.

EasyInstall will let us install there if we ask it to....

The big problem is that we/users aren't doing any of this all that
well at the moment because we/they forget the command lines.

Maybe I don't know buildout too well... so I don't know the issue..

Maybe you just need a better tool for installing packages under
buildout.. maybe I could modify my python package manager in some
way to support buildout better....

http://sourceforge.net/projects/pythonpkgmgr/

In a future version of the python package manager, I will
probably add capabilities to move packages from one place
to another... haha... generate buildout files..

None of these things really require anything extra from
distutils than is already there....

Regards

David