[Python-ideas] Add a site.cfg to keep a persistent list of paths

[Tarek Ziadé]

On Wed, Oct 20, 2010 at 11:57 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 19 October 2010 22:26, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
>> There's one feature I want to add in distutils2: the develop command
>> setuptools provides. Basically it adds a "link" file into
>> site-packages, and does some magic at startup to load the path that is
>> contained in the link file. The use case is to be able to have a
>> project added in the python path without installing it.
>
> Can you explain the requirement in more detail? I don't use the
> setuptools develop command, so I don't have the background, but it
> seems to me that what you're proposing can be done simply by adding
> the relevant directory to PYTHONPATH. That's all I ever do when
> developing (but my needs are pretty simple, so there may well be
> subtle problems with that approach).

Sorry that was vague indeed.

It goes a little bit farther than than: the project packages and
modules have to be found in the path, but we also need to publish the
project metadata that would be installed in a normal installation, so
our browsing/query APIs can find the project.

So, if a project 'Boo' has two packages 'foo' and 'bar' and a module
'baz.py', we need those in the path but also the Boo.dist-info
directory that is created at installation time (see PEP 376).
Setuptools' metadata directory is called Boo.egg-info, and distutils 1
has a file called Boo.egg-info since python 2.5

And since a python project can publish several top level directories,
all of them needs to be added in the path. so adding the current dir
to PYTHONPATH will not work in every case even if the metadata are
built and dropped there.

I am not sure what would be the best way to handle this, maybe having
these metadata built in place, then listing all the paths that need to
be included and write them to a .pth file Distutils2 manage.

So:

0. have a distutils2.pth file installed with distutils2

Then, to add the project in the path:

1. build the project metadata in-place
2. get the project paths by listing its packages and directories (by
invoking a pseudo-install command)
3. inject these paths in distutils2.pth

To remove it:

1. get the project paths by listing its packages and directories
2. remove these paths from distutils2.pth

Another problem I see is that any module or package that is not listed
by the project and that would not be installed in the site-packages
might be added in the path, but that's probably not a huge issue.

The goal is to be able to avoid re-installing a project you are
working on to try it, every time you make a change. This is used a
lot, and in particular with virtualenv.

So in any case, it turns out .pth files are a good way to do this so I
guess this thread does not belong to python-ideas anymore.

Cross-posting to the D2 Mailing list to move it there !

Tarek

-- 
Tarek Ziadé | http://ziade.org