[Python-ideas] Add a site.cfg to keep a persistent list of paths

[Paul Moore]

On 21 October 2010 00:35, Floris Bruynooghe <flub at devork.be> wrote:

> I've read your and Ian's responses and still don't understand what
> setup.py develop brings to the party which can't be done with simple
> PYTHONPATH.

I'm glad it's not just me!

> Again, apologies if I understand the problem wrongly.  But I too am
> worried about too many complexities and "magic".  One of my main
> issues with setuptools is that it tries to handle my python
> environment (sys.path) outside of normally expected python mechanisms
> by modifying various custom files.  I would hate to see distutils2
> repeat this.

I think that the key issue here is that PEP 376 introduces the idea of
a "distribution" which is a somewhat vaguely defined concept, which
can contain one or more packages or modules. Distributions don't have
a well-defined directory structure, and don't participate properly in
Python's standard import mechanism (PEP 302, PYTHONPATH, all that
stuff). The distribution metadata (dist-info directory) is not
package-based, and so doesn't fit the model.

Suggestions:

1. PEP 376 clearly defines what a "distribution" (installed or
otherwise) is, in terms of directory structure, whether/how it
supports PEP302-style non-filesystem access, etc. I don't see a reason
here why we can't mandate some structure, rather than leaving things
as a "free for all" like the current setuptools/adhoc approach.
2. Mechanisms for dealing with distributions are *only* discussed in
terms of the PEP 376 definitions, so we have a common understanding.

As a first cut, I'd say that a distribution is defined purely in terms
of its metadata (dist-info directory). On that basis, there should be
a definition of where dist-info directories are searched for, PEP 376
seems to state that this is only in site-packages ("This PEP proposes
an installation format inspired by one of the options in the
EggFormats standard, the one that uses a distinct directory located in
the site-packages directory."). And yet, this whole "develop"
discussion seems to be about locating dist-info directories located
elsewhere.

Having said that, PEP 376 later states:

get_distributions() -> iterator of Distribution instances.
Provides an iterator that looks for .dist-info directories in sys.path
and returns Distribution instances for each one of them.

This implies dist-info directories are searched for in sys.path. OK,
fine. That's broader than just site-packages, but still well-defined
and acceptable. And that's where I get my expectations that
manipulating PYTHONPATH should work.

So what's this directory structure we're talking about with Foo
containing two packages, and Foo.dist-info being alongside Foo? Foo
itself isn't on PYTHONPATH, so why should Foo.dist-info be found at
all? Based on PEP 376, it's not meant to be found.

Maybe if this *is* a requirement, it needs a change to PEP 376, which
I guess means the PEP discussion and approval process needs to be gone
through again. I, for one, would be OK with that, as I remain to be
convinced that the complexity and confusion is worth it.

Paul.