[Distutils] PEP 376 - site-directories and site.addsitedir

[Tarek Ziadé]

On Thu, May 14, 2009 at 12:33 PM, Noah Gift <noah.gift at gmail.com> wrote:
> One thing that isn't clear to me is whether this could then lead to a
> potentially new third party packaging system, or library, that will just do
> the same thing again, and create a linear scanning algorithm.  Is there a
> way to architect things in a way that will never increase the number of
> stats to the filesystem or limit them?

I'd like to make a difference between what is "importable" and what is
"installed"
in Python.

what's "installed" for me is what is in a site-packages directory that
contains egg-info
directories.

At startup, you have to visit at least once every site-packages
directory, that's linear
and that won't go away.  But it's OK eventually, with a cache or a
memory index that
takes care of making the lookup code non-linear after Python has started
*as long as sys.path is not involved*

So, by restricting this API to site-packages (e.g. "installed"),
rather than sys.path (eg "importable"),
is just a way of limiting the scope to some specific places where you
are supposed to install
packages (and where egg-info files are located).

When a program is launched, the site-packages places should be limited to:

- the python site-packages (wether is central, wether it's local,
using virtualenv)
- the per user site-packages (PEP 370)
- maybe a third place added manually using site.addsitedir()


> One other thing that hopefully isn't off topic is the idea of scanning for
> imports in general.  Does it at all seem logical to have an option for
> python to have zero scanning to load a module?  After all if you really know
> the path to a module, why do you need to scan anything first to find it?

well if it's not loaded already, you have to find it don't you ? And
you don't know
where it is, you have to locate it  (see imporlib and pep 302)

But for me that's another story, than locating the metadata info of an installed
package.

Tarek