[Python-Dev] New relative import issue

[Phillip J. Eby]

At 12:40 PM 9/22/2006 +1200, Greg Ewing wrote:
>Guido van Rossum wrote:
>
> > While I agree with your idea(l), I don't think that's what Greg meant.
> > He clearly say "sys.path should not exist at all".
>
>Refining that a bit, I don't think there should be
>a *single* sys.path for the whole program -- more
>like each module having its own sys.path. And, at
>least in most cases, its contents should be set
>up from static information that exists outside the
>program, established when the module is installed.

Now I'm a little bit more in agreement with you, but not by much.  :)

In the Java world, the OSGi framework (which is a big inspiration for many 
aspects of setuptools) effectively has a sys.path per installed project 
(modulo certain issues for dynamic imports, which I'll get into more below).

And, when Bob Ippolito and I were first designing the egg runtime system, 
we considered using some hacks to do the same thing, such that you actually 
*could* have more than one version of a package present and being used by 
two other packages that require it.

However, we eventually tossed it as a YAGNI; dependency resolution is too 
damn hard already without also having to deal with crud like C extensions 
only being able to be loaded once, etc., in addition to the necessary 
import hackery.

So, in principle, it's a good idea.  In practice, I don't think it can be 
achieved on the CPython platform.  You need something like a Jython or 
IronPython or a PyPy+ctypes-ish platform, where all the C is effectively 
abstracted behind some barrier that prevents the module problem from 
occurring, and you could in principle load the modules in different 
interpreters (or "class loaders" in the Java context).

Amusingly, this is one of the few instances where every Python *except* 
CPython is probably in a better position to implement the feature...  :)


>One reason for this is the lack of any absolute
>notion of a "program". What is a program on one
>level can be part of a larger program on another
>level. For example, a module with test code that
>is run when it's invoked as a main script.
>Sometimes it's a program of its own, other times
>it's not. And it shouldn't *matter* whether it's
>a program or not when it comes to being able to
>find other modules that it needs to import. So
>using a piece of program-wide shared state for
>this seems wrong.

An interesting point about this is that it coincidentally solves the 
problem of dynamic interpretation of meta-syntactic features.  That is, if 
there is a static way to know what modules are accessible to the module, 
then the resolution of meta-syntax features like macros or custom 
statements is simpler than if a runtime resolution is required.

Of course, in all of this you're still ignoring the part where some modules 
may need to perform dynamic or "weak" imports.  So at least *some* portion 
of a module's import path *is* dependent on the notion of the "current 
program".

(See the documentation for the "Importing" package at 
http://peak.telecommunity.com/DevCenter/Importing for an explanation of 
"weak" importing.)