[Python-Dev] Alternative Approach to Relative Imports

[Greg Stein]

On Mon, 27 Sep 1999, Gordon McMillan wrote:
> M.-A. Lemburg wrote:
> [msg 1]
> > Currently, the imputil apporach uses a simple chaining
> > technique. Unfortunately, it doesn't allow inspecting the chain
> > for already loaded hooks, so the same type of hook could be
> > loaded more than once.
> 
> I was hoping Greg would jump in, but since he hasn't -

I'm in a middle of a move back to CA. Unpacking now...

> You're associating the hook with the strategy. That's the old 
> style. The imputil style is to associate the hook with the 
> actual stuff being managed. The strategy is a property of the 
> hook.

Quite true. The chaining is simply an artifact of what has been installed
as the import hook. I've always envisioned the potential for a "Importer
Manager" that installs just like any other hook, but provides higher-level
functions for importers to install themselves. The manager simply
delegates the get_code() function to the sub-importers. Of course, the
manager could use whatever technique to improve the speed of Importer
selection.

With respect to speed, I think the main point is to realize that the
imputil technique is not inherently slow. It just depends on how you
design your Importer subclasses -- do you install one or a hundred
Importers?

The imputil scheme is more about simplifying how people hook into the
process (implement get_code() rather than a load/import combo). It also
provides a simple capability (chaining) to allow *multiple* hooks to be
installed.

> > Also, there are at least two types of hooks:
> > 
> > 1. hooks that redirect the import to some other data source
> > 
> > 2. hooks that modify the way modules are searched

Just one way -- your second is a variant of the first. "other data source"
is a functional superset which includes searching. Importers don't simply
alter searching -- they must perform the actual import (from wherever).

This is the big change in mindset from the "ihooks" method -- find it and
import it on the spot. The net effect is an Importer either imports a
module or it doesn't (and the system can fallback to try another
Importer).

[ one the examples that people always like to specify was importing via
URL which was actually quite difficult to use in the old scheme -- how do
you separate an HTTP GET into a find/load step? Effectively, you had to
double-fetch, or you had to place the whole module (which you retrieved
during the find step) into your context for passing to the load. The other
issue was the distinct semantics also implied that you could separate the
functions -- I believe that to be quite unnecessary functionality. ]

> > Since the first variant may well also be suited to used by the
> > second, the simple chaining method probably won't be powerful
> > enough to handle it.
> 
> The top level question is "is it mine to import?". Greg provides 
> a framework that makes it easy to use alternate data sources, 
> and alternate ways of finding things but that's not really the 
> key thing. You're a "good" importer if you can (when 
> appropriate) way "no it's not mine" efficiently.

Very true!

> [msg 2]
> > Another quirk that I think needs fixing:
> > 
> > When I issues an import:
> > 
> >  import mx.DateTime
> > 
> > the whole import is handled by the importer installed at
> > the start of the import. It is not possible to install a
> > different importer e.g. in mx/__init__.py to handle the rest of
> > the import (in this case the import of subpackage DateTime). I
> > think that the importer should honor the __importer__ function
> > (this is set by imputil) if present to let it continue the import
> > of subsequent elements in the dotted name.
> 
> Sure you can. Your first importer is the "mx" importer. It has a 
> dict of sub-importers. When mx/DateTime/__init__.py runs, it 
> puts itself into that dict. The importer chain is now a tree.

Gordon's on top of it here... :-)  Yes, it is simply a matter of
perspective on the import process. An importer does not have to be a
static entity. It also can be much more than a way to search a path... it
can be highly dynamic and flexible. Whatever you like. Just implement
get_code() to map a module "mx.DateTime" to a code/module object. There
are a bazillion ways to do that :-)

> This means, I think, that a "general" relative-path importer (ie, 
> one that uses the default PYTHONPATH strategy), should be 
> careful to install itself as the penultimate importer in the chain, 
> (ie, the last before __builtin__.imp). But putting a relative-path 
> search strategy into the "mx" importer is fine if it can quickly 
> determine that the target is / is not a valid name in the "mx" 
> namespace.

Part of the Importer work was done to satisfy importing modules from the
COM+ namespace. I wanted to be able to say "import COM.foo.bar". The
importer would handle all "COM." imports and delegate the "foo.bar" to the
underlying Python/COM framework.

In other words... yes, the Importer scheme should work *very* well for
the "whatever...." type of module namespace.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/