On Mon, 27 Sep 1999, Gordon McMillan wrote:
M.-A. Lemburg wrote: [msg 1]
Currently, the imputil apporach uses a simple chaining technique. Unfortunately, it doesn't allow inspecting the chain for already loaded hooks, so the same type of hook could be loaded more than once.
I was hoping Greg would jump in, but since he hasn't -
I'm in a middle of a move back to CA. Unpacking now...
You're associating the hook with the strategy. That's the old style. The imputil style is to associate the hook with the actual stuff being managed. The strategy is a property of the hook.
Quite true. The chaining is simply an artifact of what has been installed as the import hook. I've always envisioned the potential for a "Importer Manager" that installs just like any other hook, but provides higher-level functions for importers to install themselves. The manager simply delegates the get_code() function to the sub-importers. Of course, the manager could use whatever technique to improve the speed of Importer selection. With respect to speed, I think the main point is to realize that the imputil technique is not inherently slow. It just depends on how you design your Importer subclasses -- do you install one or a hundred Importers? The imputil scheme is more about simplifying how people hook into the process (implement get_code() rather than a load/import combo). It also provides a simple capability (chaining) to allow *multiple* hooks to be installed.
Also, there are at least two types of hooks:
1. hooks that redirect the import to some other data source
2. hooks that modify the way modules are searched
Just one way -- your second is a variant of the first. "other data source" is a functional superset which includes searching. Importers don't simply alter searching -- they must perform the actual import (from wherever). This is the big change in mindset from the "ihooks" method -- find it and import it on the spot. The net effect is an Importer either imports a module or it doesn't (and the system can fallback to try another Importer). [ one the examples that people always like to specify was importing via URL which was actually quite difficult to use in the old scheme -- how do you separate an HTTP GET into a find/load step? Effectively, you had to double-fetch, or you had to place the whole module (which you retrieved during the find step) into your context for passing to the load. The other issue was the distinct semantics also implied that you could separate the functions -- I believe that to be quite unnecessary functionality. ]
Since the first variant may well also be suited to used by the second, the simple chaining method probably won't be powerful enough to handle it.
The top level question is "is it mine to import?". Greg provides a framework that makes it easy to use alternate data sources, and alternate ways of finding things but that's not really the key thing. You're a "good" importer if you can (when appropriate) way "no it's not mine" efficiently.
Very true!
[msg 2]
Another quirk that I think needs fixing:
When I issues an import:
import mx.DateTime
the whole import is handled by the importer installed at the start of the import. It is not possible to install a different importer e.g. in mx/__init__.py to handle the rest of the import (in this case the import of subpackage DateTime). I think that the importer should honor the __importer__ function (this is set by imputil) if present to let it continue the import of subsequent elements in the dotted name.
Sure you can. Your first importer is the "mx" importer. It has a dict of sub-importers. When mx/DateTime/__init__.py runs, it puts itself into that dict. The importer chain is now a tree.
Gordon's on top of it here... :-) Yes, it is simply a matter of perspective on the import process. An importer does not have to be a static entity. It also can be much more than a way to search a path... it can be highly dynamic and flexible. Whatever you like. Just implement get_code() to map a module "mx.DateTime" to a code/module object. There are a bazillion ways to do that :-)
This means, I think, that a "general" relative-path importer (ie, one that uses the default PYTHONPATH strategy), should be careful to install itself as the penultimate importer in the chain, (ie, the last before __builtin__.imp). But putting a relative-path search strategy into the "mx" importer is fine if it can quickly determine that the target is / is not a valid name in the "mx" namespace.
Part of the Importer work was done to satisfy importing modules from the COM+ namespace. I wanted to be able to say "import COM.foo.bar". The importer would handle all "COM." imports and delegate the "foo.bar" to the underlying Python/COM framework. In other words... yes, the Importer scheme should work *very* well for the "whatever...." type of module namespace. Cheers, -g -- Greg Stein, http://www.lyra.org/