[Python-Dev] Write All New Import Hooks (PEP 302) in Python, Not C

[Just van Rossum]

Guido van Rossum wrote:

> Two new hooks is arguably still too much, but the existing __import__
> hook is inadequate because it requires you to reimplement too much
> functionality (the PEP argues this point convincingly IMO).  So we
> need at least one new hook.  Personally, I think sys.meta_path is the
> lesser important of the two hooks proposed by the PEP.  It would be
> needed if you have to override the standard builtin or frozen import
> behavior, but you can already do that with the heavier gun of
> overriding __import__.

The sys.meta_path feature adds 32 lines to import.c, it's *really*
useful and easy to use. I would be *extremely* sad to see it go.

(I had an immediate use for it in test_importhooks.py: after a test is
run I wanted to unload the modules that were imported during the test.
The ImportTracker is a meta importer and only *records* imports. It's
seven lines long and helped me clean up the test script quite a bit. Not
a typical use case perhaps, but to me still demonstrates the power of
meta_path quite well. Doing the same with __import__ is of course
possible, but is much more cumbersome and doesn't have the same
semantics.)

> > PEP 302 enables non-strings on sys.path, and adds two new
> > Python objects "importer" and "loader".  It changes the meaning
> > of imp.find_module() and imp.load_module(), and adds a new
> > imp.find_module2().  It changes the meaning of __import__.
> > It proposes to deprecate __path__ manipulations.
> 
> Yes, this is definitely too much. 

Removing the importer-object-on-sys.path feature I could live with. It's
clearly not needed, and for quick experimentation sys.meta_path is your
pal (another reason I don't want to see it go). It's only there because
it was easy: it's 17 lines in import.c.

> I'd like to limit this to
> implementing sys.path_hooks -- there should be only one way to do it.

There should be only one way to do it if import.c did it only one way.
Overriding or emulating builtin and frozen imports *must* be possible,
and sys.meta_path is the feature for it. A key aspect of sys.meta_path
is that it allows to refactor import.c in a really nice way; in the
future, sys.meta_path would look like this:

    [BuiltinImporter, FrozenImporter, PathImporter]

sys.meta_path is then the central import dispatcher. PathImporter would
be responsible for sys.path/pkg.__path__ handling and invoking
sys.path_hooks and dealing with sys.path_import_cache. It is *key* to
PEP 302, and to me simply a killer feature.

Just