[Python-Dev] Write All New Import Hooks (PEP 302) in Python, Not C

[James C. Ahlstrom]

Let's focus on PEP 302 as a new import hook mechanism, not
as a way to implement zip imports.  Just's PEP 302 code can
be used to implement zip imports without making his import
hooks part of the sys module and thus making them a public
feature of Python forever.  THIS IS NOT ABOUT ZIP IMPORTS!

Python already has an import hook, namely __import__.  PEP
302 adds three more hooks: sys.path_hooks, sys.meta_path and
sys.path_importer_hooks.  The idea of four import hooks is
already fishy.

PEP 302 enables non-strings on sys.path, and adds two new
Python objects "importer" and "loader".  It changes the meaning
of imp.find_module() and imp.load_module(), and adds a new
imp.find_module2().  It changes the meaning of __import__.
It proposes to deprecate __path__ manipulations.

That is a lot of external changes.  That is a lot of code
written in C.

I think the proper import hook design is to write Python's
import mechanism in Python along the lines of Greg's imputil.py
and Gordon's iu.py.  Import.c would be responsible for flat
non-package imports from directories and zip files, including
the import of the real importer iu.py.  The imp module would be
extended with simple C import utilities that can be used to
speed up iu.  Once iu.py is imported (probably from site),
all imports are satisfied using the iu module.

To provide custom import hooks, the user overrides the iu
module by writing Python code.  For example, site.py can
attempt an import of custom_iu before importing iu, and the
user provides custom_iu.py, probably by copying iu.py.
I am not sure of the best way to do the override, but I am
sure it is done in Python code. That enables the user to
create custom hooks in Python, while relying on the source
iu.py and the utilities in the imp module for basic
functionality.

If sys.path_hooks and the other hooks described in PEP 302
are approved, they can be implemented in iu.py in Python.

This design still requires C support for zip imports, but
there are two implementations for that available (from JimA
and Just).  Other problems such as bootstrapping are easily
solved.

I am tired of these endless import hook discussions, which
always seem to start from nifty ideas instead of from a
formal solicitation of hook requirements.  I don't object to
useful features, but I don't think anything other than easy
replacement of the Python import mechanism will bring this
perennial topic to an end.

JimA