[Python-Dev] Complexity of import.c

James C. Ahlstrom jim@interet.com
Mon, 09 Dec 2002 12:24:11 -0500


I see that my patch to import.c has been criticized
as adding to the complexity of an already complex module.
I don't entirely agree.  If you want to have the standard
library available as a zip module, you must have zip file
import available in C.  The patch I proposed in PEP 273
was a minimalist patch; it adds as little as possible
while still keeping package import and preserving all
import semantics.

IMHO, the real complexity of import.c comes not from
the double loop
   for path in sys.path:
      for suffix in (".pyc", ".pyo",...):
          fopen(...)
and not from the caching of zip archive names.  It comes
from the use of import.c to perform package imports.
Package imports are performed by creating recursive calls
to functions that would otherwise be flat, and by replacing
sys.path with the package path.  This is darned confusing.

At the time PEP 273 was discussed, I proposed moving package
imports out of import.c into a Python module such as Gordon's
iu.py or Greg's imputil.py.  This was rejected because package
users feared that package imports would be slowed down.  The
speed of Python startup and import was a concern.

I understand that people want to generalize zip imports, and
create a better import hook mechanism.  Some feel that
now is the time to do it.

This is my humble opinion:

Replacing import.c plus zip import with an equally complex
import.c is a fundamental mistake.  The current import.c
is understood (by few), and the zip file import patch adds
the minimum required.  If a different import.c is accepted,
then it should be MUCH simpler, and support only flat
directory searches and zip directories.  Package imports
and import hooks should be moved to a Python module such
as Gordon's iu.py.  And import.c must bootstrap iu.py and
all its imports from either a directory search or a zip file.

An improved import hook mechanism should have its own PEP.

I tried my best not to break the Macintosh port, which has
a lot of special code.  A replacement import.c should do
the same.

My patch modified many other Python .c and .py files to
solve several difficult bootstrap problems.  If you replace
just import.c you will get a painful lesson in bootstrapping.
You probably need these other patches even if you reject my
import.c patch.

Jim Ahlstrom